8000 Allow path from normal volume existing to overwrite in start Binds by cpuguy83 · Pull Request #10365 · moby/moby · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Allow path from normal volume existing to overwrite in start Binds #10365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

cpuguy83
Copy link
Member

Fixes #9981
Allows a volume which was created by docker (ie, in
/var/lib/docker/vfs/dir) to be used as a Bind argument via the container
start API and overwrite an existing volume.

For example:

docker create -v /foo --name one
docker create -v /foo --name two

This allows the volume from one to be passed into the container start
API as a bind to two, and it will overwrite it.

This was possible before 7107898

@cpuguy83 cpuguy83 force-pushed the 9981_fix_cannot_overwrite_nonbind_as_bind branch from 1d29a01 to ba215d4 Compare January 27, 2015 00:29
@cpuguy83 cpuguy83 added this to the 1.5.0 milestone Jan 27, 2015
@cpuguy83 cpuguy83 changed the title Allow normal volume to overwrite in start Binds Allow path from normal volume existing to overwrite in start Binds Jan 27, 2015
@@ -80,7 +81,7 @@ func (m *Mount) initialize() error {
if hostPath, exists := m.container.Volumes[m.MountToPath]; exists {
// If this is a bind-mount/volumes-from, maybe it was passed in at start instead of create
// We need to make sure bind-mounts/volumes-from passed on start can override existing ones.
if !m.volume.IsBindMount && m.from == nil {
if (!m.volume.IsBindMount && !m.isBind) && m.from == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference between IsBindMount and isBind? why would m.from not be nil if it's a bind mount?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of the problem why this needs to be refactored cause this looks weird but....

We have m.volume.IsBindMount, which is set if the volume itself is a bind-mount, which in the tested case it is not since it was created as a normal volume.

m.IsBind is (now) set when it is created during parsing HostConfig.Binds.

< 8000 /div>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need another bool when m.from should be nil if it's a bind mount?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the issue this is fixing isn't with volumes-from, it's doing something like docker run -v /var/lib/docker/vfs/dir/55a6f3a06952bf1c6f8a93a85c55493107a6cbe8191a8d9e83ff75fad705e268:/foo ...

@cpuguy83
Copy link
Member Author

To clear up some confusion:

10:04 cpuguy83: icecrime: Yeah, I can see the confusion... Probably easiest to checkout the test in there, but basically this...
10:05 cpuguy83: icecrime: Some people are wanting to take the host path of a normal volume (ie, /var/lib/docker/vfs/dir/) and use that as the host path of a bind-mount in another container.
10:07 cpuguy83: icecrime: Which would be ok too, however in these cases there is often already a volume specified for the dir in the container
10:07 cpuguy83: icecrime: for example VOLUME /foo, and then they want to docker run -v /var/lib/docker/vfs/dir/<id>:/foo
10:09 cpuguy83: icecrime: since Docker goes through the whole "createVolumes" thing on both start and create, it also has to check each volume if it needs to be re-initialized or not (ie, was a new one passed in via volumes-from or Binds)
10:10 cpuguy83: icecrime: The volumes subsystem is storing, by host path, if a volume is a Bind volume or a normal one (handled by the vfs graph driver)
10:11 cpuguy83: icecrime: So currently docker is looking up that path people are wanting in their bind /var/lib/docker/vfs/dir/<id> and it says, "oh nope, that's not a bind" and skips it, since it thinks it's just a normal volume, and as such should never need to be re-initalized
10:12 cpuguy83: icecrime: So, one of the things I've done separately is re-factor this whole mess so that's a bit more obvious, but the fix in gh10365 is just a minimal fix to get it working for this situation.

@icecrime
Copy link
Contributor

Thanks @cpuguy83, this helps a lot! So if I understand correctly: bind-mounts and normal volumes are essentially the same behavior, what matters to Docker is keeping track of directories it created itself because it's responsible for their disposal.

@icecrime
Copy link
Contributor

One extra question: do we absolutely need this in 1.5.0 (i.e., is it a regression from 1.4.1)? If we don't, should we wait for the ongoing refactor?

@thaJeztah
Copy link
Member

what matters to Docker is keeping track of directories it created itself because it's responsible for their disposal.

(this caught my eye); @cpuguy83 would that mean that with this change, docker will see volumes "copied" during the Fig/Compose re-creation step to be a "bind mount", thus docker rm -v container would not cleanup the volume?

TBH I don't know how Fig/Docker handles this currently, just wondering, because that would make it harder to cleanup volumes.

(I think this PR was because of a regression when using Fig/Compose with Docker 1.5)

@LK4D4
Copy link
Contributor
LK4D4 commented Jan 28, 2015

@cpuguy83 I really don't understand :( Can you explain this magic about isBind: true?

@crosbymichael crosbymichael modified the milestones: 1.5.1, 1.5.0 Jan 28, 2015
@cpuguy83
Copy link
Member Author

@icecrime This is a regression that occurred in 1.4 when volumes started being created at the same time the container is created (as opposed to just when the container was started).

@LK4D4 "volumes" are really two things, mount points and data. The stuff in daemon/volumes is really dealing with mount points. isBind is saying that the mount is a bind (even though the volume is not). On the /volumes side, IsBindMount is unfortunately named at this point, but is really saying that it's using the host fs as opposed to the vfs graph driver fs.

@LK4D4
Copy link
Contributor
LK4D4 commented Jan 28, 2015

I got idea, but some day we need to refactor volumes code :)
LGTM

@crosbymichael
Copy link
Contributor

@LK4D4 @cpuguy83

I need to prioritize for the 1.6 timeframe to refactor this code because having this many issues and churn in the same files is a huge sign that there are serious problems.

@cpuguy83
Copy link
Member Author

@crosbymichael Agreed, much of this is already done here: https://github.com/cpuguy83/docker/tree/cleanup_daemon_volumes

@tiborvass
Copy link
Contributor

@cpuguy83 rebase please! :)

Fixes moby#9981
Allows a volume which was created by docker (ie, in
/var/lib/docker/vfs/dir) to be used as a Bind argument via the container
start API and overwrite an existing volume.

For example:

```bash
docker create -v /foo --name one
docker create -v /foo --name two
```

This allows the volume from `one` to be passed into the container start
API as a bind to `two`, and it will overwrite it.

This was possible before 7107898

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
@cpuguy83 cpuguy83 force-pushed the 9981_fix_cannot_overwrite_nonbind_as_bind branch from ba215d4 to 49e1ad4 Compare February 11, 2015 01:46
@cpuguy83
Copy link
Member Author

rebased

@cpuguy83
Copy link
Member Author

ping

@cpuguy83
Copy link
Member Author
cpuguy83 commented Mar 4, 2015

That cleanup is here: #10422
I think we can close this one in favor of that in the absence of a 1.5.1 release.

@icecrime
Copy link
Contributor
icecrime commented Mar 9, 2015

Considering that 1.6.0 will be ~March 22nd, it's highly unlikely we do a 1.5.1. I'm closing this but keeping the milestone in case it happens for some reason. Thanks!

@icecrime icecrime closed this Mar 9, 2015
@dmp42 dmp42 reopened this Mar 9, 2015
@icecrime
Copy link
Contributor
icecrime commented Mar 9, 2015

@dmp42 You really have to unauth waffle.io :)

@icecrime icecrime closed this Mar 9, 2015
@jessfraz jessfraz removed this from the 1.5.1 milestone Mar 16, 2015
@jessfraz jessfraz modified the milestones: 1.6.0, 1.5.1 Mar 16, 2015
@jessfraz jessfraz reopened this Mar 23, 2015
@jessfraz
Copy link
Contributor

LGTM

jessfraz pushed a commit that referenced this pull request Mar 23, 2015
…bind_as_bind

Allow path from normal volume existing to overwrite in start Binds
@jessfraz jessfraz merged commit 1fe55b2 into moby:master Mar 23, 2015
@cpuguy83 cpuguy83 deleted the 9981_fix_cannot_overwrite_nonbind_as_bind branch September 20, 2017 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mounting a host folder is broken?
8 participants
0