container error after upgrade to containerd.io 1.4.6-1 #5547
Comments
|
I also have problems after upgrading from 1.4.4-1 to 1.4.6-1. I get the following error:
Downgrading to 1.4.4-1 also helped me immediately. |
|
Same here, needed to downgrade from 1.4.6 to 1.4.4. |
|
Same thing! Had to downgrade. It fails even for a simple Dockerfile, such as: FROM ubuntu:18.04
RUN echo asddocker build -t asd .
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM ubuntu:18.04
---> 81bcf752ac3d
Step 2/2 : RUN echo asd
---> Running in 71fc8a0abb62
OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "proc" to rootfs at "/proc" caused: mount through procfd: no such file or directory: unknownOn Ubuntu 18:04 |
|
I was getting:
Downgrading as pointed out by @KeyserSoeze worked. |
|
@KeyserSoeze not sure what happened in your case; do you happen to be using the "live restore" option on Docker? I tried reproducing by:
wget -q -O- "https://download.docker.com/linux/debian/gpg" | apt-key add -qq -
echo "deb [arch=amd64] https://download.docker.com/linux/debian buster stable" > /etc/apt/sources.list.d/docker.list
# install docker 20.10.6 and containerd v1.4.4 + runc v1.0.0-rc93
apt-get update && apt-get install -y -qq --no-install-recommends \
docker-ce-cli=5:20.10.6~3-0~debian-buster \
docker-ce=5:20.10.6~3-0~debian-buster \
containerd.io=1.4.4-1
# run some containers
docker run -d --name web nginx:alpine
docker run -it ubuntu:18.04
# upgrade containerd
apt-get install -y containerd.io=1.4.6-1
# try starting the "web" container
docker start web
# try running an ubuntu container
docker run -it --rm ubuntuDoes upgrading docker to 20.10.7 solve your issue? w.r.t. #5547 (comment) and #5547 (comment) That looks related to a breaking change in runc (see opencontainers/runc#2928). We applied changes in Docker 20.10.7 to work around that change (see moby/moby#42370), as well as changes in BuildKit (moby/buildkit#2123) but it's possible that (e.g. in situations where you're running a swarm service which was already created) that this is causing issues. I'm not sure if the runc maintainers want to revert that change for v1.0.0 (but I can try) @goodspark I tried reproducing your issue (which looks to be separate from the above), but wasn't able to: what distro are you running on? (perhaps you can edit your comment and add the output of |
|
I'm having this same issue with Github workflows using (I have already tried downgrading to This is the error output: The error is reproducible by running the following command in a GitHub workflow: # Create configuration files
mkdir -p /tmp/config/
cat > /tmp/config/salt-api.conf <<EOF
external_auth:
pam:
salt_api:
- .*
- '@runner'
- '@wheel'
- '@jobs'
EOF
# Run test instance
docker run --rm --detach --name saltstack_master \
--publish 4505:4505 --publish 4506:4506 --publish 8000:8000 \
--env 'SALT_API_SERVICE_ENABLED=true' \
--env 'SALT_API_USER_PASS=4wesome-Pass0rd' \
--platform linux/amd64 \
--volume /tmp/config:/home/salt/data/config:ro \
cdalvaro/docker-salt-master:latest
# Wait for salt-master bootup
sleep 60 |
|
Hm... in your case, I wonder if you made a typo somewhere, or if some script produced an incorrect output; Seems to indicate you're attempting to mount to |
Thank you for your reply, @thaJeztah! Initially, that was my first thought. But then I realized that these same settings were running well until a dependabot PR where checks began to fail. I haven't approved that PR yet, but checks are failing now in the |
|
Yeah, it's definitely odd. The "source" of that mount ( VOLUME ["/some/path/", "/some-other/path/", "/yet/another/path/"]If that JSON was invalid (e.g., using single quotes instead of double quotes, or a missing comma) during building the image, Docker will fallback to treating that JSON as a literal string (which would explain the error message). Are you able to do a |
|
I ran into the same mounting error after upgrading from 1.4.4-1 to 1.4.6-1. As the quotation marks are missing, this is an incorrect declaration of a JSON array. Changing it to the following (and removing stopped containers beofre running build) solved it: |
|
You are my hero @thaJeztah 🎉 By doing "Volumes": {
"/home/salt/data/3pfs": {},
"/home/salt/data/config": {},
"/home/salt/data/keys": {},
"/home/salt/data/logs": {},
"/home/salt/data/srv": {},
"[": {},
"]": {}
},I had a multiple directories Thank you again for your help! |
|
Looks like this is really a common problem with the mount volume while using docker image by nginx $ I have have similar issue with OCI runtime create failed: invalid mount while mounting my mount file. I observed a new folder was created which is totally different from my current directory where my html files are..This is absolutely strange to me. the spa was what i specified in my code while spa;C kept popping up after i run my command. any help would be greatly appreciated Docker version 20.10.7, build f0df350 |
|
Okay so i figured out a workaround. I changed my terminal and stopped using bash script in interi,m. So i tried powershell instead because of windows os. However, something changed. in my code, () now {} |
|
Downgrading worked for me, too. It just wouldn't budge. Why did it work the first time, though? |
Hard to tell. If you have a link to the script, perhaps that provides more info as to what the script is doing (and why it failed) |
|
The entire script consists of setting bash variables to passwords and paths for a So, there is absolutely nothing else in the script, besides the action of running a single Docker container. |
|
I had the same issue services:
frontend:
build:
context: ./frontend
dockerfile: ./Docker/Dockerfile
image: react_frontend_service
container_name: frontend
restart: always
command: npm start
volumes:
- ./frontend/:/usr/src/app/ # it wants this
- usr/src/app/ # it does not want this
ports:
- "8000:3000"
env_file:
- frontend/environments/.develpment.env
depends_on:
- postgres_database |
That line only has a single path, in which case it's considered the path inside the container to create a volume for. When using the shorthand ( However, it looks like you specify two volumes/mounts for the same container path;
I think this could potentially lead to a race-condition, where (depending on which mount gets mounted first), the bind-mount masking the content of the anonymous volume, or vice-versa. Was is intentional to have to mounts? I have a feeling only the first one is needed? |
|
I've run into a similar problem with the latest version. In my Dockerfile I have the following When I try to start a container using that image, I get the following error: Note that it appears to be grabbing the first word of the comment "My" as part of the volume definition. This was not the case before the update. Moving the comment above the VOLUME line as depicted below resolved the error, but requiring such formatting contradicts my understanding of the documentation regarding Dockerfile comments https://docs.docker.com/engine/reference/builder/#format. |
|
Yes, The Dockerfile syntax only supports comments on their own line, and does not support trailing comments.
In case of VOLUME /one /two /threeSo what happens if you add the trailing comment; VOLUME /path/to/dir # My CommentIs that it defines four volumes: You can see this when building an image; FROM busybox
VOLUME /path/to/dir # My CommentThen, inspecting the image, you can see the 4 volumes in the configuration: DOCKER_BUILDKIT=1 docker build -t myimage .
docker image inspect --format '{{json .Config.Volumes}}' myimage
{"#":{},"/path/to/dir":{},"Comment":{},"My":{}} |
|
Just to add to the fray, the update also affected me. I don't doubt that the error is on my part, but so far I just haven't had the time to spend more time than I already have troubleshooting this particular minor app. It worked fine until one day my dev server crashed with about 10 apps on it (unattended upgrades had updated containderd.io which broke the app and caused nginx to panic since it could no longer find the crashed app) but it sounds like the upgrade isn't to blame... just that it now enforces proper syntax, which should have probably failed before. So until I have some free time, I'll probably keep the last "working" version pinned. |
|
This issue should be fixed by opencontainers/runc#3004, which relaxed the validation in runc, and is part of runc v1.0.0, which is included in containerd v1.5.3 (and up) and containerd v1.4.7 (and up). |
|
Building this Dockerfile on the latest Docker 3.5.2 mac Os also shows the problem. Is this fixed on mac ? build command
result |
|
Upcoming patch release of Docker Desktop will include runc v1.0.0 But in your case looks like there's a mistake in your Dockerfile; VOLUME [ /usr/src ]Is not valid JSON, so will create 3 volumes: Change it to: VOLUME ["/usr/src"](with quotes) or VOLUME /usr/src |
|
Tested and it works. Thanks for your explanation. |
Since containerd.io upgrade 1.4.6 it seems the VOLUME syntax needs to be valid JSON. Thanks to containerd/containerd#5547 (comment) I pinned point some existing errors when using this image. 🙇♂️Thanks @thaJeztah for analysing that issue!
Hi @theAkito, did you figure out why it worked at the first time. I also faced the same issue. I could run a container without any issue at the first time. After a while I tried to restart the container, then it showed the |
|
I'm sorry, didn't figure it out and I doubt I will be able to. |
Based on this changes for containerd.io - containerd/containerd#5547 (comment) - adding quotes to VOLUME definition ;)
Since containerd.io upgrade 1.4.6 it seems the VOLUME syntax needs to be valid JSON. Thanks to containerd/containerd#5547 (comment) I pinned point some existing errors when using this image. 🙇♂️Thanks @thaJeztah for analysing that issue!
System: docker-ce version 20.10.6 on Debian buster 10.9
Bug: After last update of conainerd.io from version 1.4.4-1 to 1.4.6-1 docker container immediately stop working.
Error messages from docker status:
Restarting docker did not solved the problem.
Workaround: Downgrade containerd.io to previous version (
sudo apt-get install containerd.io:amd64=1.4.4-1) and container immediately worked again.The text was updated successfully, but these errors were encountered: