8000 Multiline Dockerfile syntax · Issue #1799 · moby/moby · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Multiline Dockerfile syntax #1799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mzdaniel opened this issue Sep 4, 2013 · 25 comments
Closed

Multiline Dockerfile syntax #1799

mzdaniel opened this issue Sep 4, 2013 · 25 comments
Labels
area/builder impact/dockerfile kind/enhancement Enhancements are not bugs or new features but can improve usability or performance.

Comments

@mzdaniel
Copy link
Contributor
mzdaniel commented Sep 4, 2013

Dockerfile is a powerful building tool.
It will be more visually appealing adding multiline capability to it.
As example:

run /bin/echo -e '#!/bin/bash\necho This is a long
shell line; echo Next line

To detect the multiline we probably want to use a regex like '[ \t]*\n'

@mzdaniel
Copy link
Contributor Author
mzdaniel commented Sep 5, 2013

/cc @creack

@vieux
Copy link
Contributor
vieux commented Sep 11, 2013

Fixed by #1838

@from-nibly
Copy link

+1 on that. running commands against a postgres server requires a bunch of &, && and ; and makes a really cluttered looking RUN command

@d11wtq
Copy link
d11wtq commented Mar 13, 2014

Anyone know how to use heredoc with this syntax? Docker wants to bunch it onto a single line and causes a bash syntax error:

RUN cat > /etc/example.conf <<'EOF' \
This is one multiple lines \
and should stay that way \
EOF

@crosbymichael
Copy link
Contributor

@d11wtq Why not just add the file instead of catting in the dockerfile?

@d11wtq
Copy link
d11wtq commented Mar 13, 2014

@crosbymichael because (and correct me if I'm wrong), if I edit the file (and I will), Docker won't use my edits, due to the way it caches. I could completely disable caching, but this image includes the entire llvm toolchain and is very big as it is, so I do want caching of the commands, I just need ADD to be aware of changes in the files it is adding. Maybe it already works that way, but IIRC it does not.

@crosbymichael
Copy link
Contributor

Yes, it will use your edits. It does a checksum on the content to pick up chagnes.

@d11wtq
Copy link
d11wtq commented Mar 13, 2014

@crosbymichael Ah, that is excellent. Is that new? I could have sworn I'd been bitten by this in the past. That does indeed solve my issue then :)

@crosbymichael
Copy link
Contributor

Yes as of 0.8 maybe

@DevElCuy
Copy link

Try this one:

RUN echo 'All of your\n\
multiline that you ever wanted\n\
into a dockerfile\n'\
>> /etc/example.conf

@thiagokronig
Copy link

This works for me:

RUN echo $'All of your\n\
multiline that you ever wanted\n\
into a dockerfile\n'\
>> /etc/example.conf

@RichardJECooke
Copy link

And if your string contains quotes you can do this:

RUN echo $'All of your\n\
multiline'"'"'s that you ever wanted\n\
> /etc/example.conf

@VelorumS
Copy link

Fixed by #1838

No, it's not.

@searene
Copy link
searene commented Mar 4, 2018

@DevElCuy @thiagokronig What if my command contains quotes, like the following one

RUN echo 'a\n\
'b'\n\
c\n'\
>> /test

It doesn't work.

@lattice0
Copy link
lattice0 commented Mar 5, 2018

Would be awesome if possible to do:

RUN echo $'All of your\n
multiline that you ever wanted\n
into a dockerfile\n'
>> /etc/example.conf

That is, identation in the dockerfile without getting tabs in the example.conf

@thaJeztah
Copy link
Member

See #34423

@lonix1
Copy link
lonix1 commented Sep 27, 2018

I want to write a small script, so I tried:

RUN echo '\n\
  #!/bin/bash\n\
  foo\n\
  bar\n'\
  > /usr/local/bin/myscript.sh
RUN chmod +x /usr/local/bin/myscript.sh

But the first line is considered to be a comment, and so is ignored. Can it be escaped somehow?

@aarzamasov
Copy link

I want to write a small script, so I tried:

RUN echo '\n\
  #!/bin/bash\n\
  foo\n\
  bar\n'\
  > /usr/local/bin/myscript.sh
RUN chmod +x /usr/local/bin/myscript.sh

But the first line is considered to be a comment, and so is ignored. Can it be escaped somehow?

RUN echo  $'\n\
  #!/bin/bash\n\
  foo\n\
  bar\n'\
  > /usr/local/bin/myscript.sh
RUN chmod +x /usr/local/bin/myscript.sh

Hi, try ti add $ after the echo.

@jen-soft
Copy link
jen-soft commented May 3, 2019

It's help for me ( https://github.com/jen-soft/pydocker )

[ Dockerfile.py ]

from pydocker import DockerFile  # sudo pip install -U pydocker

d = DockerFile(base_img='debian:8.2', name='jen-soft/custom-debian:8.2')

d.RUN_bash_script('/opt/set_repo.sh', r'''
cat >/etc/apt/sources.list <<EOL
deb     http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main
EOL
apt-get clean && apt-get update
''')

d.EXPOSE = 80
d.WORKDIR = '/opt'
d.CMD = ["python", "--version"]

# d.generate_files()
d.build_img()
# sudo wget -qO- https://get.docker.com/ | sh

python Dockerfile.py
docker images

@JamshadAhmad
Copy link

I have tried several different varients of that echo command, none of them worked for me.
So I end up creating a file and copying it into container like
COPY ./uploads.ini /etc/example.ini

@comrumino
Copy link
comrumino commented Feb 11, 2020

Fanatics of printf may prefer a little something like this

RUN printf ''\
'#!/usr/bin/env bash\n'\
"printf 'qrcode command: qrencode -o user.png otpauth://totp/%s@%s?secret=%s'\n"\
'vsftpd /etc/vsftpd.conf'\
 "${ftpuser?}" "${pasv_address?}" "${oathuser_b32seed?}" > "/usr/bin/vsftpd-docker-init"

when it makes sense of course.
Caveat: be mindful of quotes and whitespace

@thondeboer
Copy link

I tried a couple of things with the echo command and could not get it to work, until I did this:

RUN echo  '#!/bin/bash\n\
export GRANT_SUDO=no\n\
export HOME=''"''/data''"''\n\
start-notebook.sh \n\
--NotebookApp.base_url=/${BGP_NAMESPACE}-n-${BGP_NOTEBOOK}-i${BGP_NODEINSTANCE} \n\
--NotebookApp.token=${BB_TOKEN} \n\
  --NotebookApp.allow_password_change=False \n\
  --NotebookApp.tornado_settings=''"''{'"'"'headers'"'"': { '"'"'Content-Security-Policy'"'"': '"'"'frame-ancestors *;'"'"' } }''"'' \n\
  --ip=0.0.0.0 \n\
  --config=/config/config.json \n\
  --Session.username=bluebee \n\
  --LabApp.user_settings_dir=/data/.jupyter/lab/user-settings \n\
  --LabApp.workspaces_dir=/data/.jupyter/lab/workspaces \n\
'> /usr/local/bin/bb_script.sh
RUN chmod +x /usr/local/bin/bb_script.sh

It wil then show up as this: (Note the quotes)

#!/bin/bash
export GRANT_SUDO=no
export HOME="/data"
start-notebook.sh
--NotebookApp.base_url=/${BGP_NAMESPACE}-n-${BGP_NOTEBOOK}-i${BGP_NODEINSTANCE}
--NotebookApp.token=${BB_TOKEN}
  --NotebookApp.allow_password_change=False
  --NotebookApp.tornado_settings="{'headers': { 'Content-Security-Policy': 'frame-ancestors *;' } }"
  --ip=0.0.0.0
  --config=/config/config.json
  --Session.username=bluebee
  --LabApp.user_settings_dir=/data/.jupyter/lab/user-settings
  --LabApp.workspaces_dir=/data/.jupyter/lab/workspaces

Note I did NOT use the echo $' trick, since that just inserted a $ in my code.
I could only get it to work to use the #!/bin/bash right in the same line as the echo, otherwise lines with # will still be considered comments, but this is a functioning script.

@RichardBronosky
Copy link

How is it that thousands of engineers waste millions of hours on this horrible UX decision in one of the most important Free/Libre Open Source Software projects? If Docker™ are not going to fix this, then we need to just create a Dockerfile preprocessor that converts:

RUN <<ANY_DELIMETER
...
ANY_DELIMETER

or even

BEGINRUN
...
ENDRUN

into the unintelligible format that Docker™ finds superior. @jen-soft has proven this is possible.

@thaJeztah
Copy link
Member

This issue is not the right place for that discussion. This is a 7 year old enhancement request, asking for RUN lines to be wrapped over multiple lines, which was not possible before. That enhancement was addressed by #1838, which is why this issue was closed.

How is it that thousands of engineers waste millions of hours on this horrible UX decision in one of the most important Free/Libre Open Source Software projects? If Docker™ are not going to fix this

It's ironic that you mention "free/libre" and "open source", but then expect a company to do the work. This project is open source, especially so that people can participate and contribute fixes, enhancements, and features. While this project was started by "Docker™", it is driven by its maintainers, who maintain the project, and are responsible for making decisions on features and enhancements. While Docker™ pays some of their engineers to work on maintaining this project and developing features, the majority of the maintainers is not even employed by Docker.

into the unintelligible format that Docker™ finds superior.

No-one mentions it's superior; support for newlines (using \ as a line-continuation symbol to allow you to wrap lines) fixed the issue at hand. While adding support for wrapping lines was sufficient at the time, things evolved since, and people started creating more complex Dockerfiles, which asks for more flexible/advanced options.

Commenting on a 7 year old closed issue is not the right channel for proposing changes; to propose changes, open a new issue with a proposal. Here's a proposal that I wrote (based on some other proposals) to add support for a here-doc-style syntax: #34423. While (I think) maintainers agree that would be a good addition to the syntax, no work was done on it yet (there's a finite amount of engineering time, and other changes may have priority). That said; this project is open source, and I welcome people to participate on that proposal, or perhaps work on the implementation.

then we need to just create a Dockerfile preprocessor that converts:

Since this ticket was opened, the "builder" has been rewritten from scratch, and the next-generation builder (BuildKit https://github.com/moby/buildkit) was developed. BuildKit can both be used in Docker (currently opt-in by setting DOCKER_BUILDKIT=1, but it will become the default in a future release: #40379), or run standalone.

BuildKit allows adding alternative syntaxes ("front-ends") to extend the Dockerfile syntax. For example, the experimental Dockerfile syntax that is used to test / "incubate" new features in the Dockerfile syntax is distributed as a Docker image, but you could even implement your own syntax that's wholly different as Dockerfiles 9E88 . For example, if you want to build your images using a YAML file: here's a blogpost about someone creating a format called "Mockerfile"), or even build using Buildpacks.

Distributing those alternative syntaxes/front-ends as images provides a mechanism to implement them without making code changes in Docker itself (they can be used on Docker 18.09 and higher when enabling BuildKit, or even older versions when using buildx) - see the "experimental" Dockerfile syntax mentioned earlier.

The proposed here-doc syntax could be implemented as a custom front-end, and very likely would be accepted for inclusion in the experimental Dockerfile syntax (before promoting it to the "stable" syntax). Contributions for that are welcome if someone wants to work on that.

@thaJeztah thaJeztah added area/builder impact/dockerfile kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. labels Feb 27, 2020
@Bessonov
Copy link

@thaJeztah thank you very much for your great answer!

trebonian pushed a commit to trebonian/docker that referenced this issue Jun 3, 2021
Fixing issue with bit allocation byteoffset calculation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/builder impact/dockerfile kind/enhancement Enhancements are not bugs or new features but can improve usability or performance.
Projects
None yet
Development

No branches or pull requests

0