10000 Non-working Buildkit GC policies · Issue #40086 · moby/moby · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Non-working Buildkit GC policies #40086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
johnbizokk opened this issue Oct 13, 2019 · 4 comments
Closed

Non-working Buildkit GC policies #40086

johnbizokk opened this issue Oct 13, 2019 · 4 comments
Assignees
@tonistiigi @tiborvass
Labels
area/builder version/19.03

Comments

@johnbizokk
Copy link

Description

Suddenly, we cannot use the builder garbage collector policy as described in the PR #37846 and explained by the Docker Captain (@sudo-bmitch) report in the latest DockerCon - Tips and Tricks of the Docker Captains. Also, I am frustrating why so many cool features do not have related documentation. On the evaluation cycle, the build cache garbage collector disregards the defined policies and drops the cache to the lowest value of the keepStorage key in the set of the rules.

Steps to reproduce the issue:

  1. Run the docker system df command to check the currect disk usage of the build cache stored localy
  2. Create the docker daemon config file with the following content - see the Annex №1 below
  3. Run the build of the whatever docker image you have to produce the build cache and evaluate the garbage collection (greater than the smallest size defined in the policies)
  4. Run the docker system df command to check the currect disk usage of the build cache stored localy
  5. Reload the docker daemon
  6. Run the build of the whatever docker image you have to produce the build cache and evaluate the garbage collection (at least - 300mb) - Optional
  7. Run the docker system df command to check the currect disk usage of the build cache stored localy

Describe the results you received:

At the end of manipulations, which described above - I lost my build cache and the size of the remained cache equal to the smallest keepStorage policy I have defined in the docker daemon config.

Describe the results you expected:

I am expecting the garbage collector to respect and follow the policy defined in the docker daemon config.

Additional information you deem important (e.g. issue happens only occasionally):

According to the current garbage collector behavior, we forced to use only one policy that limits the maximum size of the build cache, which remains on the disk.

Output of docker version:


Client: Docker Engine - Community
 Version:           19.03.3
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        a872fc2f86
 Built:             Tue Oct  8 00:58:10 2019
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.3
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.10
  Git commit:       a872fc2f86
  Built:            Tue Oct  8 00:56:46 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:


Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.1-tp-docker)

Server:
 Containers: XX
  Running: XX
  Paused: XX
  Stopped: XX
 Images: XX
 Server Version: 19.03.3
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: local
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: ID
  Is Manager: true
  ClusterID: ID
  Managers: X
  Nodes: X
  Default Address Pool: XX.XX.XX.XX/X
  SubnetSize: 24
  Data Path Port: XXXX
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: XX.XX.XX.XX
  Manager Addresses:
   XX.XX.XX.XX:XXXX
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.3.6-1.el7.elrepo.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: XX
 Total Memory: XX.XGiB
 Name: NAME
 ID: ID
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
  processor-cores=XX
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: bridge-nf-call-ip6tables is disabled

Additional environment details (AWS, VirtualBox, physical, etc.):

Annex №1 (daemon.json)


{
  "experimental": true,
  "features": {
    "buildkit": true
  },
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "200GB",
      "policy": [
        {"keepStorage": "256MB", "filter": {"unused-for": {"960h": true}}},
        {"keepStorage": "1GB", "filter": {"unused-for": {"480h": true}}},
        {"keepStorage": "2GB", "filter": {"unused-for": {"360h": true}}},
        {"keepStorage": "5GB", "filter": {"unused-for": {"240h": true}}},
        {"keepStorage": "10GB", "filter": {"unused-for": {"192h": true}}},
        {"keepStorage": "20GB", "filter": {"unused-for": {"144h": true}}},
        {"keepStorage": "40GB", "filter": {"unused-for": {"120h": true}}},
        {"keepStorage": "60GB", "filter": {"unused-for": {"96h": true}}},
        {"keepStorage": "80GB", "filter": {"unused-for": {"48h": true}}},
        {"keepStorage": "100GB", "filter": {"unused-for": {"1m": true}}},
        {"keepStorage": "200GB", "all":  true}
      ]
    }
  }
}
@thaJeztah
Copy link
Member

ping @tonistiigi @tiborvass @AkihiroSuda PTAL

@thaJeztah
Copy link
Member

ping @tonistiigi @tiborvass could one of you have a look at this?

@thaJeztah
Copy link
Member
thaJeztah commented Aug 13, 2020 •
edited
Loading

@johnbizokk I see you also opened commented on #39601 at the time, is this related? (a fix for that was merged in #39979)

@tiborvass
Copy link
Contributor

I believe this was indeed fixed in #39979. Please comment if not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tonistiigi tonistiigi

@tiborvass tiborvass

Labels
area/builder version/19.03
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@tonistiigi @tiborvass @thaJeztah @johnbizokk
0