8000 Proxy does not follow re-direct · Issue #4545 · distribution/distribution · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Proxy does not follow re-direct #4545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ChandonPierre opened this issue Dec 31, 2024 · 5 comments
Open

Proxy does not follow re-direct #4545

ChandonPierre opened this issue Dec 31, 2024 · 5 comments

Comments

@ChandonPierre
Copy link
ChandonPierre commented Dec 31, 2024 •
edited
Loading

Description

Working with an upstream registry that returns 307 when pulling layer blobs, registry returns BLOB_UNKNOWN: blob unknown to registry

I suspect registry is not following the re-direct, and is attempting instead to pull from the configured remoteurl as the base url.

An example curl with returned location header (adding -L successfully pulls the tarball)

curl https://docker.cloudsmith.io/v2/org/repo/app/blobs/sha256:sha  --http1.1  -v
* Host docker.cloudsmith.io:443 was resolved.
* IPv6: 2600:9000:2512:2600:4:c7b4:fd40:93a1, 2600:9000:2512:4800:4:c7b4:fd40:93a1, 2600:9000:2512:1200:4:c7b4:fd40:93a1, 2600:9000:2512:5800:4:c7b4:fd40:93a1, 2600:9000:2512:2e00:4:c7b4:fd40:93a1, 2600:9000:2512:d000:4:c7b4:fd40:93a1, 2600:9000:2512:c00:4:c7b4:fd40:93a1, 2600:9000:2512:5000:4:c7b4:fd40:93a1
* IPv4: 18.164.124.37, 18.164.124.15, 18.164.124.76, 18.164.124.101
*   Trying 18.164.124.37:443...
* Connected to docker.cloudsmith.io (18.164.124.37) port 443
* ALPN: curl offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /usr/lib/ssl/cert.pem
*  CApath: /usr/lib/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519 / RSASSA-PSS
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=*.cloudsmith.io
*  start date: Nov 24 00:00:00 2024 GMT
*  expire date: Dec 22 23:59:59 2025 GMT
*  subjectAltName: host "docker.cloudsmith.io" matched cert's "*.cloudsmith.io"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
> GET /v2/org/repo/app/blobs/sha256:sha HTTP/1.1
> Host: docker.cloudsmith.io
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 307 Temporary Redirect
< Content-Type: application/octet-stream
< Content-Length: 1716
< Connection: keep-alive
< Date: Tue, 31 Dec 2024 16:57:24 GMT
< location: https://artifacts.org.com/signed/repo/docker/images/sha256:sha?created=123456789&expires=987654321&signature=gAAAAABoXQd8nvhCMJcYkUP3kfR0g9sUrpG3I-QRZ3IVkFh1H6v-dPpMZxokQ7Hzq0Ut6NSxAzklC-fWzMj9fyaxHYZDIpSjoQ==
< docker-content-digest: sha256:sha
< allow: GET, DELETE, HEAD, OPTIONS
< Cache-Control: public, max-age=0
< docker-distribution-api-version: registry/2.0
< vary: Cookie, origin
< referrer-policy: same-origin
< Referrer-Policy: no-referrer-when-downgrade
< cross-origin-opener-policy: same-origin
< Server: Cloudsmith Docker
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 0
< Public-Key-Pins: pin-sha256="a9QX7yHr4KpL3Bc2X9V6cCmZ5XQh+NgYaUoTz5GnJhE="; pin-sha256="bKMT9xPq7CdW6Bn2L4F8tRmA2KwJ+NqXtYoVz5HlKiQ="; pin-sha256="cLOP8zGr5BpX7Lm3K9V2dNmB4XQj+NgWaUqTz5HlJkF="; max-age=86400; report-uri="https://cloudsmith.report-uri.io/r/default/hpkp/enforce"
< Strict-Transport-Security: max-age=31536000; includeSubdomains;
< Expect-CT: enforce, max-age=86400, report-uri="https://cloudsmith.report-uri.io/r/default/ct/enforce"
< Via: 1.1 e04ec889239bf67ef206ad086add2d7a.cloudfront.net (CloudFront), 1.1 eef964f7ded2584b0acfd4f410d14ff2.cloudfront.net (CloudFront)
< X-Amz-Cf-Pop: IAD66-C2
< X-Cache: Miss from cloudfront
< X-Amz-Cf-Pop: JFK50-P7
< Alt-Svc: h3=":443"; ma=86400
< X-Amz-Cf-Id: Ow-a9QX7yHr4KpL3Bc2X9V6cCmZ5XQh+NgYaUoTz5GnJhE=
< 
* TLSv1.3 (IN), TLS alert, close notify (256):
* transfer closed with 1716 bytes remaining to read
* Closing connection
* TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (18) transfer closed with 1716 bytes remaining to read

Reproduce

Configure registry to proxy to an upstream that re-directs layer pulls

Expected behavior

No response

registry version

main / "v3.0.0-rc.2.m+unknown"

Additional Info

No response
@milosgajdos
Copy link
Member
milosgajdos commented Dec 31, 2024 •
edited
Loading

Can you please paste the config you use.

@ChandonPierre
Copy link
Author

Can you please paste the config you use.

version: 0.1
log:
  level: debug
  formatter: json
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /tmp/registry
  tag:
    concurrencylimit: 8
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
auth: none
proxy:
  remoteurl: https://docker.cloudsmith.io

@ChandonPierre
Copy link
Author

With some more time to look into this, the actual problem is the upstream returns 404 if the blob is requested with HEAD instead of GET.

Using MethodGet here allows the pull-through to successfully complete:

distribution/internal/client/repository.go

Line 862 in 3270367

req, err := http.NewRequestWithContext(ctx, http.MethodHead, u, nil)

@milosgajdos
Copy link
Member

I haven't had much time for this, unfortunately., but I find this very....Odd.

HEAD should do everything GET does except for returning the actual content (as per the HTTP spec).

They both hit the same API handler:

distribution/registry/handlers/blob.go

Lines 35 to 36 in 3270367

http.MethodGet: http.HandlerFunc(blobHandler.GetBlob),
http.MethodHead: http.HandlerFunc(blobHandler.GetBlob),

But I'm assuming the registry running on docker.cloudsmith.io is running the latest v3 release.

@ChandonPierre
Copy link
Author

I haven't had much time for this, unfortunately., but I find this very....Odd.

HEAD should do everything GET does except for returning the actual content (as per the HTTP spec).

They both hit the same API handler:

distribution/registry/handlers/blob.go

Lines 35 to 36 in 3270367

62F1
http.MethodGet: http.HandlerFunc(blobHandler.GetBlob),
http.MethodHead: http.HandlerFunc(blobHandler.GetBlob),

But I'm assuming the registry running on docker.cloudsmith.io is running the latest v3 release.

Not sure what implementation they are using under the hood, but it doesn't seem entirely spec compliant. On GET it returns a 307 with location header which is why I initially thought re-directs weren't being followed. HEAD just returns 404 with no location header. We're going to open a ticket with them to understand this behavior, not sure how many upstreams have similar implementations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@milosgajdos @ChandonPierre
0