8000 ECH (Encrypted client hello) support · Issue #1924 · haproxy/haproxy · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

ECH (Encrypted client hello) support #1924

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Tristan971 opened this issue Nov 12, 2022 · 40 comments
Open

ECH (Encrypted client hello) support #1924

Tristan971 opened this issue Nov 12, 2022 · 40 comments
Labels
type: feature This issue describes a feature request / wishlist.

Comments

@Tristan971
Copy link
Member
Tristan971 commented Nov 12, 2022 •
edited
Loading

Your Feature Request

ECH (Encrypted client hello) is a developing specification for encrypting the original client hello in an HTTP1.1/HTTP2 context.

The main purpose is making SNI sniffing impossible by middle boxes and other such adversarial systems. The practical implementation is not dissimilar to how SSL certificates trust is established, by using certain new DNS records (+ DNS-over-HTTPS/TLS) as source of public keys involved (instead of a few CAs).

I had a short chat on the topic with @wlallemand at HAProxyConf, and he was aware of it and of the PoC referenced below. He hinted at it maybe being less relevant than before due to QUIC bringing encryption all the way through, but QUIC reaching the same level of usage as HTTP1.1/2 will take years. Especially when it still relies on Alt-Svc response headers at the moment, and while one will soon be able to advertise QUIC at the DNS level directly (see https://datatracker.ietf.org/doc/draft-ietf-dnsop-svcb-https/) this is also going to take a hot minute to be widely available, so I'm still quite interested in ECH in general (and hopefully I'm not alone in that).

Some relevant references/notes:

More specifically for HAProxy, the work done by the DEfO PoC people has progressed quite a bit on the OpenSSL side:

This is still somewhat early days (need HPKE merged, ECH to go from draft to RFC, and OpenSSL to adopt ECH) but I thought I'd raise this issue to have it in the tracker.

What are you trying to do?

Use ECH with HAProxy

Output of haproxy -vv

HAProxy version 2.7-dev8-7941ead+mangadex-cd2a7ce 2022-11-01T14:10+00:00 - https://haproxy.org/
Status: development branch - not safe for use in production.
Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
Running on: Linux 5.4.143-1-pve #1 SMP PVE 5.4.143-1 (Tue, 28 Sep 2021 09:10:37 +0200) x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -ggdb3 -gdwarf-4 -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wnull-dereference -fwrapv -Wno-unknown-warning-option -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment -DMAX_SESS_STKCTR=5
  OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_STATIC_PCRE2=1 USE_LIBCRYPT=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_TFO=1 USE_NS=1 USE_SYSTEMD=1 USE_QUIC=1 USE_PROMEX=1
  DEBUG   = -DDEBUG_MEMORY_POOLS -DDEBUG_STRICT

Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL +THREAD -PTHREAD_EMULATION +BACKTRACE -STATIC_PCRE +STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -ENGINE +GETADDRINFO +OPENSSL +LUA +ACCEPT4 -CLOSEFROM -ZLIB +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP -EVPORTS -OT +QUIC +PROMEX -MEMORY_PROFILING +SHM_OPEN

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=16, MAX_THREADS=256, default=8).
Built with OpenSSL version : OpenSSL 1.1.1q+quic-mangadex-cd2a7ce  1 Nov 2022
Running on OpenSSL version : OpenSSL 1.1.1q+quic-mangadex-cd2a7ce  1 Nov 2022
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.6
Built with the Prometheus exporter as a service
Built with network namespace support.
Support for malloc_trim() is enabled.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.40 2022-04-14
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with clang compiler version 14.0.6

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
       quic : mode=HTTP  side=FE     mux=QUIC  flags=HTX|NO_UPG|FRAMED
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=

Available services : prometheus-exporter
Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace
@Tristan971 Tristan971 added the type: feature This issue describe 8000 s a feature request / wishlist. label Nov 12, 2022
@Tristan971
Copy link
Member Author

And the HPKE prerequisite on OpenSSL side was just merged in openssl/openssl@ad06248 🙂

@sftcd
Copy link
sftcd commented Nov 25, 2022

Hiya,

I did the PoC referred to above. While it'll be a while before that'd be ready, there are two things on which some discussion could be useful in the meantime:

  1. Are my proposed new OpenSSL API's for ECH ok for haproxy? My current attempt at that is here - that differs a little from what's used in the PoC (I'll update the PoC sometime in the not-too-distant) and hasn't been reviewed by OpenSSL folk, but if some haproxy people had comments on those APIs, that'd be good input.

  2. The ECH integration with haproxy is mostly easy, but there is one hard problem. When operating in ECH split-mode, if one encounters a TLS HelloRetryRequest (HRR) then the haproxy instance has to be able to do ECH processing (decryption) of two ClientHello messages, but that just doesn't seem to match the haproxy architecture which (when in tcp mode which maps to ECH split-mode) assumes that haproxy only needs to examine the first message/ClientHello (which is entirely reasonable until one gets to ECH and HRR). My PoC didn't attempt to resolve that problem.

I'd be happy to chat about either of the above as I work on a PR for OpenSSL for ECH. (That won't be done in a few weeks, but maybe early in the new year is reasonable.)

@wlallemand
Copy link
Member

Thank you Tristan, that's good news. We had discussions about this in the past but nothing was merged in master.

HAProxy isn't well suited to handle "split-mode", because it will only be used as TCP-mode, and this mode is basically a tunnel once everything is connected, and handling HTTP connections with the TCP mode is not really recommended. We could discuss this more later but I think we should focus on the "shared" mode for now, which will be more easily integrated with the haproxy architecture.

In my opinion the "shared" mode, aka HAProxy as a TLS endpoint, could be integrated in haproxy, I don't know what are the state of the patches at the moment, but tit looks like the fork is not up to date.

@Tristan971
Copy link
Member Author
Tristan971 commented Nov 25, 2022 •
edited
Loading

Seconding William in that if one REALLY wants TCP mode haproxy + ECH they can use HAProxy as a pure « dumb » TCP forwarder (and yes it likely can’t do TCP SNI routing reliably in that case) and have whatever handles the actual TLS layer support the ECH bits. (And at that point you likely ought to be using something like LVS instead but that’s a whole other recurring topic because we all like haproxy too much to accept using something else :-) )

At least in my case only the so-called shared mode is really of interest, so it’s good that it’s the easier one 😄

@Tristan971
Copy link
Member Author

wrt to @sftcd:

Are my proposed new OpenSSL API's for ECH ok for haproxy?

It's been a few billion years since I last wrote some C but I'll try and see if I can find time to trial your OpenSSL API proposal patches and get somewhere (in ~2-3 weeks likely)

@git001 git001 mentioned this issue Jun 1, 2023
Closed
@wenerme
Copy link
wenerme commented Sep 13, 2023

Chrome 117 support ECH now, any news on HAProxy ECH ?

@Tristan971
Copy link
Member Author

Certainly didn't have time to look into it myself just yet, unfortunately :/ In other news, I updated the tracking bug for FF as they now migrated they meta-issue from ESNI to ECH here: https://bugzilla.mozilla.org/show_bug.cgi?id=1725938

@sftcd
Copy link
sftcd commented Sep 13, 2023 via email

@Tristan971
Copy link
Member Author
Tristan971 commented Sep 29, 2023 •
edited
Loading

Fyi some more updates and links (since the Chrome shipping was spotted before I did :).

Since wolfSSL has support for it (when built with HPKE support), and given the woes with OpenSSL already, is it perhaps something to consider without waiting for OpenSSL?

I ask because I am quite interested in it (and very willing to provide a testing ground for it, just like with QUIC before) and from the last few discussions on the ML and other issues here, there's less of a will to have HAProxy tied to OpenSSL than before and they seem intent on waiting until it is an RFC at least to consider it, which sftcd was guessing here would be at best Q1 2024.

Also to note that when OpenSSL comes to support it, it will necessarily end up being a 3.1.x thing at best anyway... Which is going to be problematic wrt performance and distro availability (since the current distro generation is on 3.0.x mostly, and unlikely to ship 3.1.x before late-2024 at best...)

@Fireant456
Copy link

We are running into issues utilizing HAProxy and our CloudFlare DNS as it appears CF is enforcing ECH by default causing issues with updated Chrome users. Is there a solution or work around to this?

I am guessing HAProxy would need to support ECH or we would need to use something other than HAProxy till it is implemented?

@sftcd
Copy link
sftcd commented Oct 4, 2023

Not sure I understand the problem, but am interested - can you elaborate?

@Fireant456
Copy link
8000

Not sure I understand the problem, but am interested - can you elaborate?

We use CloudFlare for DNS and proxy on our production domains. I am unsure all the ins and outs of ECH but it seems there might be a DNS component that although our development subdomains are not proxied by Cloudflare ECH is still being enforced somehow. This is causing our older 2.1 instances of HAProxy to throw the error "ERR_ECH_FALLBACK_CERTIFICATE_INVALID" when a Chrome 217 client is trying to access one of those domains. I am guessing this is due to lack of ECH support in HAProxy?

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

If Cloudflare forcefully publishes eager ECH records (HTTPS-type) on the zone forcefully, that sounds like it might cause issues indeed when clients trying ECH directly, even when they reach HAProxy without passing via CF.

Easiest would be to have CF proxying for your environment too, or to see if overriding HTTPS record on your development domain’s zone fixes it. But since CF forcefully appends their CAA records and similar, I have a feeling this wouldn’t quite work…

Maybe contact CF support asking them to opt-out your development domain zone from ECH. It’s likely they can do that.

@Fireant456
Copy link

I think that we are having luck with the proxy, but not sure if that will change or not. We are forced to not proxy WSS connections as CF tends to break those from time to time causing connection issues for our users. So we are seeing those subdomains affected in our production environment...

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

Thing is, even if HAProxy did support ECH, you would have the same issue.

ECH essentially uses 2 different certs:

  • 1 outer cert, visible to network operators
  • 1 inner cert, hidden from network operators

And the outer cert domain is defined by the DNS zone, and must be a valid cert (cloudflare-ech.com for Cloudflare iirc).
ECH keeps privacy by assuming that the outer cert is either non-specific (generic to all CF sites in this case) or at least somewhat innocuous.
It’d have been lovely to have a generic one anyone could use (imagine everyone using example.com as outer name/cert for example), but the cert authorities consider that evil so it won’t happen ever.

The inner cert is the site-specific one you currently have. So that one is no problem.

But naturally you wouldn’t be able to get such an outer cert yourself since you don’t control cloudflare-ech.com.

I’m not familiar enough with the spec to know how HTTPS records deal with multiple possible outer certs and priorities though, but my guess is you’d need to set it up with a higher-priority record for a domain you control and generate an outer cert for.

Edit: as clarified below by @sftcd, calling the identification material of an ECH exchange a "certificate" is incorrect

@Fireant456
Copy link

We can see cloudflare-ech.com being queried in the failed requests so maybe there is some mechanism there to allow our downstream certs as a fallback?

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

I wonder. You’ll have to bring it up with Cloudflare at this point. It’s not something HAProxy can help with directly (whether it supports ECH or not), as it’s a side effect of CF’s implementation of common zone records for their customers really.

@sftcd
Copy link
sftcd commented Oct 4, 2023

Don't have much time to comment now, but just to note that ECH does not involve use of new "certs" so the descriptions above are off-base a bit.

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

My bad, let's say "key-pairs-of-some-sort" then.

But unless I totally misunderstood, a relevant point is that you can't respond to an outer ClientHello requesting cloudflare-ech.com unless you have some private key material specific to that domain name, which only Cloudflare has as they publish the public portion of it on that zone, as far as I understand it.

So the effect is analoguous unless CF were to make that material public, which I'm not sure they're even allowed to (just like publishing a cert private key voluntarily is forbidden by the CAB and will result in any such cert being revoked).

@sftcd
Copy link
sftcd commented Oct 4, 2023 •
edited
Loading

Sorta. With ECH a browser will emit an initial TLS message (the ClientHello) that can only be fully processed by the entity with the relevant ECH private key. As I imagine CF deploy stuff, they assume that message will be sent to them. If somehow that message is sent somewhere else, then yes, ECH decryption won't happen, and the TLS handshake will fail with a (non-informative;-) message like the one quoted above.

@Tristan971

This comment was marked as off-topic.

Sign in to view
@Fireant456
Copy link

This seems like a large oversight and not much education was put out to compensate for this massive change. There are TONs of people that just use CloudFlare for DNS.... I have opened a ticket and replied on 1 of 2 semi-related ECH forum posts in the CF community forums so here's hoping they have some "good" solution to give us.

@sftcd
Copy link
sftcd commented Oct 4, 2023 •
edited
Loading

If you only use CF for DNS, then ECH shouldn't be in the picture. CF would have to be adding "ech=" values into your HTTPS RRs for that to happen. Are they doing that? If so, that'd be visble via dig so have you an example?

@sftcd
Copy link
sftcd commented Oct 4, 2023

For example:

$ dig +short https rte.ie
1 . alpn="h3,h2" ipv4hint=104.18.142.17,104.18.143.17 ipv6hint=2606:4700::6812:8e11,2606:4700::6812:8f11

Is for a CF customer (I think) but has no ech= and hence ECH will not be attempted.

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

If you only use CF for DNS, then ECH shouldn't be in the picture. CF would have to be adding "ech=" values into your HTTPS RRs for that to happen. Are they doing that? If so, that'd be visble via dig so have you an example?

Most likely they do it on @ if it is set to proxy mode, and then it applies to subdomains? Because on a zone of mine I do indeed not see it yet. Tho rte.ie might be on a non-free CF plan and they're delaying the rollout for those. idk.

But if for free CF accounts people are in a situation of:

  • domain.com (@) set to proxied
  • dev.domain.com set to dns-only

have the second one falling back to using the HTTPS record on @?

Then again, that sounds odd since HTTPS is more akin to an A/AAAA record than anything else, but idk... maybe just browser bug atm

@sftcd
Copy link
sftcd commented Oct 4, 2023

Not sure - I'm not familiar with CF's biz model(s), sorry. Doesn't (so far) sound like a browser bug though, but one could tell for a given DNS name using dig and wireshark pretty quickly, so shouldn't be hard to figure what behaviour is happening vs. expected.

@Fireant456
Copy link
Fireant456 commented Oct 4, 2023 •
edited
Loading

For example:

$ dig +short https rte.ie
1 . alpn="h3,h2" ipv4hint=104.18.142.17,104.18.143.17 ipv6hint=2606:4700::6812:8e11,2606:4700::6812:8f11

Is for a CF customer (I think) but has no ech= and hence ECH will not be attempted.

image

I don't seem to be getting the same output as you. If you would like to check on your end I don't mind sharing some domains that are affected. My other developer that has been looking into this issue with me noted that the ECH SSL_ECH_CONFI_LIST they are seeing might be from cache. So maybe caching is our issue here since some sub domains are proxied and have ECH properly working and when the user tries to access a DNS-only domain the ECH is still trying to be queried.
image

Working Proxied:
sonoransoftware.com
sonorancms.com

Not Working DNS-only:
ws.sonorancms.com (WSS only, but putting into chrome should still show the error mentioned)
cms.dev.sonoransoftw 6D40 are.com
git.sonoran.software

NOTE: the ws.sonorancad.com request shown in the chrome logs is now working as we have moved to a paid CF plan to disable ECH on that domain entirely. We currently are using that to have an active ticket with CF on this issue.

@Tristan971
Copy link
Member Author 
$ dig +short sonoransoftware.com https
1 . alpn="h3,h2" ipv4hint=104.21.38.120,172.67.222.151 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3031::6815:2678,2606:4700:3037::ac43:de97
$ dig +short git.sonoransoftware.com https
1 . alpn="h3,h2" ipv4hint=104.21.38.120,172.67.222.151 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3031::6815:2678,2606:4700:3037::ac43:de97
$ dig +short git.sonoran.software https
git.dev.sonoransoftware.com.
1 . alpn="h3,h2" ipv4hint=188.114.96.0,188.114.97.0 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2a06:98c1:3120::,2a06:98c1:3121::
$ dig +short sonoran.software https
1 . alpn="h3,h2" ipv4hint=104.21.54.71,172.67.136.96 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3035::6815:3647,2606:4700:3037::ac43:8860

I do see the records everywhere here, might be your dig is outdated or something

@Fireant456
Copy link 
$ dig +short sonoransoftware.com https
1 . alpn="h3,h2" ipv4hint=104.21.38.120,172.67.222.151 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3031::6815:2678,2606:4700:3037::ac43:de97
$ dig +short git.sonoransoftware.com https
1 . alpn="h3,h2" ipv4hint=104.21.38.120,172.67.222.151 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3031::6815:2678,2606:4700:3037::ac43:de97
$ dig +short git.sonoran.software https
git.dev.sonoransoftware.com.
1 . alpn="h3,h2" ipv4hint=188.114.96.0,188.114.97.0 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2a06:98c1:3120::,2a06:98c1:3121::
$ dig +short sonoran.software https
1 . alpn="h3,h2" ipv4hint=104.21.54.71,172.67.136.96 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3035::6815:3647,2606:4700:3037::ac43:8860

I do see the records everywhere here, might be your dig is outdated or something

Not sure, just installed it via my WSL AlmaLinux 8 environment 🤷

@sftcd
Copy link
sftcd commented Oct 4, 2023

I see the same dig outputs as the post just above, e.g.:

$ dig +short sonoransoftware.com https
1 . alpn="h3,h2" ipv4hint=104.21.38.120,172.67.222.151 ech=AEX+DQBB+QAgACDX9TC2tUA8p/hiOiEab+YAtdI17bmXO1p6YjCnlkiXcwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:3031::6815:2678,2606:4700:3037::ac43:de97

@Tristan971
Copy link
Member Author
Tristan971 commented Oct 4, 2023 •
edited
Loading

That said we strictly stick to DNS-only usage of CF and indeed these records don't show up yet on our zone so idk.

$ dig +short mangadex.org https
$

But since they do still generate TLS certs for our domain, and inject their own CAA records as well, I suspect that it is an oversight of theirs and we'll soon be in the same situation as you. So at this point we're just going to move out of CF for auth NS, just in case.

Nonetheless, all of this should very much be a discussion with CF.

@Fireant456
Copy link

Yep ageed, I appreciate you assisting me a bit in diagnosing this. Hopefully not downplaying this issue :)

@lukastribus
Copy link
Member

The Cloudflare issue is indeed off topic. I guess people will be on the lookout for a workaround and then may come back here posting:

It looks like disabling TLSv1.3 in your Cloudflare settings works around this problem even on the free plan, so you avoid the need to upgrade to a paid plan just to restore DNS-only services.

Source: https://community.cloudflare.com/t/getting-ech-fallback-certificate-error/565167/16

All further discussions regarding Cloudflare belong into their community forums (or support) of course.

Let's keep this feature request on topic.

@Ralle
Copy link
Ralle commented Aug 29, 2024

Hello guys,

I used this ticket as a reference for a discussion on Cloudflare I made today. The team responded and are rolling back ECH. Thank you for helping me helping us :)

https://community.cloudflare.com/t/err-ech-fallback-certificate-invalid-with-subdomain/704703/8

@crabique
Copy link
crabique commented Sep 4, 2024

It appears ECH is now supported by all major browsers and seems to be the final form of that solution, would be very cool to have it available in HAProxy. TLS temination mode on high-density things like CDNs and app hosting is where it would truly shine, due to the frontend to backend numbers ratio.

@awptechnologies
Copy link

I use OPNsense with haproxy. This is backed by acme certs. Since ECH has come out i get errors with certs. I use a single cert with multiple subject alt names and it seems the browsers don't understand this. After disabling ech in firefox with a custom json file it works fine. My issue is mobile firefox refuses to work. Does anyone have any idea if this is happening because of ech or the lack of ech support in haproxy? If i seperate all of the subject alt names into individual certs would that put a band-aid on the issue?

@sftcd
Copy link
sftcd commented Dec 17, 2024

FWIW, I'd be surprised were that due to ECH. haproxy doesn't currently include any ECH code in releeases AFAIK. Perhaps your 'custom json file' is doing more than disabling ECH?

@awptechnologies
Copy link

It doesn't.

Also it seams anything other then firefox doesn't have issues so idk what they have going on.

Here is contents of my json file.
{ "policies": { "DisableEncryptedClientHello": true } }

@lukastribus
Copy link
Member

This has nothing to do with haproxy, most of the web does not support ECH.

Instead you need to troubleshoot this between your nameserver configuration and firefox. Do your nameserver advertise HTTPS records? Do those HTTPS records show a ECH configuration?

Are you on a free plan of Cloudflare? Then you are subject to their beta testing.

None of this is in any way related to haproxy.

@DemiMarie
Copy link

ECH would be useful in layer 4 proxying as well as for layer 7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: feature This issue describes a feature request / wishlist.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@wlallemand @Ralle @Tristan971 @Fireant456 @wenerme @sftcd @crabique @DemiMarie @lukastribus @awptechnologies
0