Quic HTTP/3 with HAProxy

Thomas315 · Jul 4, 2025

Hello,

I've tried to enable Quic/HTTP3 with HAProxy on Freebsd, but it seems that it is not properly working.
Haproxy daemon doesn't seems to answer to UDP/QUIC packets.

Here are some info of my setup.

Code:

root@www1:~ # uname -a
FreeBSD www1.brozs.net 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC amd64

Code:

[root@www1:~ # pkg info | grep haproxy
haproxy-wolfssl-3.2.1          Reliable, high performance TCP/HTTP load balancer

Code:

root@www1:~ # haproxy -vv
HAProxy version 3.2.1-f4d1a4e 2025/06/11 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2030.
Known bugs: http://www.haproxy.org/bugs/bugs-3.2.1.html
Running on: FreeBSD 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC amd64
Build options :
  TARGET  = freebsd
  CC      = cc
  CFLAGS  = -O2 -g -O2 -pipe -I/usr/local/include/wolfssl -fstack-protector-strong -fno-strict-aliasing -fwrapv -DFREEBSD_PORTS
  OPTIONS = USE_GETADDRINFO=1 USE_OPENSSL_WOLFSSL=1 USE_ACCEPT4=1 USE_ZLIB=1 USE_CPU_AFFINITY=1 USE_TFO=1 USE_THREAD_DUMP=1 USE_QUIC=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
  DEBUG   =

Feature list : -51DEGREES +ACCEPT4 -BACKTRACE +CLOSEFROM +CPU_AFFINITY -CRYPT_H -DEVICEATLAS -DL -ENGINE -EPOLL -EVPORTS +GETADDRINFO +KQUEUE -LIBATOMIC +LIBCRYPT -LINUX_CAP -LINUX_SPLICE -LINUX_TPROXY -LUA -MATH -MEMORY_PROFILING -NETFILTER -NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_AWSLC +OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL -PRCTL +PROCCTL +PROMEX -PTHREAD_EMULATION +QUIC -QUIC_OPENSSL_COMPAT -RT -SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL +ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=32, MAX_THREADS=1024, default=4).
Built with SSL library version : wolfSSL 5.8.0
Running on SSL library version : wolfSSL 5.8.0
SSL library supports TLS extensions : yes
SSL library supports SNI : yes
SSL library FIPS mode : no
SSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
QUIC: connection socket-owner mode support : no
QUIC: GSO emission support : no
Built with the Prometheus exporter as a service
Built with zlib version : 1.3.1
Running on zlib version : 1.3.1
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY
Built with PCRE2 version : 10.45 2025-02-05
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with clang compiler version 18.1.6 (https://github.com/llvm/llvm-project.git llvmorg-18.1.6-0-g1118c2e05e67)

Available polling systems :
     kqueue : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use kqueue.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
       quic : mode=HTTP  side=FE     mux=QUIC  flags=HTX|NO_UPG|FRAMED
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
       spop : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
  <default> : mode=SPOP  side=BE     mux=SPOP  flags=HOL_RISK|NO_UPG
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=

Available services : prometheus-exporter
Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace

The configuration wich allow to enable QUIC/HTTP/3 is there

Code:

frontend fe_main
        mode http
        capture request header User-Agent len 64
        option forwardfor
        bind X.Y.Z.W:443 ssl crt /usr/local/etc/ssl/certs/haproxy.pem alpn h2
        bind quic4@X.Y.Z.W:443 ssl crt /usr/local/etc/ssl/certs/haproxy.pem alpn h3
        .....
               use_backend %[req.hdr(Host),lower]
        http-after-response add-header alt-svc 'h3=":443"; ma=60'

When started, haproxy listen to UDP properly

Code:

root@www1:~ # sockstat | grep haproxy | grep udp4
www      haproxy    54862 5   udp4   X.Y.Z.W:443      *:*

The IP binding is a virtual IP (CARP).

But while receiving packet, haproxy does not answer and I can't see any debug log which could explain this behaviour.

Code:

root@www1:~ # tcpdump -n -i vlan2401 host A.B.C.D and port 443 and proto 17
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vlan2401, link-type EN10MB (Ethernet), snapshot length 262144 bytes
08:11:26.159336 IP A.B.C.D.14190 > X.Y.Z.W.443: UDP, length 1200
08:11:27.159528 IP A.B.C.D.14190 > X.Y.Z.W.443: UDP, length 1200
08:11:29.158788 IP A.B.C.D.14190 > X.Y.Z.W.443: UDP, length 1200
08:11:33.156193 IP A.B.C.D.14190 > X.Y.Z.W.443: UDP, length 1200

At the client side, it is beeing seen as request timeout

Code:

tom@vm-debian-3:~$ ./curl -v -sI --http3-only https://www.example.net
* Host www.example.net:443 was resolved.
* IPv6: (none)
* IPv4: X.Y.Z.W
*   Trying X.Y.Z.W:443...
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* QUIC connection has been shut down
* QUIC connect to X.Y.Z.W port 443 failed: Could not connect to server
* Failed to connect to www.example.net port 443 after 30016 ms: Could not connect to server
* closing connection #0

I've check firewalling, also try to disable it without success.

Could you give hints to make it work properly ?

Many Thanks

Thomas

flecaille · Jul 4, 2025

Hello,
Could you enable the QUIC traces please as follows:

Code:

ring buf1
    size <size>
    format timed
    backing-file <filename>

traces
    trace quic sink buf1 level developer start now
    trace qmux sink buf1 level developer verbosity minimal start now

Thomas315 · Jul 5, 2025

Hello,

Thanks for your help. Here's the requested trace.

flecaille · Jul 9, 2025

Still investigating...

dark.initr0 · Jul 11, 2025

Try to add ICMP filter, maybe it will show some clues
or ICMP

Thomas315 · Jul 15, 2025

dark.initr0 said:
Try to add ICMP filter, maybe it will show some clues
or ICMP

No ICMP packet during the Quic Request

flecaille · Jul 15, 2025

I think we have found the source of the problem. A patch will be soon produced.

flecaille · Jul 16, 2025

Here is the patch: https://github.com/haproxy/haproxy/commit/1c33756f7803247c81354f55678322b636279089
I do not know if you have the possibility to test it.

Thomas315 · Jul 17, 2025

Thanks i'll try and let you know wether it is working or not

Thomas315 · Jul 17, 2025

I've replaced quic_sock.c by this one (the one with the patch) https://raw.githubusercontent.com/h...3247c81354f55678322b636279089/src/quic_sock.c

But compilation failed with error

Code:

root@www1:/usr/ports/net/haproxy # make install
===>  Patching for haproxy-3.2.3
===>  Applying FreeBSD patches for haproxy-3.2.3 from /usr/ports/net/haproxy/files
===>   haproxy-3.2.3 depends on package: gmake>=4.4.1 - found
===>   haproxy-3.2.3 depends on shared library: libpcre2-8.so - found (/usr/local/lib/libpcre2-8.so)
===>   haproxy-3.2.3 depends on shared library: libwolfssl.so - found (/usr/local/lib/libwolfssl.so)
===>  Configuring for haproxy-3.2.3
===>  Building for haproxy-3.2.3
  CC      dev/flags/flags.o
  CC      src/ev_poll.o
  CC      src/ev_kqueue.o
  CC      src/cpuset.o
  CC      src/cpu_topo.o
  CC      src/ssl_sock.o
  CC      src/ssl_ckch.o
  CC      src/ssl_ocsp.o
  CC      src/ssl_crtlist.o
  CC      src/ssl_sample.o
  CC      src/cfgparse-ssl.o
  CC      src/ssl_gencert.o
  CC      src/ssl_utils.o
  CC      src/jwt.o
  CC      src/ssl_clienthello.o
  CC      src/jws.o
  CC      src/acme.o
  CC      src/ssl_trace.o
  CC      src/mux_quic.o
  CC      src/h3.o
  CC      src/quic_rx.o
  CC      src/quic_tx.o
  CC      src/quic_conn.o
  CC      src/quic_frame.o
  CC      src/quic_sock.o
  CC      src/quic_tls.o
  CC      src/quic_ssl.o
src/quic_sock.c:90:42: error: no member named 'target' in 'struct quic_conn'
   90 |                 struct listener *l = objt_listener(qc->target);
      |                                                    ~~  ^
src/quic_sock.c:704:40: error: no member named 'target' in 'struct quic_conn'
  704 |         return (!is_addr(&__objt_listener(qc->target)->rx.addr) &&
      |                                           ~~  ^
src/quic_sock.c:842:41: error: no member named 'target' in 'struct quic_conn'
  842 |         struct listener *l = objt_listener(qc->target);
      |                                            ~~  ^
src/quic_sock.c:854:37: error: no member named 'max_udp_payload' in 'struct quic_conn'
  854 |                 BUG_ON(b_contig_space(&buf) < qc->max_udp_payload);
      |                                               ~~  ^
include/haproxy/bug.h:349:44: note: expanded from macro 'BUG_ON'
  349 | #  define BUG_ON(cond, ...)   _BUG_ON     (cond, __FILE__, __LINE__, 3, "FATAL: bug ",     "", __VA_ARGS__)
      |                                            ^~~~
include/haproxy/bug.h:282:18: note: expanded from macro '_BUG_ON'
  282 |         (void)(unlikely(cond) ? ({                                              \
      |                         ^~~~
include/haproxy/compiler.h:322:40: note: expanded from macro 'unlikely'
  322 | #define unlikely(x) (__builtin_expect((x) != 0, 0))
      |                                        ^
src/quic_sock.c:861:42: error: no member named 'max_udp_payload' in 'struct quic_conn'
  861 |                 ret = quic_recv(qc->fd, dgram_buf, qc->max_udp_payload,
      |                                                    ~~  ^
src/quic_sock.c:951:39: error: no member named 'target' in 'struct quic_conn'
  951 |                 quic_dgram_parse(new_dgram, qc, qc->target);
      |                                                 ~~  ^
src/quic_sock.c:978:43: error: no member named 'target' in 'struct quic_conn'
  978 |         struct listener *l = __objt_listener(qc->target);
      |                                              ~~  ^
src/quic_sock.c:1086:43: error: no member named 'target' in 'struct quic_conn'
 1086 |         struct listener *l = __objt_listener(qc->target);
      |                                              ~~  ^
8 errors generated.
gmake: *** [Makefile:1052: src/quic_sock.o] Error 1
gmake: *** Waiting for unfinished jobs....
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/net/haproxy
*** Error code 1

Stop.
make: stopped in /usr/ports/net/haproxy
[\code]

flecaille · Jul 17, 2025

Ok. I see. This is because you use 3.2 version in place of current 3.3 dev branch. Let's try with this new patch which is a backport.

flecaille · Jul 17, 2025

It seems the patch was not attached. Let's try again.

Thomas315 · Jul 18, 2025

With this patch, quic/http3 is ok. thanks

flecaille · Jul 18, 2025

So the next step for FreeBSD would be to wait for the next 3.2 release and provide a new package for haproxy.

Quic HTTP/3 with HAProxy

Thomas315

flecaille

Thomas315

Attachments

flecaille

dark.initr0

Thomas315

flecaille

flecaille

Thomas315

Thomas315

flecaille

flecaille

Attachments

Thomas315

flecaille