FreeBSD + NGINX + Varnish - limit of processed requests.

Hi!

I've one issue:

NGINX + VARNISH

Code:
wrk -c 9500 -t 16 -d 15s -H "Host: example.com" -s request-only.lua https://example.com
Running 15s test @ https://example.com
  16 threads and 9500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   372.92ms  388.82ms   1.91s    81.94%
    Req/Sec     1.42k   450.02    10.04k    90.32%
  283049 requests in 15.10s, 2.76GB read
  Socket errors: connect 0, read 0, write 0, timeout 5861
  Non-2xx or 3xx responses: 8
Requests/sec:  18739.43
Transfer/sec:    187.39MB

TCP connection count by state:
    0 connections in CLOSED state
    97 connections in LISTEN state
    6884 connections in SYN_SENT state
    3906 connections in SYN_RCVD state
    12123 connections in ESTABLISHED state
    2 connections in CLOSE_WAIT state
    2452 connections in FIN_WAIT_1 state
    28 connections in CLOSING state
    9801 connections in LAST_ACK state
    106 connections in FIN_WAIT_2 state
    66 connections in TIME_WAIT state

VARNISH

Code:
wrk -c 9500 -t 16 -d 15s -H "Host: example.com" -s request-only.lua http://example.com
Running 15s test @ http://example.com
  16 threads and 9500 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    28.60ms   76.12ms   1.93s    90.84%
    Req/Sec     7.20k     4.17k   18.97k    61.67%
  688140 requests in 12.06s, 6.71GB read
  Socket errors: connect 0, read 0, write 0, timeout 70
Requests/sec:  57053.58
Transfer/sec:    569.57MB

TCP connection count by state:
    0 connections in CLOSED state
    97 connections in LISTEN state
    0 connections in SYN_SENT state
    0 connections in SYN_RCVD state
    9495 connections in ESTABLISHED state
    0 connections in CLOSE_WAIT state
    0 connections in FIN_WAIT_1 state
    61 connections in CLOSING state
    1 connection  in LAST_ACK state
    0 connections in FIN_WAIT_2 state
    71 connections in TIME_WAIT state

All connections to the Varnish goes through 127.0.0.1 (NGINX -> Varnish which listen on localhost). Both solutions are actually very similar (working separately) in terms of performance (number of requests handled with the same number of connections) with a slight advantage of NGINX (up to 10k req/s). However, working together there is a problem with a certain limit of processed requests. If there were some limits on the loopback in a FreeBSD? Or maybe LAST_ACK packets and FreeBSD system limits (i.e. kern.ipc.somaxconn or others). What commands or outputs may still be helpful in debugging this problem?

Thanks a lot!
 
Last edited by a moderator:
Back
Top