PF PF not passing over 3GB of data

[networknewb]

I have a 10Gb environment that I'm trying to setup PF to do NAT.

Without NAT, I can pass about 7Gb of data through. With PF turned on, it won't go over 2.8Gb of data.

I increased some limits, but they don't seem to be the issue.

Here's my config

set limit states 2048000
set limit frags 2048000
set limit table-entries 2048000
set optimization aggressive

set skip on lo

scrub in

nat on $ext_if from $client_net -> ($ext_if)

Here's my output

# pfctl -s info
No ALTQ support in kernel
ALTQ related functions disabled
Status: Enabled for 0 days 00:12:21           Debug: Urgent

State Table                          Total             Rate
  current entries                     1950              
  searches                       391593407       528466.1/s
  inserts                             4662            6.3/s
  removals                            2712            3.7/s
Counters
  match                          195806375       264246.1/s
  bad-offset                             0            0.0/s
  fragment                               0            0.0/s
  short                                  0            0.0/s
  normalize                              0            0.0/s
  memory                                 0            0.0/s
  bad-timestamp                          0            0.0/s
  congestion                             0            0.0/s
  ip-option                              0            0.0/s
  proto-cksum                            0            0.0/s
  state-mismatch                       322            0.4/s
  state-insert                           0            0.0/s
  state-limit                            0            0.0/s
  src-limit                              0            0.0/s
  synproxy                               0            0.0/s


# pfctl -vvsr
No ALTQ support in kernel
ALTQ related functions disabled
@0 scrub in all fragment reassemble
  [ Evaluations: 786206565  Packets: 394114174  Bytes: 199351549448  States: 0     ]
  [ Inserted: uid 0 pid 5212 State Creations: 0     ]
@0 pass out quick on vlan90 all flags S/SA keep state
  [ Evaluations: 197176759  Packets: 197657002  Bytes: 199829202694  States: 1927  ]
  [ Inserted: uid 0 pid 5212 State Creations: 4236  ]
@1 pass quick on vmx0 all flags S/SA keep state
  [ Evaluations: 197152431  Packets: 292       Bytes: 35656       States: 1     ]
  [ Inserted: uid 0 pid 5212 State Creations: 41    ]