1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

PF + ALTQ + FTP = fail? (a bug?)

Discussion in 'Firewalls' started by romeor, May 8, 2012.

  1. romeor

    romeor New Member

    Messages:
    89
    Likes Received:
    0
    Hi there,

    As we all know, there is only one way to shape the bandwidth the right way: the outgoing traffic (egress). Today I got in situation with PF in which it seems to not be working the right way.
    Here are my rules:
    Code:
    altq on em0 hfsc bandwidth 10Mb queue {std_up ftp_up sig_up}
        queue std_up bandwidth 8% hfsc (default realtime 2% ecn )
        queue ftp_up bandwidth 46% hfsc (realtime 39% ecn)
        queue sig_up bandwidth 46% hfsc (realtime 39% ecn)
    
    altq on em1 hfsc bandwidth 10Mb queue { std_down ftp_down sig_down}
        queue std_down bandwidth 8% hfsc (default realtime 2% ecn )
        queue ftp_down bandwidth 46% hfsc (realtime 39% ecn)
        queue sig_down bandwidth 46% hfsc (realtime 39% ecn)
    
    
    #FTP
    nat on $ext_if from $ftp_lan to any -> $ftp_wan
    
    #RDR
    
    #FTP
    rdr pass on $ext_if proto tcp from any to $ftp_wan port 21 -> $ftp_lan port 21
    rdr pass on $ext_if proto tcp from any to $ftp_wan port 51000:65534 -> ftp_lan port 51000:65534
    
    
    #RULES
    block in log all
    pass out
    ################### WAN RULES ##########################
    #discard traffic from or to bogon nets on external interfaces
    block drop in log quick on $ext_if from $priv_nets to any
    block drop out log quick on $ext_if from any to $priv_nets
    #drop any ipv6 as not implemented in our net
    block drop in log quick proto ipv6
    
    
    ################## LAN RULES #########################
    pass in on vlan1112 from vlan1112:network no state
    
    ############### Shaping
    
    #Upload shaping pipe (external interface)
    pass out on vlan1041 from $ftp_wan queue ftp_up
    pass out on vlan1041 from $sig_lin_wan queue sig_up
    pass out on vlan1041 from $sig_win_wan queue sig_up
    
    #Download shaping pipe (outgoing to lan)
    pass out on vlan1112 to $ftp_lan queue ftp_down no state
    pass out on vlan1112 to $sig_lin_lan queue sig_down no state
    pass out on vlan1112 to $sig_win_lan queue sig_down no state
    
    Rules are as simple, as they can be (I didn't show the macros definitions, but it seems obvious, that it is not necessary).

    So the problem is with the FTP rule: As soon as I try these rules, FTP outgoing traffic goes to the std_up queue. Why? Download traffic goes into the right one. If I test the sig_lin_wan rule, it goes into the right queue.

    Code:
    QUEUE                             BW SCH  PRIO     PKTS    BYTES   DROP_P   DROP_B QLEN BORROW SUSPEN     P/S     B/S
    root_em0                         10M hfsc    0        0        0        0        0    0                     0       0
     std_up                         800K hfsc          2680  3970890       11    16654    0                    17   25235
     ftp_up                        4600K hfsc             0        0        0        0    0                     0       0
     sig_up                        4600K hfsc             0        0        0        0    0                     0       0
    root_em1                         10M hfsc    0        0        0        0        0    0                     0       0
     std_down                       800K hfsc             0        0        0        0    0                     0       0
     ftp_down                      4600K hfsc          1424    79468        0        0    0                     7     431
     sig_down                      4600K hfsc             0        0        0        0    0                     0       0
    
    And it keeps going there whatever I do.

    So I decided to try to cheat it and added this rule right after my first LAN rule:
    Code:
    pass in on vlan1112 from $ftp_lan queue ftp_up no state
    From my network experience it is the wrong way of shaping (as traffic first gets in, then it is shaped), but it is the only way I can get it working.

    So is this a bug or some other specific PF and FTP problem? My FTP uses passive mode, all ports are redirected and matching passive ports defined in proftpd.conf. Also have the MasqueradeAddres in proftpd.conf matching my $ftp_wan in pf.conf

    Any ideas? The OS is FreeBSD 9.0.
     
  2. romeor

    romeor New Member

    Messages:
    89
    Likes Received:
    0
    And now I marked this in pftop:

    Code:
     
      [B]28  Pass     In        vlan11                 8242 12358775        0       inet from 192.168.20.0/24 to any[/B]
     [I] 29  Pass     Out       vlan10           K        0        0        0       inet from "ftp_wan"/32 to any  flags S/SA queue ftp_up[/I]
      30  Pass     Out       vlan10           K        0        0        0       inet from "sig_lin_wan"/32 to any  flags S/SA queue sig_up
      31  Pass     Out       vlan11                 4234   174971        0       inet from any to "ftp_lan"/32  queue ftp_down
    

    The 29'th rule counter is not moving, like the network connection was directly established between my client and the FTP server behind this router. Weird? If I test the 30th rule, the counter moves.
     
  3. romeor

    romeor New Member

    Messages:
    89
    Likes Received:
    0
  4. fbsd1

    fbsd1 New Member

    Messages:
    213
    Likes Received:
    0
    Freebsd FreeBSD 7.x through 9.0 is using the OpenBSD 4.5 version of PF firewall. The current PF OpenBSD 5.0 version has different syntax for ftp-proxy. You have to use the Freebsd FreeBSD pf manpages for pf syntax and sample rules.
     
  5. romeor

    romeor New Member

    Messages:
    89
    Likes Received:
    0
    Why should I use ftp-proxy?
     
  6. bbzz

    bbzz New Member

    Messages:
    867
    Likes Received:
    0
    Shaping is always done egress, as ingress interface can't hold large buffer queue, among other things. Your rule of matching queue ingress is only marking traffic that will be shaped egress later, through that same interface, and is the right way to do it here. The actual shaping will only occur egress.
     
  7. ecazamir

    ecazamir New Member

    Messages:
    215
    Likes Received:
    0
    1. NAT is messing your rules. Using $ftp_lan instead of $ftp_wan for upload may help.
    2. Match packets with ipfw and shape them with ALTQ.
    3. Use packet marks/tags.
    4. Use ipfw queues/pipes, any IPv4 packet that can match an ipfw rule can be sent to queues/pipes. A HFSC without "childs" can be simulated fair enough with WF2Q+ weights.
     
  8. romeor

    romeor New Member

    Messages:
    89
    Likes Received:
    0
    bbzz
    Thanks for explanation about applying rules to incoming traffic. Very useful.
    ecazamir
    I don't think it is a NAT-related problem, as it happens only with FTP traffic, as I did mention before. Anyway, ipfw is not an option for me at the moment. Had problems with its kernel-NAT in 8.0-8.2 with port redirection (could not redirect more than 19 ports, as it started to complain about memory - the only way to fix it was source hacking - not my approach to production things, so I don't even remember the ipfw syntax anymore :( )
     
  9. ecazamir

    ecazamir New Member

    Messages:
    215
    Likes Received:
    0
    IPFW can be used together with pf. It's possible to filter/classify with ipfw while NATing and queueing with pf.

    Did you try adding a queue on the wan side using $ftp_lan as source?