PF Misbehaving nested inline anchors

mickey

mickey

Hi all

While I was experimenting with nested (inline) anchors in PF, I found some behaviour that I have no explanation for. The following simple pf.conf is somewhat similar in structure to an example given in pf.conf(5) within the section discussing the use of inline anchors. All testing was done on a releng/11.2 amd64 machine.
Code: 
ext_if="re0"
set loginterface $ext_if
set skip on lo0

scrub in all

block log

pass quick inet proto icmp
pass quick inet6 proto icmp6

anchor "an1" in on $ext_if {
    pass quick inet proto tcp to port time
    anchor "an2" {
        pass quick inet proto tcp to port daytime
    }
}

pass out quick on $ext_if
When this ruleset is loaded, pfctl -a '*' -sr shows the following:
Code: 
scrub in all fragment reassemble
block drop log all
pass quick inet proto icmp all keep state
pass quick inet6 proto ipv6-icmp all keep state
anchor "an1" in on re0 all {
  pass quick inet proto tcp from any to any port = time flags S/SA keep state
  anchor "an2" all {
    pass quick inet proto tcp from any to any port = daytime flags S/SA keep state
  }
}
pass out quick on re0 all flags S/SA keep state
Using telnet from a remote machine I can successfully connect to the time port, but connections to the daytime port are blocked by PF although the rule in anchor an2 should allow them through. Another strange thing happens when I now flush the rules from anchor an2 using pfctl -a "an1/an2" -Fr and then display the active rules again using pfctl -a '*' -sr :
Code: 
scrub in all fragment reassemble
block drop log all
pass quick inet proto icmp all keep state
pass quick inet6 proto ipv6-icmp all keep state
anchor "an1" in on re0 all {
  pass quick inet proto tcp from any to any port = time flags S/SA keep state
  anchor "an2" all {
pfctl: DIOCGETRULES: Invalid argument
  }
}
pass out quick on re0 all flags S/SA keep state
Notice the error message when pfctl tries to read the rules from anchor an2. When I now populate the anchor an2 again by using echo "pass quick inet proto tcp to port daytime" | pfctl -a "an1/an2" -f - the output of pfctl -a '*' -sr looks exactly as it did after initially loading the pf.conf but connections to the daytime port are still blocked by PF.

Now the following pf.conf is identical to the one used above but uses a normal non-inline anchor for an2:
Code: 
ext_if="re0"
set loginterface $ext_if
set skip on lo0

scrub in all

block log

pass quick inet proto icmp
pass quick inet6 proto icmp6

anchor "an1" in on $ext_if {
    pass quick inet proto tcp to port time
    anchor "an2"
}

pass out quick on $ext_if
After loading it and populating the an2 anchor using echo "pass quick inet proto tcp to port daytime" | pfctl -a "an1/an2" -f - the output of pfctl -a '*' -sr looks pretty much the same as it did before:
Code: 
scrub in all fragment reassemble
block drop log all
pass quick inet proto icmp all keep state
pass quick inet6 proto ipv6-icmp all keep state
anchor "an1" in on re0 all {
  pass quick inet proto tcp from any to any port = time flags S/SA keep state
  anchor "an2" all {
    pass quick inet proto tcp from any to any port = daytime flags S/SA keep state
  }
}
pass out quick on re0 all flags S/SA keep state
However in contrast to before, now both time and daytime ports are reachable from a remote machine, just as one would expect. Also flushing the rules from anchor an2 by means of pfctl -a "an1/an2" -Fr and then displaying the ruleset with pfctl -a '*' -sr does NOT yield the ioctl error message as it did with the inlined an2 anchor.

Any ideas as to why the nested inline anchor is not behaving as one would expect or whether this could possibly be a bug are highly appreciated.
 
While this may not be the answer. It looks to me, that it doesn't like your nesting of rules, as you have in your first example. I couldn't comment definitively, as I don't like the way they read. So I don't do that. A limitation?
Just thought I'd mention it. :)

--Chris
 
Chris_H said:
While this may not be the answer. It looks to me, that it doesn't like your nesting of rules, as you have in your first example. I couldn't comment definitively, as I don't like the way they read. So I don't do that. A limitation?
Just thought I'd mention it. :)
Click to expand...
I don't think that it got a personality that dislikes the way I write my ruleset :p
Nested anchors are documented in pf.conf(5) and a number of existing solutions are using nested anchors, like ftp-proxy(8) for example. They are also great for partitioning your ruleset and to perform conditional filtering.

Anyway, I did some further experimentation:
  • inline anchor / inline anchor = FAIL
  • inline anchor / regular anchor = PASS
  • regular anchor / inline anchor = FAIL
  • regular anchor / regular anchor = PASS
I guess this draws a pretty clear picture: Inline anchors nested within another (inline or regular) anchor are ineffective, or in other words simply do not work. They appear exactly as they should, but the rules contained therein are simply not effective.
 
mickey said:
I don't think that it got a personality that dislikes the way I write my ruleset :p
Click to expand...
Cute. But I mean't pf(4). But I can't blame pf, I don't like reading them, either. :p
Nested anchors are documented in pf.conf(5) and a number of existing solutions are using nested anchors, like ftp-proxy(8) for example. They are also great for partitioning your ruleset and to perform conditional filtering.

Anyway, I did some further experimentation:
  • inline anchor / inline anchor = FAIL
  • inline anchor / regular anchor = PASS
  • regular anchor / inline anchor = FAIL
  • regular anchor / regular anchor = PASS
I guess this draws a pretty clear picture: Inline anchors nested within another (inline or regular) anchor are ineffective, or in other words simply do not work. They appear exactly as they should, but the rules contained therein are simply not effective.
Click to expand...
Hard to say why. But that's what it looked like, to me.
I think you were safe to say it was a bug (at least within the documentation).

Good detective work, mickey ! Pat yourself on the back, for a job well done! :)

--Chris
 
Chris_H said:
I think you were safe to say it was a bug (at least within the documentation).
Click to expand...
I found an older PR 196314, according to which this problem should have been fixed quite some time ago. Now I am puzzled... did this patch never hit the streets or is this a regression?
 
mickey said:
While I was experimenting with nested (inline) anchors in PF, I found some behaviour that I have no explanation for.
Click to expand...
I would need to do some testing myself but at first impression I get the strong sense that your definition could be at fault here.

See, from pf.conf(5):

Anchors may be nested, with
components separated by `/' characters, similar to how file system
hierarchies are laid out. The main ruleset is actually the default
anchor, so filter and translation rules, for example, may also be
contained in any anchor.
Click to expand...
And that's what I'm missing on your end. Your name space doesn't really show nesting.

The other reason why I suspect that your definition is at fault is because my main server actually has a nested anchor set up, yet your shared command of # pfctl -a '*' -sr does not work on that:
Code: 
root@breve:/home/peter # pfctl -a blacklistd/21 -sr
block drop in quick proto tcp from <port21> to any port = ftp
root@breve:/home/peter # pfctl -a '*' -sr | grep port21
pfctl: DIOCGETRULES: Invalid argument
See, the thing is: this command doesn't show any anchors at all, at best it shows me this:
Code: 
anchor "*" in on bge1 all {
}
block drop quick from <shitlist> to any
block drop log on bge1 all
Making me question the validity of that command.

What does: # pfctl -a an1 -sr show? Also: what does # pfctl -a an1 -sA tell you?

(edit)

mickey said:
I guess this draws a pretty clear picture: Inline anchors nested within another (inline or regular) anchor are ineffective, or in other words simply do not work.
Click to expand...
That's incorrect.

blacklistd(8) uses inline anchors as I've briefly shown above and I can assure you that the anchors and their containing rules (and tables) fully work.
 
Sorry for a double post, but above I addressed previous comments, yet in this post I'm addressing a test I just performed. Hence I figured it was more appropriate to post instead of edit.

I concur that something odd seems to be going on.

In my main firewall setup I added an extra anchor:
Code: 
anchor "blacklistd/*" in on $net_if
anchor test
load anchor test from "/root/temp/anchor.conf"
The file anchor.conf contains the following rules:
Code: 
win7vm = "10.0.1.26/32"

block from $win7vm
anchor allow {
  pass from $win7vm
}
So here comes the weird part:

Code: 
root@zefiris:/home/peter # pfctl -sA
  blacklistd
  test
root@zefiris:/home/peter # pfctl -a test -sA
  test/allow
  test/test
root@zefiris:/home/peter # pfctl -a test/test -sA
  test/test/allow
root@zefiris:/home/peter # pfctl -a test/test/allow -sA
root@zefiris:/home/peter # pfctl -a test/allow -sr
pass inet from 10.0.1.26 to any flags S/SA keep state
root@zefiris:/home/peter # pfctl -a test/test/allow -sr
Right now I don't quite understand how I ended up with a seemingly "double nested" anchor (test/test). This even happens if I define the anchor entirely in the main pf.conf file:
Code: 
anchor "test" {
  block from $win7vm
  anchor allow {
    pass from $win7vm
  }
}
This also results in test/test to appear for which I don't have a reasonable explanation at this time.
 
ShelLuser said:
I would need to do some testing myself but at first impression I get the strong sense that your definition could be at fault here.
Click to expand...
Anchors are declared within the context of the current anchor, that's what makes them nested.Use of slashes to separate the anchor components is only necessary when you need to specify an anchor-path, like when manipulating nested anchors externally by means of pfctl -a .... The following is the example given in pf.conf(5), which is quite similar in structure to the ruleset I have used for testing:
Code: 
     Filter rule anchors can also be loaded inline in the ruleset within a
     brace ('{' '}') delimited block.  Brace delimited blocks may contain
     rules or other brace-delimited blocks.  When anchors are loaded this way
     the anchor name becomes optional.

           anchor "external" on egress {
                   block
                   anchor out {
                           pass proto tcp from any to port { 25, 80, 443 }
                   }
                   pass in proto tcp to any port 22
           }

     Since the parser specification for anchor names is a string, any
     reference to an anchor name containing `/' characters will require double
     quote (`"') characters around the anchor name.
ShelLuser said:
The other reason why I suspect that your definition is at fault is because my main server actually has a nested anchor set up, yet your shared command of # pfctl -a '*' -sr does not work on that:
Code: 
root@breve:/home/peter # pfctl -a blacklistd/21 -sr
block drop in quick proto tcp from <port21> to any port = ftp
root@breve:/home/peter # pfctl -a '*' -sr | grep port21
pfctl: DIOCGETRULES: Invalid argument
Click to expand...
I wasn't the one to come up with that command, it's in pfctl(8). Actually until just recently when I was revisiting pf.conf(5) and pfctl(8) what feels like an additional fifty times, I didn't even know it existed:
Code: 
             To print the main ruleset recursively, specify only `*' as the
             anchor name:

                   # pfctl -a '*' -sr
But you are right, pfctl -a '*' -sr chokes on anchors declared as anchor "blah/*"., I have noticed that too.

Which brings us to another important point. You need to draw a fat red line between anchor "blah" and anchor "blah/*" because they work quite differently, which may not be apparent at first.
  • anchor "blah" evaluates all the rules contained in that anchor recursively, including any attached child anchors.
  • anchor "blah/*" however does not evaluate any rules contained in that anchor at all (at least that's what my tests indicated), instead it evaluates all rules in all anchors directly attached to anchor blah in alphabetical ordering. But it does not descend deeper into the anchor structure (i.e. no recursing), it's merely a flat operation.
Programs like ftp-proxy(8) or blacklistd(8) use the latter form because that's exactly what they require. Usually they will create flat sub-anchors named after the user, or process ID or similar and populate these with only a few filter rules, but no nested anchors.

ShelLuser said:
I concur that something odd seems to be going on.

In my main firewall setup I added an extra anchor:
Code: 
anchor "blacklistd/*" in on $net_if
anchor test
load anchor test from "/root/temp/anchor.conf"
The file anchor.conf contains the following rules:
Code: 
win7vm = "10.0.1.26/32"

block from $win7vm
anchor allow {
  pass from $win7vm
}
So here comes the weird part:

Code: 
root@zefiris:/home/peter # pfctl -sA
  blacklistd
  test
root@zefiris:/home/peter # pfctl -a test -sA
  test/allow
  test/test
root@zefiris:/home/peter # pfctl -a test/test -sA
  test/test/allow
root@zefiris:/home/peter # pfctl -a test/test/allow -sA
root@zefiris:/home/peter # pfctl -a test/allow -sr
pass inet from 10.0.1.26 to any flags S/SA keep state
root@zefiris:/home/peter # pfctl -a test/test/allow -sr
Right now I don't quite understand how I ended up with a seemingly "double nested" anchor (test/test). This even happens if I define the anchor entirely in the main pf.conf file:
Code: 
anchor "test" {
  block from $win7vm
  anchor allow {
    pass from $win7vm
  }
}
This also results in test/test to appear for which I don't have a reasonable explanation at this time.
Click to expand...
That could be attributed to remnants of former test runs, as anchors (and their contents) do not get automatically deleted by simply overloading a (modified) ruleset. To be absolutely certain you needed to flush the entire ruleset/nat/etc first using pfctl -F all, then use pfctl -sA to see if there are still any anchors in existence. If that is the case, check those anchors for any rules, nat rules and sub-anchors, and work your way from bottom to top, flushing all anchors of their rules / nat rules, until eventually there's nothing more left within the main ruleset.
 
mickey said:
Anchors are declared within the context of the current anchor, that's what makes them nested. Use of slashes to separate the anchor components is only necessary when you need to specify an anchor-path, like when manipulating nested anchors externally by means of pfctl -a ....
Click to expand...
Ayups, I stand corrected on that part. I did some extra studying of my own (pf is my favorite firewall right now) and actually learned some new things there.

Thanks for your comments.

mickey said:
Which brings us to another important point. You need to draw a fat red line between anchor "blah" and anchor "blah/*" because they work quite differently, which may not be apparent at first.
  • anchor "blah" evaluates all the rules contained in that anchor recursively, including any attached child anchors.
  • anchor "blah/*" however does not evaluate any rules contained in that anchor at all (at least that's what my tests indicated), instead it evaluates all rules in all anchors directly attached to anchor blah in alphabetical ordering. But it does not descend deeper into the anchor structure (i.e. no recursing), it's merely a flat operation.
Click to expand...
I'm not too sure. But I am going to set up a more thorough test case of my own because your thread honestly intrigued me. But going on what I know about blacklistd this doesn't seem fully correct. Well... gray zone I guess:

Code: 
root@breve:/home/peter # pfctl -a blacklistd/21 -sr
block drop in quick proto tcp from <port21> to any port = ftp
See, the table port21 does get processed. I'm fully positive because I tested blacklistd within a LAN environment before putting it into production. It definitely blocked my logon attempts.

Of course, and this is where my 'gray zone' comment comes into play: one can argue that a table isn't really nested but merely a "parallel" entity. It's most definitely not a rule.

[about the test/test anchor]

mickey said:
That could be attributed to remnants of former test runs, as anchors (and their contents) do not get automatically deleted by simply overloading a (modified) ruleset.
Click to expand...
Good point, but I ruled that out. Before I started pfctl -sA would only show blacklistd. Then I applied my new rules, checked, and for sure: test/test was suddenly a new anchor. And it makes no sense to me what so ever.
 
ShelLuser said:
Code: 
root@breve:/home/peter # pfctl -a blacklistd/21 -sr
block drop in quick proto tcp from <port21> to any port = ftp
See, the table port21 does get processed. I'm fully positive because I tested blacklistd within a LAN environment before putting it into production. It definitely blocked my logon attempts.

Of course, and this is where my 'gray zone' comment comes into play: one can argue that a table isn't really nested but merely a "parallel" entity. It's most definitely not a rule.
Click to expand...
Tables are a different beast of their own and there is indeed a fallback mechanism in place so that rules within an anchor which are referencing a table that does not exist as a private table within the same anchor fall back to using a table of that name from the main ruleset. Somewhat like variable scoping. So if in your example pfctl -a "blacklistd/21" -sT does not show any private tables in that anchor, but pfctl -sT shows a table named <port21>, then the rule in the anchor will use the table from the main ruleset. Anyhow: table != anchor.

ShelLuser said:
[about the test/test anchor]

Good point, but I ruled that out. Before I started pfctl -sA would only show blacklistd. Then I applied my new rules, checked, and for sure: test/test was suddenly a new anchor. And it makes no sense to me what so ever.
Click to expand...
Lets just hope there are no more strange things going on with anchors, cause I really like the idea of using anchors for conditional filtering.
Let me know how your experiments turn out.
 
You must log in or register to reply here.
Back
Top