IPFW rules for torrent - undesirable blocking

Nyakov · Mar 29, 2019

I have.
1. pfSense router with NAT
2. Server on lan with transmission jail and $jtrip
3. pfSense forwards $tr_peer_port
4. ipfw on server with such rules (transmission related part)

Bash:

# Set rules command prefix
cmd="ipfw -q add

# No restrictions on Loopback Interface
$cmd 00010 allow all from any to any via lo0

# allows the packet through if it matches an existing entry in the dynamic rules table:
$cmd 00101 check-state

$cmd 00106 deny ip from any to 255.255.255.255 in via $host_if
$cmd 00108 deny ip from any to 224.0.0.0/4 in via $host_if

##############################Transmission jail##########################
# Allow everything out
$cmd 00280 allow tcp from $jtrip to any out via $host_if keep-state
$cmd 00281 allow udp from $jtrip to any out via $host_if keep-state
# Allow outbound ping
$cmd 00282 allow icmp from $jtrip to any out via $host_if keep-state
##################################################################

# deny and log all other outbound connections
$cmd 00299 deny log logamount 0 all from any to any out via $host_if

#######################Transmission JAIL#######################
#Transmission peer
$cmd 00451 allow tcp from any to $jtrip $tr_peer_port in via $host_if keep-state
$cmd 00452 allow udp from any to $jtrip $tr_peer_port in via $host_if
# Allow some inbound icmps - echo reply, dest unreach, source quench, echo, ttl exceeded.
$cmd 00453 allow icmp from any to $jtrip icmptypes 0,3,4,8,11 in via $host_if
#########################################################################

# Reject and log all other incoming connections
$cmd 00499 deny log logamount 0 all from any to any in via $host_if

What i get:
Deny messages in log

Code:

kernel: ipfw: 499 Deny TCP 95.73.216.122:43171 jail-ip:24929 in via re0
kernel: ipfw: 499 Deny TCP 178.94.35.138:23267 jail-ip:23410 in via re0
kernel: ipfw: 499 Deny TCP 77.82.55.126:47449 jail-ip:45621 in via re0

Why this happening?

linux->bsd · Mar 30, 2019

What does ipfw -tea list show for rules {00451..00453}, and 00499? You can try simplifying the allow rules by removing the port number, protocol, and interface restrictions and see if things change, then work from there.

Nyakov · Mar 31, 2019

linux->bsd said:
What does ipfw -tea list show for rules {00451..00453}, and 00499?

Code:

00451 19127181 16445452046 Sun Mar 31 16:24:00 2019 allow tcp from any to jail-ip 51417 in via re0 keep-state :default
00452   257049    16458317 Sun Mar 31 16:22:05 2019 allow udp from any to jail-ip 51417 in via re0
00453    23823     2015521 Sun Mar 31 16:18:38 2019 allow icmp from any to jail-ip icmptypes 0,3,4,8,11 in via re0

00499      981      246555 Sun Mar 31 16:22:06 2019 deny log ip from any to any in via re0

linux->bsd · Mar 31, 2019

Nyakov said:

Code:

00451 19127181 16445452046 Sun Mar 31 16:24:00 2019 allow tcp from any to jail-ip 51417 in via re0 keep-state :default
00452   257049    16458317 Sun Mar 31 16:22:05 2019 allow udp from any to jail-ip 51417 in via re0
00453    23823     2015521 Sun Mar 31 16:18:38 2019 allow icmp from any to jail-ip icmptypes 0,3,4,8,11 in via re0

00499      981      246555 Sun Mar 31 16:22:06 2019 deny log ip from any to any in via re0

Your rules are being hit by some packets, clearly. You can zero-out the counters with ipfw zero and logs with ipfw resetlog, then check which packets are falling through to rule 00499 that should be matching the other three rules.

At this point I would try to simplify the problem to find out which packets are bypassing those rules by filtering the inbound packets at the source, not the destination. But I'm not too familiar with dynamic rules, so if you're lucky someone that is might be able to spot if a tweak to your rules is required. The ipfw() manpage describes commands for monitoring stateful rules, namely, show states.

Do you have much expirience with tcpdump?

A tool you might find useful during your testing: sysutils/cmdwatch, the Linux watch program.

Hiroo Ono · Apr 1, 2019

For tcp packets out, add "setup keep-state", not "keep-state" only.
cf. "STATEFUL FIREWALL" section of ipfw() manual.

Nyakov · Apr 4, 2019

Hiroo Ono said:
For tcp packets out, add "setup keep-state", not "keep-state" only.

Then I get constant flow of denied packets in log.
When without "setup" they appears only time to time.

D-FENS · Apr 4, 2019

Nyakov said:
Then I get constant flow of denied packets in log.
When without "setup" they appears only time to time.

You have to make sure you understand the behavior of your firewall configuration completely. If you don't understand it 100% that's bad, you have no idea if you left any security holes in it.

So the firewall design process goes something like this:

First make a list of the applications you want to enable (everything else will be protected) - in your case it is the transmission program.
Then goto the documentation and see which packets, protocols, ports and directions need to be allowed. For transmission it is described here briefly: https://trac.transmissionbt.com/wiki/PortForwardingGuide . In summary, you need to open a port for others to connect to your node, this is done through port forwarding. You Internet router should do a DNAT on the bittorrent protocol and port and redirect incoming packets to your bittorrent jail. If necessary, you should do multiple port forwardings, i.e. router to host and host to jail, unless your jail is bridged directly to the LAN.
When creating rules for the dynamic firewall ( keep-state and check-state) keep in mind that TCP is a session oriented protocol. So you do rules only for the initial packet of a connection (use keyword " setup" in your rule) and create a dynamic rule (" keep-state"). For UDP, ICMP etc. you use only " keep-state" without " setup", because they do not have sessions. If one packet is allowed, all subsequent packets will come through via a dynamic rule.

So, in your rules I do not see any port forwarding as required per documentation above, and as Hiroo Ono wrote, your " tcp" rules are missing a " setup" keyword. Everywhere where you have " tcp" and " keep-state" put also " setup"! And if packets get blocked, this means you missed something else but definitely need " setup".

For port forwarding (DNAT) you do something like this:

Code:

ipfw add nat 123 all from any to any
ipfw nat 123 config if $host_if redirect_port tcp $jtrip:$tr_peer_port $tr_peer_port

Also, keep in mind you have to do port forwarding for the whole path from Internet to your jail. So, do DNAT on the router (with external IP), and on your host (if the host acts as a router for the jail).

Nyakov · Apr 7, 2019

roccobaroccoSC said:
So, in your rules I do not see any port forwarding as

Port forwarded in router firewall.
Jail just have alias LAN ip.
Other jails services and host do just fine.

Perhaps I need some special nat processing in firewall which is pfsence...

Nyakov · Apr 7, 2019

Nyakov said:
Then I get constant flow of denied packets in log.

I get denied TCP packets to and from $jail_ip $tr_peer_port.
So, they somehow do not satisfy setup rule.

If they was only to me, than I can imagine it is malicious traffic.
But my own transmission program generate this packets...

driesm · Apr 7, 2019

I think your setup works just fine ... The packets getting denied that you mentioned don't satisfy your allow rule 453 because of the port you specify in that rule (51417).
This is what you want, no? I think your transmission port is opened just fine to the outside world. What you see getting logged are packets that should be dropped and are actually dropped.

As you can see what you stated:

Code:

kernel: ipfw: 499 Deny TCP 95.73.216.122:43171 jail-ip:24929 in via re0
kernel: ipfw: 499 Deny TCP 178.94.35.138:23267 jail-ip:23410 in via re0
kernel: ipfw: 499 Deny TCP 77.82.55.126:47449 jail-ip:45621 in via re0

The destination port behind jail-ip does not match your rules 51417 port and therefore doesn't match and gets subsequently denied by your 499 deny rule.
Also as a side note, if you want to port forward transmission to the internet, you do not need to open UDP and TCP, one is fine. Default is TCP port 51413.
You can also check if your transmission is in fact correctly opened to the internet, either in the web interface or the client.

Nyakov · Apr 7, 2019

My setup is: wan <-> pfsense <-> transmission_host
pfsense - mostly default settings, forwards one tcp and udp port to transmission_host
transmission_host - have its own ipfw - and this is what I am looking at.

Duffyx said:
What you see getting logged are packets that should be dropped and are actually dropped.

Because my router forward only one port to jail lan IP, the only way I can see such incoming packets is because transmission in my side established connection thru router NAT and then for some reason firewall in transmission host system stop honor it, but router NAT still do.
Or I am missing something?

Duffyx said:
I think your setup works just fine ...

It seems so eventually...
And it is actually working, yes, but this denied packets what bothers me.

What I am after rigth now, it seams transmission is tended to send ack packets in freely manner...
So my working theory is torrent protocol not really obey ordinary tcp rules...
But then, just keep-state must be able to make the trick anyway.
Perhaps keep-state rules getting timed out?

VladiBG · Apr 7, 2019

At your pfsense you need UPnP and on your transmission daemon enable portmapping via UPnP.