Virtual Network Interface for use with Jails

Code: 
jail_myjail_ip="192.168.13.201"
jail_myjail_interface="rl0"

Keep in mind the 192.168.13.201 address will be added as an alias to rl0. You will not get a "virtual" interface.
 
Code: 
ifconfig_em0_alias0="inet 192.168.13.[B]201[/B]/32"
ifconfig_em0_alias1="inet 192.168.13.[B]202[/B]/32"
and etc. in /etc/rc.conf. You are supposed to restrict the network mask (/32) so that no other network IP# can be available for Jail's IP.

P.S. It was SirDice who taught me this stuff in the first place. Thanks again, pal.
 
Good day, I'm still trying to find the ideal way to do the same thing. I have two physical interfaces (em0 as the DMZ and em1 as the LAN) using ezjail with some bound to a LAN alias and some bound to my DMZ alias. An excerpt of my rc.conf is below. My fix was to manually add a route to 10.100.0.0/16 which is my whole LAN so that the internal jails can talk to everything. This works but I think using pf I can have it be a little bit cleaner.

Code: 
defaultrouter="192.168.102.1"
ipv6_defaultrouter="2001:123:4567:890::1"
static_routes="-net 10.100.0.0 10.100.102.1 255.255.0.0"
ipv6_static_routes=""

ifconfig_em0="inet 192.168.102.2 netmask 0xffffff00"
ifconfig_em0_ipv6="inet6 2001:123:4567:890::2 prefixlen 64"
ifconfig_em0_alias0="inet 192.168.102.11 netmask 0xffffffff"
ifconfig_em0_alias1="inet6 2001:123:4567:890::11 prefixlen 128"

ifconfig_em1="inet 10.100.102.2 netmask 0xffffff00"
ifconfig_em1_ipv6="inet6 2001:123:4:5678::2 prefixlen 64"
ifconfig_em1_alias0="inet 10.100.102.11 netmask 0xffffffff"
ifconfig_em1_alias1="inet6 2001:123:4:5678::11 prefixlen 128"

Put it into effect by:
# service netif restart; service routing restart; route add -net 10.100.0.0 10.100.102.1 255.255.0.0
 
Main System Grabbing IPs

Ok, thank you all for the tips.

I built my jail with the IP set, and my /etc/rc.local looks like:

Code: 
$ more /etc/rc.conf
hostname="SG"
ifconfig_em0="DHCP"
ifconfig_em1="DHCP"
ifconfig_em2="inet 192.168.13.201/32"
sshd_enable="YES"
ntpd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
ezjail_enable="YES"
ifconfig_em2_alias0="inet 192.168.13.202/32"
ifconfig_em2_alias1="inet 192.168.13.203/32"
pf_enable="YES"

Any ideas on how I can set this in the jail?

I tried to ssh into the alias, and it is directed to the host system. I wasn't surprised when this happened. I think I'm supposed to push the IPs on the Jails in here?

Code: 
$ jls
   JID  IP Address      Hostname                      Path
     1  192.168.13.202  build                         /usr/jails/build
 
Alright, I suppose that could be correct. Can you provide a bit more detail on your physical interfaces that tie back to the /etc/rc.conf? It looks a bit odd to see three interfaces with the first two DHCP and the last one static. Is there a default gateway on em2 or is that just a switched network directly to internal clients?

In the meantime, SSHD listens on all IP addresses so you need to lock it down to only certain addresses.

Change your /etc/ssh/sshd_config from this...
Code: 
#ListenAddress 0.0.0.0
#ListenAddress ::

To something like this
Code: 
ListenAddress 10.100.102.2
ListenAddress 2001:123:4:5678::2

I'm not sure how to lock the ISC's NTPD down to do the same. I just use net/openntpd as it doesn't listen on anything by default.

Code: 
cd /usr/ports/net/openntpd/ && make install clean
echo 'openntpd_enable="YES"' >> /etc/rc.conf
service openntpd start
 
Also, didn't notice this at first, your netmask should only be /32 on your aliases.

Change /etc/rc.conf
Code: 
ifconfig_em2="inet 192.168.13.201/32"
ifconfig_em2_alias0="inet 192.168.13.202/32"
ifconfig_em2_alias1="inet 192.168.13.203/32"

To something like this or modified to the appropriate netmask of your LAN.
Code: 
ifconfig_em2="inet 192.168.13.201/24"
ifconfig_em2_alias0="inet 192.168.13.202/32"
ifconfig_em2_alias1="inet 192.168.13.203/32"
 
I would suggest not using aliases but use the way I described in post #2. This way the IP address is only added (i.e. active) when the jail is started and it's removed again when the jail stops. It also keeps all the relevant jail variables in one place.

On 9.1 (not sure about 9.0) there's also now a config file specifically for jails so you don't have to clutter up rc.conf with a whole bunch of jail_* definitions. See jail.conf(5) for more information.
 
Your title "Virtual Network Interface for use with Jails" is incorrect. There are no "Virtual Network Interfaces". Correct title would be "How to assign network interfaces to jails using ezjail?

Short answer is ezjail does not have ability to do this. Also mixing manual jail statements in rc.conf for ezjail jails have no effect on ezjail configurations.

I suggest you use sysutils/qjail instead of ezjail. qjail has ability to assign interface to jail IE: qjail create -n em2 jailname ip-address-on-em2

Note: Your pf firewall MUST DO NAT on all private LAN ip address.
 
fbsd1 said:
Correct title would be "How to assign network interfaces to jails using ezjail?

Short answer is ezjail does not have ability to do this.
Click to expand...
Sure it's possible. Similar to FreeBSD with jail.conf as of 9.1. Just edit the jail config file in /usr/local/etc/ezjail.

e.g. for my mailserver I have somthing like this in /usr/local/etc/ezjail/mail:

Code: 
export jail_mail_hostname="mail.example.com"
export jail_mail_interface="rl0"
export jail_mail_ip="192.168.178.202,2001:DB8::FFFF:C0A8:B2CA"
 
Are the lines in the correct places?

junovitch
Thanks, yeah the networking is a little strange, but the two have DHCP running. The other is basically just a switch​


SirDice
I'm still trying to learn so I'll try and figure it out. I didn't understand the jail_ stuff really. It looks like I can put it in rc.local or in the file with all the exports in the jail folder /usr/local/etc/ezjail/NAME_OF_JAIL​


fbsd1
Why does it need to do network address translation?​


Wiedmann
When you execute "jls" it shows mail under path/name of the jail? (Similar to the one I use above called build) So I would need to have the jail_myjail_interface set as sirdice said above and then I can set it in the jails?​


Beeblebrox
I changed that line.​


Everyone

I'm currently trying to do this, putting everything right now in rc.local, but I want to move them to the ezjail/NAME_OF_JAIL file, I think they are the same with just an "EXPORT " prefixing the line.

So I removed the alias lines from /etc/rc.conf
Code: 
hostname="host.example.com"
ifconfig_em0="DHCP"
ifconfig_em1="DHCP"
ifconfig_em2="inet 192.168.13.201/24"
sshd_enable="YES"
ntpd_enable="YES"
dumpdev="NO"
ezjail_enable="YES"

pf_enable="YES"

and compensated by changing the following in /usr/local/etc/ezjail/build
Code: 
# To specify the start up order of your ezjails, use these lines to
# create a Jail dependency tree. See rcorder(8) for more details.
#
# PROVIDE: build_jail
# REQUIRE:
# BEFORE:
#

export jail_build_hostname="build"
export jail_build_interface="em2"
export jail_build_ip="192.168.13.202"
export jail_build_rootdir="/usr/jails/build"

Looks like it may not have worked
Code: 
build# pkg_add -r wget
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9.0-release/Latest/wget.tbz: Protocol not supported
pkg_add: unable to fetch 'ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9.0-release/Latest/wget.tbz' by URL

Thanks again everybody.
​

PS, I'm guessing that I can un-comment the PROVIDE/REQUIRE/BEFORE lines and add "build_jail" to the REQUIRE line of other config files to enforce order?
PPS, What is the difference between "rl0" and "em0" aka 'rl' vs 'em'
 
WhyWontThisWork said:
When you execute jls(8) it shows mail under path/name of the jail? (Similar to the one I use above called build)​
Click to expand...
Looks like:
Code: 
freebsd# jls
   JID  IP Address      Hostname                      Path
     5  192.168.178.202 mail.example.com              /usr/jails/mail
Well, I'm creating the jail with "ezjail-admin create mail 192.168.178.202" using only hostname (for shorter paths) and IPv4. And then I edit "/usr/local/etc/ezjail/mail" to my need. In this case FQDN and adding IPv6:
Code: 
export jail_mail_hostname="mail.example.com"
export jail_mail_ip="192.168.178.202,2001:DB8::FFFF:C0A8:B2CA"


WhyWontThisWork said:
So I would need to have the jail_myjail_interface set as sirdice said above and then I can set it in the jails?​
Click to expand...
That's in my /usr/local/etc/ezjail/mail:
Code: 
export jail_mail_interface="rl0"


Of course. If all your jails are using the same interface (and/or if it's not defined in ezjail), you can set a default in /etc/rc.conf:
Code: 
jail_interface="rl0"

WhyWontThisWork said:
PPS, What is the difference between "rl0" and "em0" aka 'rl' vs 'em'
Click to expand...
The name depends on your hardware.
em -- Intel(R) PRO/1000 Gigabit Ethernet adapter driver
rl -- RealTek 8129/8139 Fast Ethernet device driver

From your posts above, you should have 3 Intel network adapters.
 
WhyWontThisWork said:
PS, I'm guessing that I can un-comment the PROVIDE/REQUIRE/BEFORE lines and add "build_jail" to the REQUIRE line of other config files to enforce order?
Click to expand...
No, those remarks are supposed to be there. If you remove them you will get shell syntax errors. See rcorder(8).
 
Wiedmann: Thank You for all that information (it really helped my understanding), is there somewhere that I can check that my IP is being handled by the jail?

SirDice: Thank you, similar to the c++ preprocessors.

I'm still having problems getting the jails to talk to the world, can I test something in PF? I'm not sure where I should be trying to figure what is wrong. nmap of the jail's IP claims one host is up, and I think the /etc/resolv.conf file is ok. I used "#jexec 1 sh" to bring up a shell in the jail, and the /etc/resolv.conf matches with my router IP and google DNS (just to test). From inside I cannot install packages because name resolution doesn't function. I also don't have tracert or anything similar.

Please and Thank you, again.
 
WhyWontThisWork said:
is there somewhere that I can check that my IP is being handled by the jail?
Click to expand...
jls -V or ezjail-admin list should show all ip's from the jail.
Or ifconfig(8) inside the jail: ezjail-admin console -e "ifconfig" jailname


WhyWontThisWork said:
I used jexec 1 sh to bring up a shell in the jail,
Click to expand...
With ezjail you can also use ezjail-admin console jailname.


WhyWontThisWork said:
I also don't have tracert or anything similar.
Click to expand...
Doesn't work in a default jail (security.jail.allow_raw_sockets).


WhyWontThisWork said:
can I test something in PF?
Click to expand...
Sorry, can't help you with PF. Is it working without?


BTW:
WhyWontThisWork said:
PS, I'm guessing that I can un-comment the PROVIDE/REQUIRE/BEFORE lines and add "build_jail" to the REQUIRE line of other config files to enforce order?
Click to expand...
That's also possible. Of course, don't un-comment these lines. Edit the name after PROVIDE from standard_ezjail to e.g. jail1_ezjail / jail2_ezjail. And use these names in REQUIRE/BEFORE to have a specify start order.
 
Getting Network Connection In Jail

Wiedmann said:
Sorry, can't help you with PF. Is it working without?
Click to expand...
It is not, but I was under the impression that I needed PF to use the jails, I would rather not use PF and use a network based firewall. It would be nice, however, to use a NAT so I believe that PF might be a solution to implement.

I'm currently doing research on how to get a NAT firewall working with FreeBSD jails. I'll post results here if/when I find something.

Thank you again!

UPDATE:

Turns out I can ssh from the jail to the main IP of my jail only network interface.
Code: 
#ezjail-admin console build
build# ssh 192.168.13.201
Password:

Code: 
$ ifconfig
em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 08:00:27:df:cf:75
        inet 192.168.13.201 netmask 0xffffff00 broadcast 192.168.13.255
        inet6 fe80::a00:27ff:fedf:cf75%em2 prefixlen 64 scopeid 0x4
        inet 192.168.13.202 netmask 0xffffffff broadcast 192.168.13.202
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

Also, I realize that I need something to bridge traffic between my external host interface, and the interfaces where the jails talk between themselves and the main system.
 
Some Network Functions but not others

Hi All,
So I have everything configured. I tried to have pf log by adding the "log" keyword, however, I may have put it in an incorrect place. I hope /var/log/pf.today is not the incorrect log.
Code: 
$ ls -alh /var/log/pf.today
-rw-------  1 root  wheel     0B Dec 13 03:30 /var/log/pf.today

In /etc/pf.conf
Code: 
pass in [B]log[/B] all
pass out [B]log[/B] all

All attempts to trace the log have said there is no data in the file (tail confirms)
Code: 
tcpdump -n -e -ttt -r /var/log/pf.today

The funny anomaly comes when dig functions, "ssh HOST" fails for anything outside of my network, yet, functions for other hosts on my network. Queries with dig report the responding DNS server is the server at my external bordering network, same gateway that communicates with my ISP. I *can* ssh a host which is connected with a physical external switch, therefore, I believe that network traffic is being translated correctly.

I have tried a few things to test:

An attempt to ssh a public host from the host works
Code: 
$ ssh -v {PUBLIC_HOST_TCBU}
OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503, OpenSSL 0.9.8q 2 Dec 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to {PUBLIC_HOST_TCBU} [128.205.32.51] port 22.
debug1: Connection established.
debug1: identity file /home/main/.ssh/id_rsa type -1
debug1: identity file /home/main/.ssh/id_rsa-cert type -1
debug1: identity file /home/main/.ssh/id_dsa type -1
debug1: identity file /home/main/.ssh/id_dsa-cert type -1
debug1: identity file /home/main/.ssh/id_ecdsa type -1
debug1: identity file /home/main/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4*
debug1: Remote is not HPN-aware
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
The authenticity of host '{PUBLIC_HOST_TCBU} (128.205.32.51)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? no
Host key verification failed.
$
However, the same command from the jail fails
Code: 
build# ssh -v {PUBLIC_HOST_TCBU}
OpenSSH_5.8p2_hpn13v11 FreeBSD-20110503, OpenSSL 0.9.8q 2 Dec 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to {PUBLIC_HOST_TCBU} [128.205.32.51] port 22.
debug1: connect to address 128.205.32.51 port 22: Operation timed out
ssh: connect to host {PUBLIC_HOST_TCBU} port 22: Operation timed out
build#
Yet for an internal host they both function. I changed the port number to make sure that it was specifically that host.
Host:
Code: 
$ ssh 192.168.2.y -p 123123
The authenticity of host '[192.168.2.y]:123123 ([192.168.2.y]:123123)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? no
Host key verification failed.
$
Jail:
Code: 
build# ssh 192.168.2.y -p 123123
The authenticity of host '[192.168.2.y]:123123 ([192.168.2.y]:123123)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? no
Host key verification failed.
build#

Any ideas on a check where the data is dropping?

Short Version (tl;dr): I cannot talk to hosts outside of my firewall/gateway/router device. However, I can talk to internal hosts where data passes through that same firewall/gateway/router device. Why would this occur, are the other better methods of troubleshooting, and how can this be fixed?
 
Reading Network Traffic Help

kpa: Thank you, I put that in and now I get /var/log/pflog and data is being written to it.

It looks like the data is going out through two different interfaces. I thought that the em0 interface also went out to the internet, through a much more complex route, guess not.
For ease of reading, I cleared out a bunch of lines to my DNS pushing through on port 53. They looked like
Code: 
00:00:03.309444 rule 1..16777216/0(match): pass out on em1: 192.168.2.22.55927 > 192.168.2.1.53: 36811+[|domain] 
00:00:00.111359 rule 1..16777216/0(match): pass out on em1: 192.168.2.22.53318 > 192.168.2.1.53: 36812+[|domain] 
00:00:00.037753 rule 1..16777216/0(match): pass out on em1: 192.168.2.22.63796 > 192.168.2.1.53: 36813+[|domain]
Code: 
$ sudo tcpdump -n -e -ttt -r /var/log/pflog
 reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file) 
00:00:00.000000 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13344 > 192.168.2.22.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:00.087415 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.37101 > 127.0.0.1.25: Flags [S], seq 2967429600, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.000100 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.37101 > 127.0.0.1.25: Flags [S], seq 2967429600, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.443710 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 50.77.167.25.123: NTPv4, Client, length 48 
00:00:00.968095 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 4.53.160.74.123: NTPv4, Client, length 48 
00:00:00.997850 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 72.8.140.200.123: NTPv4, Client, length 48 
00:00:00.050509 rule 1..16777216/0(match): pass out on em1: 192.168.2.22.31981 > 192.168.2.1.53: 4625+[|domain] 
00:00:00.390287 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.28076 > 127.0.0.1.512: UDP, length 11 
00:00:00.000146 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.28076 > 127.0.0.1.512: UDP, length 11 
00:00:10.114356 rule 0..16777216/0(match): pass in on em1: 192.168.2.199.17500 > 192.168.2.255.17500: UDP, length 112 
00:00:00.039790 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13365 > 192.168.2.22.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
00:02:28.703303 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13384 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:00.579388 rule 0..16777216/0(match): pass in on em1: 192.168.2.199.17500 > 192.168.2.255.17500: UDP, length 112 
00:00:00.015000 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48 
00:00:00.993610 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.102.46.73.123: NTPv4, Client, length 48 
00:00:01.000821 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 128.113.28.67.123: NTPv4, Client, length 48 
00:00:00.120697 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.14206 > 127.0.0.1.25: Flags [S], seq 534643596, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.000173 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.14206 > 127.0.0.1.25: Flags [S], seq 534643596, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.046991 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.10183 > 127.0.0.1.113: Flags [S], seq 3508853374, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.000060 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.10183 > 127.0.0.1.113: Flags [S], seq 3508853374, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]>  
00:00:00.332565 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.56469 > 127.0.0.1.512: UDP, length 11 
00:00:00.000072 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.56469 > 127.0.0.1.512: UDP, length 11 
00:00:10.890908 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13405 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:25.001799 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13406 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:30.193636 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48 
00:00:00.061214 rule 1..16777216/0(match): pass out on em0: 192.168.31.202.25143 > 128.205.36.8.22: Flags [S], seq 3388894413, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:03.770650 rule 0..16777216/0(match): pass in on em1: 192.168.2.199.17500 > 192.168.2.255.17500: UDP, length 112 
00:00:07.257436 rule 0..16777216/0(match): pass in on em1: 192.168.2.199.6347 > 192.168.2.23.22: Flags [S], seq 594459889, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0 
00:00:01.646411 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 128.113.28.67.123: NTPv4, Client, length 48 
00:00:34.271266 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48 
00:00:27.994535 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 128.113.28.67.123: NTPv4, Client, length 48 
00:00:11.811251 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13434 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:00.188786 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.102.46.73.123: NTPv4, Client, length 48 
00:00:00.066390 rule 1..16777216/0(match): pass out on em0: 192.168.31.202.57652 > 128.205.36.8.22: Flags [S], seq 3697669325, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:04.775822 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48 
00:00:27.997807 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 128.113.28.67.123: NTPv4, Client, length 48 
00:00:12.006890 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.102.46.73.123: NTPv4, Client, length 48 
00:00:17.032580 rule 1..16777216/0(match): pass out on em1: 192.168.2.23.60577 > 192.168.2.y.123123: Flags [S], seq 3514457325, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.770766 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13455 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:06.192841 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48 
00:00:18.806186 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13456 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:10.191965 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 128.113.28.67.123: NTPv4, Client, length 48 
00:00:13.000908 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.102.46.73.123: NTPv4, Client, length 48 
00:00:01.810798 rule 0..16777216/0(match): pass in on em1: 192.168.2.1.13457 > 192.168.2.23.137: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST 
00:00:20.185607 rule 1..16777216/0(match): pass out on em0: 10.0.2.15.123 > 199.167.29.243.123: NTPv4, Client, length 48

Maybe I have /etc/pf.conf misconfigured, or pf uses em0 for some form of default: (I would have through from the config file pf would ignore em0)
Code: 
$ more /etc/pf.conf
ext_if="em1"                      # The external interface
int_if="em2"                      # The internal interface
external_addr="192.168.2.23"              # Your public IP address
internal_net="192.168.31.0/24"             # Your internal subnet
# Translation: specify how addresses are to be mapped or redirected.
# nat: packets going out through $ext_if with source address $internal_net will
# get translated as coming from the address of $ext_if, a state is created for
# such packets, and incoming packets will be redirected to the internal address.
nat on $ext_if from $internal_net to any -> ($ext_if)
# rdr: packets coming in on $ext_if with destination $external_addr:1234 will
# be redirected to 10.1.1.1:5678. A state is created for such packets, and
# outgoing packets will be translated as coming from the external address.
rdr on $ext_if proto tcp from any to $external_addr/32 port 80 -> 192.168.31.11 port 8080
rdr on $ext_if proto tcp from any to $external_addr/32 port 443 -> 192.168.31.11 port 8443
# Make sure we don't block any traffic
pass in log all
pass out log all

I see traffic from the jail twice, looks like it was allowed and no responce came back. I dont see anything going to
Code: 
00:00:00.061214 rule 1..16777216/0(match): pass out on em0: 192.168.31.202.25143 > 128.205.36.8.22: Flags [S], seq 3388894413, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.066390 rule 1..16777216/0(match): pass out on em0: 192.168.31.202.57652 > 128.205.36.8.22: Flags [S], seq 3697669325, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]>

I don't see where my jail is talking to the host it can communicate with over ssh. Maybe a logging delay, I'll try again in the morning.
Code: 
00:00:17.032580 rule 1..16777216/0(match): pass out on em1: 192.168.2.23.60577 > 192.168.2.y.123123: Flags [S], seq 3514457325, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS[|tcp]>

Also, not sure what all this loopback traffic is doing
Code: 
00:00:00.093792 rule 1..16777216/0(match): pass out on lo0: 127.0.0.1.40357 > 127.0.0.1.25: Flags [S], seq 1671919100, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]> 
00:00:00.000105 rule 0..16777216/0(match): pass in on lo0: 127.0.0.1.40357 > 127.0.0.1.25: Flags [S], seq 1671919100, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS[|tcp]>

Anyway, this still makes me think its not at this level traffic is being dropped. Any troubleshooting suggestions?
 
You sure are going around the mulberry bush to do something which is very simple.
Or put another way “your barking up the wrong tree".

I recommend you replace ezjail with qjail because qjail does auto alias for you and ezjail does not. That alone will simplify a great deal what you have coded already.
Replace pf with ipfilter, rules syntax are almost the same, but ipfilter has a log you can understand without running dump on it, and NAT is so much simpler to define.
 
fbsd1 said:
because qjail does auto alias for you and ezjail does not.
Click to expand...
What does you mean with "auto alias"?

Well, I don't know how qjail works. ezjail is just setting "jail_<jname>_ip". It's up to you to add "jail_<jname>_interface" to the jail config file, or set a (default) "jail_interface" in "/etc/rc.conf". And that's enough to have an alias created at jail startup and removed at jail shutdown.
 
You must log in or register to reply here.
Back
Top