VNET jail with IPFW won't log

J

jtocci

I am confused. I have a working vnet jail with it's own ethernet connection. IPFW works. DNS works. pkg works. I can fetch websites.
But for some reason I get no logs in /var/log/security or in /var/log/ipfw.log

Here are a few of the log lines I have in my ipfw.rules:
Code: 
ipfw add 210 allow log tcp from me to any via ng0_web setup keep-state
ipfw add 211 allow log udp from me to any via ng0_web keep-state
ipfw add 213 allow log udp from me to 192.168.88.1 53 xmit ng0_web keep-state

Anyone have any hints for me? I've checked the handbook but it seems IPFW should work the same in a VNET jail as on host.
I've been all over the freebsd forums, various tutorials. All the information is stuff I've check over and over now.
Any help will be very appreciated.

% ifconfig
Code: 
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x8
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ng0_web: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=28<VLAN_MTU,JUMBO_MTU>
        ether 1c:86:0b:38:88:df
        hwaddr 58:9c:fc:10:30:b3
        inet 192.168.88.2 netmask 0xffffff00 broadcast 192.168.88.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

rc.conf:
Code: 
firewall_enable="YES"
firewall_script="/etc/ipfw.rules"
firewall_logging="YES"

syslog.conf
Code: 
*.err;kern.warning;auth.notice;mail.crit                /dev/console
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err   /var/log/messages
security.*                              /var/log/security
auth.info;authpriv.info                 /var/log/auth.log
mail.info                               /var/log/maillog
cron.*                                  /var/log/cron
!-devd
*.=debug                                /var/log/debug.log
*.emerg                                 *
daemon.info                             /var/log/daemon.log
!ipfw
*.*                         /var/log/ipfw.log
-snip-

sysctl.conf:
Code: 
net.inet.ip.fw.verbose: 1
net.inet.ip.fw.verbose_limit: 1

host jail.conf:
Code: 
exec.timeout=120;
stop.timeout=120;
sysvmsg = new;          # prevent security issues with sysv memory being shared across jails and to host
sysvsem = new;          # leave all three uncommented to make sure shared memory is not used
sysvshm = new;          # this one also prevents problems with postgres starting
exec.clean;             # default jails will inherit environment vars from parent. this stops that.
mount.devfs;            # minimum devices list

# shortcuts
path="/j1459/${name}"; # set to mountpath of jail, not dataset location
                     # note: if jail fails to start, try `zfs mount zroot/jail/${name}`
host.hostname="${name}.example.com";
exec.consolelog="/var/log/${name}.jail";
devfs_ruleset="1000"; # vnet rule set

vnet.interface="ng0_${name}";
allow.mount;
allow.mount.zfs;
enforce_statfs=1;
exec.created="jmz start ${name} ${path}";
exec.prestart="jng bridge ${name} igb1";
exec.start="zfs mount -a";
exec.start+="/bin/sh /etc/rc";

exec.prestop="ifconfig ng0_${name} -vnet ${name} || echo";
exec.stop="/bin/sh /etc/rc.shutdown";
exec.poststop="jng shutdown ${name}";
exec.poststop+="ngctl shutdown ng0_${name}";

web {
        vnet;
        allow.raw_sockets;
        allow.socket_af;
        exec.prestart="sudo rm -f /j1459/web/var/run/wpa_supplicant/ng0_${name}";
        exec.prestart+="jng bridge ${name} =igb1";
        host.hostname = "web.example.com";
}
 
I got VNET jail IPFW logging and monitoring working on FreeBSD 15.1-RELEASE-p3.

Logging to a file (syslog) will log to the host's log files.
Logging to ipfw0 will output to the jails's ipfw0 interface.

I do not know what your devfs_ruleset="1000" contains, but it will need to contain
Code: 
add path 'bpf*' unhide
.


To log to the security log (on the host, despite the jail doing the logging):
In the jail:
Ensure
Code: 
firewall_logging="YES"
exists in /etc/rc.conf.
Restart ipfw if you made changes to rc.conf:
service ipfw restart
Ensure
Code: 
net.inet.ip.fw.verbose=1
exists in /etc/sysctl.conf (ipfw()).
Reload sysctl variables if you made changes to sysctl.conf:
service sysctl reload
To view the log in real time:
On the host:
tail -f /var/log/security
(or whatever log file you have configured in /etc/syslog.conf on the host).

Alternatively, to output to jail's ipfw0 interface:
In the jail:
Ensure
Code: 
firewall_logging="YES"
and
Code: 
firewall_logif="YES"
exist in /etc/rc.conf (ipfw()).
Restart ipfw if you made changes to rc.conf:
service ipfw restart
Ensure
Code: 
net.inet.ip.fw.verbose=0
exists in /etc/sysctl.conf (ipfw()).
Reload sysctl variables if you made changes to sysctl.conf:
service sysctl reload
To monitor in real time:
On the jail:
tcpdump -i ng0_${name} -v -X
(You can play around with the tcpdump options as per tcpdump(), or use some other monitoring programme).

Or, if you wish to both log via syslog facility and monitor via the jail's ipfw0 interface then use IPFW's logdst (ipfw()) to specify both loging locations.

I hope this helps.
 
Thank you graue.Ritter, I totally missed that despite reading the first entry for log [logamount number].

I'm going to work with this and post my final solution. I appreciate your time.

This is the section I'm focusing on now
Code: 
log [logamount number] logdst logdst_spec
             logdst_spec is a comma-separated list of log destinations for
             logging packets matching the rule.  Destinations supported are:

             syslog  Logs a packet to syslogd(8) with a LOG_SECURITY facility.

             ipfw0   Logs a packet to the ipfw0 pseudo interface.

             rtsock  Logs a packet to the route(4) socket.  See the comments
                     of ipfw_log_rtsock() in ipfw source code for more
                     information on the message's structure.

             Note: logamount limits a number of logging events rather than
             packets being logged.  I.e. A packet matching a rule with

                    ...  log logamount 100 logdst syslog,ipfw0 ...

             will log upto 50 packets.

The man page refers to ipfw_log_rtsock() comments in the IPFW source code:
Code: 
/*
 * Logs a packet matched by a rule as a route(4) socket message.
 *
 * While ipfw0 pseudo interface provides a way to observe full packet body,
 * no metadata (rule number, action, mark, etc) is available.
 * pflog(4) is not an option either as it's header is hardcoded and does not
 * provide sufficient space for ipfw meta information.
 *
 * To be able to get a machine-readable event with all meta information needed
 * for user-space daemons we construct a route(4) message and pack as much meta
 * information as we can into it.
 *
 * RTAX_DST(0): (struct sockaddr_dl) carrying ipfwlog_rtsock_hdr_v2 in sdl_data
 *        with general rule information (rule number, set, action, mark,
 *        cmd, comment) and source/destination MAC addresses in case we're
 *        logging in layer2 pass.
 *
 * RTAX_GATEWAY(1): (struct sockaddr) IP source address
 *
 * RTAX_NETMASK(2): (struct sockaddr) IP destination address
 *
 * RTAX_GENMASK(3): (struct sockaddr) IP address and port used in fwd action
 *
 * One SHOULD set an explicit logamount for any rule using rtsock as flooding
 * route socket with such events could lead to various system-wide side effects.
 * RTF_PROTO1 flag in (struct rt_addrinfo).rti_flags is set in all messages
 * once half of logamount limit is crossed. This could be used by the software
 * processing these logs to issue `ipfw resetlog` command to keep the event
 * flow.
 *
 * TODO: convert ipfwlog_rtsock_hdr_v2 data into TLV to ease expansion.
*/
 
You must log in or register to reply here.
Back
Top