ng_nat with ng_ether on one interface

S

Sartrik

Hi.

I have the following problem, when I am starting the Virtualbox VM in bridged networking mode or creating a bridge between vboxnet0 and re0 (external interface) the ng_nat stops working, (NAT'ing) translating the incoming traffic to jail. For current configuration I have only one jail where Apahce plus PHP working and which does not have any environment/world (even sh) except required libraries.

I have found out that when VM starts the additional node appeara that is created by Vbox:
Code: 
# ngctl ls
There are 7 total nodes:
  Name: ipfw            Type: ipfw            ID: 00000001   Num hooks: 2
  Name: re0             Type: ether           ID: 00000002   Num hooks: 2
  Name: ipfw0           Type: ether           ID: 00000003   Num hooks: 0
  Name: natA            Type: nat             ID: 00000005   Num hooks: 2
  Name: ngctl1511       Type: socket          ID: 00000017   Num hooks: 0
  Name: vboxnet0        Type: ether           ID: 0000000e   Num hooks: 0
  [B]Name: vboxnetflt_re0  Type: vboxnetflt      ID: 0000000f   Num hooks: 2[/B]
Code: 
# ngctl show vboxnetflt_re0:
  N[B]ame: vboxnetflt_re0  Type: vboxnetflt      ID: 0000000f   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  output          re0             ether        00000002        upper
  input           re0             ether        00000002        lower[/B]

# ngctl show natA:
  Name: natA            Type: nat             ID: 00000005   Num hooks: 2
  Local hook      Peer name       Peer type    Peer ID         Peer hook
  ----------      ---------       ---------    -------         ---------
  in              ipfw            ipfw         00000001        61
  out             ipfw            ipfw         00000001        60
I think that this node causes the NAT to fail but I do not know how to fix that. I am more Electronic Engineering rather than IT specialist.

Any ideas?

Configurations:
jail:
Code: 
[]# jls
   JID  IP Address      Hostname                      Path
     1  192.168.0.1     apache.jail                   /usr/jail/apache
ifconfig
Code: 
re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
        ether d4:XX:7e:XX:d9:XX
        inet XXX.YY.ZZ.232 netmask 0xffffffe0 broadcast XXX.YY.ZZ.255
        inet6 XXX XXX XXX XXX%re0 prefixlen 64 scopeid 0x3
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
plip0: flags=8851<UP,POINTOPOINT,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 192.168.0.254 netmask 0xffffff00
        inet 192.168.0.1 netmask 0xffffffff
        inet 192.168.0.100 netmask 0xffffffff
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vboxnet0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 0a:00:27:00:00:00
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
ng_nat
Code: 
ngctl mkpeer ipfw: nat 60 out
ngctl name ipfw:60 natA
ngctl connect ipfw: natA: 61 in
ngctl msg natA: redirectaddr '{ local_addr=192.168.0.1 alias_addr=XXX.YY.ZZ.232 description="apache.jail" }'
ngctl msg natA: redirectport '{ local_addr=192.168.0.1 local_port=80 alias_port=33211 proto=6 }'
ngctl msg natA: setaliasaddr XXX.YY.ZZ.232

ipfw add 10060 netgraph 61 log tcp from any to XXX.YY.ZZZ.232 33211 in via re0
ipfw add 05060 netgraph 60 log tcp from 192.168.0.0/24 to any out via re0
 
I have found out similar or actually somthing that looks like my situation:

Julian Elischer wrote:
Rajkumar S wrote:
Hi,

I am trying to connect an ng_nat node between ng_ether:upper and
ng_ether:lower so that all packets traversing via ng_ether node gets
nat'd. But it does not seems to be working.

I am using the follwoing commands to connect rl1: and ng_nat

# ngctl mkpeer rl1: nat upper out
# ngctl name rl1:upper nat
# ngctl connect rl1: nat: lower in
# ngctl msg nat: setaliasaddr x.x.x.x​
ng_nat assumes you have IP packets
ng_ether gives you ethernet packets.
​
I should add, that you probably want to divert the IP packets
from the IP layer by using ipfw, a divert socket, and a netgraph
divert ksocket,
(Or a ipfw netgraph ipfw rule if you are using 7.0, I don't believe it is in 6.x)

(I have also swapped out with in in the above set of commands, just in
case, but with same results)

with warm regards,
Click to expand...
But how can I divert that in IPFW? I can see in IPFW that the counter counts matching rule and does send packets to netgraph but they do not reach the destination.
 
Unfortunately, I did not find any solution how to make it work, but I have found another way how to combine NAT and ng_ether bridging/bridging. The solution is to use instead of ng_ipfw+ng_nat, use PF NAT.

Another possible solution is to install nginx out of the jail and run as proxy or use any other TCP redirect software or install second external NIC.

It seems that when VirtualBox creates vboxnetflt_re0, the nodes re0 ipfw stops working. So the tcpdump showed that the packets were caught by netgraph ipfw rules but were not nat'd properly.
 
You must log in or register to reply here.
Back
Top