Solved Hosting says I'm using fake MAC - why?

[oscar]

Hi,

So my hosting provider (Hetzner) says that I'm using foreign MAC addresses and have politely asked me to stop doing so, to which I would happily agree to do - but I can't for the life of me understand why this is happening.

I'm using FreeBSD-STABLE and a bunch of jails (VNET bridge based), they are NATed and currently allowed to do pretty much anything.

The MAC addresses they are sending me are:

### List from provider
02:c0:da:d8:80:fa
02:f0:c1:d8:80:fa
0e:f0:c1:d8:80:fa
# This is my real MAC
11:11:11:d8:80:fa

# From ifconfig
e0a_media: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:f0:c1:d8:80:fa
[...]
e0a_backup: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:c0:da:d8:80:fa

These are some relevant configuration files, with some obvious substitution on ports and external IP address.

First, my firewall configuration (which I'm suspecting is the main issue?):

localnet = "{ 10.0.10.0/30 10.0.20.0/30 10.0.30.0/30 10.0.40.0/30 10.0.50.0/30 10.0.70.0/30 }"

IP_PUB = "123.123.123.123"
IP_MEDIA = "10.0.10.2"
IP_BACKUP = "10.0.30.2"

int_tcp_services = "{ ssh http https }"
int_udp_services = "{ 321:321321 }"

set skip on lo0
set block-policy return

nat on re0 from $localnet to any -> re0
rdr pass on re0 proto tcp from any to $IP_PUB port 123123 -> $IP_MEDIA port 123123
rdr pass on re0 proto tcp from any to $IP_PUB port 123 -> $IP_BACKUP port 123

antispoof for re0

block in on re0
pass in quick on re0 inet proto tcp from any to $IP_PUB port $int_tcp_services
pass in quick on re0 inet proto udp from any to $IP_PUB port $int_udp_services
pass out all

And then my jail.conf:

path = "/usr/local/jails/$name";
mount.fstab = "/usr/local/jails/$name.fstab";
exec.consolelog = "/var/log/jail_${name}_console.log";
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;

vnet;
vnet.interface = "e0b_$name";

# bridge
exec.prestart += "jib addm $name re0";
exec.poststop += "jib destroy $name";

media {
    exec.prestart += "ifconfig e0a_$name 10.0.10.1/30";
}

Any ideas why this is happening?

 

[jbo@]

I have never used jails myself so please excuse if I am giving wrong information. However, assuming that jails work very similar to full blown VMs (in the scope of networking) you'll usually give the jails/VMs access to your network by either bridging your physical network link or by applying NAT. Your ifconfig indicates that you're using bridging as you seem to have alias interfaces / clones / virtual interfaces (please excuse for not knowing the proper terminology).

Whenever you bridge a network connection, the host that uses the bridge will basically see a real network connection on his side. Therefore, each jails thinks that it has its own NIC. For this to work, each virtual interface needs to have a different/separate MAC address.
It is very common for hosters/datacenters to filter for invalid/foreign MAC addresses to prevent spoofing and the like.

You might want to consider using NAT instead.

 

[oscar]

I have never used jails myself so please excuse if I am giving wrong information. However, assuming that jails work very similar to full blown VMs (in the scope of networking) you'll usually give the jails/VMs access to your network by either bridging your physical network link or by applying NAT. Your ifconfig indicates that you're using bridging as you seem to have alias interfaces / clones / virtual interfaces (please excuse for not knowing the proper terminology).

Whenever you bridge a network connection, the host that uses the bridge will basically see a real network connection on his side. Therefore, each jails thinks that it has its own NIC. For this to work, each virtual interface needs to have a different/separate MAC address.
It is very common for hosters/datacenters to filter for invalid/foreign MAC addresses to prevent spoofing and the like.

You might want to consider using NAT instead.

Hi,

Thanks for your reply! I hadn't realized this would be the implication of using the bridged solution I went with, I thought that everything would go out through my re0 interface and not take any other route, especially as I'm already using nat for these connections (as indicated in my pf.conf).

I'll read up a bit on the alternatives for having my jails reach the internet. I'd like to avoid having to redo too much setup but I'm guessing I have no choice.


Thanks again,
Oscar

 

[oscar]

Just a final update - yes, it was the bridge that was the issue. I've corrected this and my machine is up and running again.

Just in case anyone else would find this and wonder about the solution, I went with a simple cloned interface ( lo1) and NAT for that interface.

Snippet from rc.conf:

cloned_interfaces="lo1"
ifconfig_lo1_alias0="inet 10.0.10.1 netmask 255.255.255.252"
# add interfaces and/or aliases as required

Snippet from jail.conf:

media {
    # exec.prestart += "ifconfig e0a_$name 10.0.10.1/30";
    ip4.addr = 10.0.10.1;
}

I was curious about using netgraph like some others have done, but I also just wanted it to work again so I went with this solution.