Loopback / Ifconfig output different in jail

z662

z662

Hello,

Recently I ran into an issue with DNS not working in a jail. The full story can be read here: https://forums.freebsd.org/threads/dns-inside-jail-not-working.49695/page-1

To make a long story short however, I purchased an Xbox One about 2 weeks ago and saw that they had a Plex app. I installed the app but was unable to get it to find my Plex server (FreeBSD Jail) since it was an outdated version. I then wanted to upgrade the plexmediaserver, so I needed to use the pkg command or build it via ports. I soon discovered that my DNS was failing and after posting on here (that thread above) we discovered that my DNS was failing because I needed to enable unbound.

The reason I mention this is because of those changes (perhaps), and information that was discovered in the thread above numerous people mentioned that my broadcast/subnet information for my interfaces was a bit peculiar (note the netmask differences and lack out output in the jail). Now after I have upgraded the plex server in the jail it cannot be found and the logs indicated that I have no loopback interface.

I have a parent/host FreeBSD 10.0 install with 3 jails running under it. One of which is the plex server. I have posted the results below of both the host and the jail output from ifconfig.

Code: 
brad@mercury:/home/brad$ ifconfig
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether b8:97:5a:23:26:32
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (none)
  status: no carrier
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether 00:14:d1:2b:9c:b5
  inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
  inet 192.168.0.104 netmask 0xffffffff broadcast 192.168.0.104
  inet 192.168.0.103 netmask 0xffffffff broadcast 192.168.0.103
  inet 192.168.0.102 netmask 0xffffffff broadcast 192.168.0.102
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
brad@mercury:/home/brad$ jls
  JID  IP Address  Hostname  Path
  1  192.168.0.104  plexJail  /usr/jails/plexJail
  2  192.168.0.103  ircJail  /usr/jails/ircJail
  3  192.168.0.102  apacheJail  /usr/jails/apacheJail
brad@mercury:/home/brad$ sudo jexec 2 sh
# ifconfig
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether b8:97:5a:23:26:32
  media: Ethernet autoselect (none)
  status: no carrier
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether 00:14:d1:2b:9c:b5
  inet 192.168.0.103 netmask 0xffffffff broadcast 192.168.0.103
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
# exit
 
You have a loopback interface in the jail, only there is no address configured. It is the lo0 entry.

There are several solutions which come on my mind:
- Set an additional alias for the lo0 interface like ifconfig lo0 alias 127.0.0.2/32
- Clone new loopback for given jail, see begining of https://www.freebsd.org/doc/handbook/jails-ezjail.html
- Check Plex docs/config, what is loopback used for and if another address/interface may be used
- Try this https://lists.freebsd.org/pipermail/freebsd-jail/2013-June/002326.html
- Go with the VIMAGE (network virtualization, still experimental)
 
Thanks for the reply. I do not wish to try VIMAGE. I want to get to the bottom of this problem

Can you expand a bit on your first suggestion with the alias? I think that may be heading in the right direction since I was told I should be cloning lo0 but I am not sure about which address it should be using. I assumed they should all be on 127.0.0.1 but could easily be wrong. I am not sure yet what all Plex needs the loopback for but it is throwing an error in debug mode and I have been told by a seasoned Plex user that I need to focus on that first. I would imagine it handles the services and network processing.

I went ahead and cloned my interfaces but just see the same output from ifconfig except for the "RUNNING" flag. Both interfaces appear in the jail but still no address assigned.

Code: 
# ifconfig
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether b8:97:5a:23:26:32
  media: Ethernet autoselect (none)
  status: no carrier
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether 00:14:d1:2b:9c:b5
  inet 192.168.0.104 netmask 0xffffffff broadcast 192.168.0.104
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
lo1: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>

Can I just simply set the lo1 address in my /etc/rc.conf of the jail, or do I need to update a few things on the host OS too?
 
z662 said:
I assumed they should all be on 127.0.0.1 but could easily be wrong.
Click to expand...
The whole 127.0.0.0/8 block is assigned to loopback use, how given software will cope with an address other than 127.0.0.1 is another thing.

Given you have lo1 interface created, assign it 127.0.0.2 address in the hosts and set the ipv4.addr variable in the jail.conf file like to 127.0.0.2 192.168.0.104 or whatever your are using for LAN address for the jail. With such setting usage of undefined address (127.0.0.1 here) should silently use the first address defined for given jail, which is 127.0.0.2 here, ending on the lo1 interface, not interfering with the hosts loopback traffic.

I am writing from top of my head and man page only, so maybe little syntactic tweaking will be needed.
 
Any time a jail asks for 127.0.0.1, it gets the address that was assigned to the jail, 192.168.0.104 in the case. Normally things just work. However, Plex may be sensitive to something here since it didn't get the address that it technically asked for.

You can add more than one address to a jail. Here is an example of BIND using both the loopback and external interface. That may give the loopback address Plex is looking for.
https://www.freebsd.org/doc/en_US.I...s-ezjail.html#jails-ezjail-example-bind-steps
 
I am a little hesitant to try the instructions in that link. I do not understand exactly what is going on in there... I am concerned if there is anything specific that needs to be adjusted for my setup I will not know to do it and will cause more problems for myself. It just seems like the output of my ifconfig(8) within my jails should be showing an address in the 127.0.0.x range like on the host machine. Maybe I am missing something but it seems like it might be a quick/easy win. How can I set that? My jail config file only has a mentioning of re1 and its address and my other files like/etc/pf.conf and /etc/rc.conf do not have anything specific to network interfaces. I do not see anything online regarding hardcoding a loopback interface. Like you said it is probably not necessary in most cases.

Let me know what your thoughts are. Thanks
 
Well, a search for it did turn up a result from the PC-BSD bug tracker (link below). You're not alone in seeing this issue. It does look like from the logs shown that attempting to connect to 127.0.0.1 is hard coded in. The odd thing is when a jail tries to connect to 127.0.0.1 it gets the address of the jail. There must be some feature in Plex that is actually looking at what is replying back. I'm not sure how things would behave if you try another 127.0.0.x address verse giving the jail access to 127.0.0.1.

https://bugs.pcbsd.org/issues/7045

Here's what I'm thinking:
/usr/local/etc/ezjail/<jailname>.
Code: 
export jail_plex_ip="192.168.0.104"
Change the above to look like what is below.
Code: 
export jail_plex_ip="192.168.0.104,127.0.0.1"
 
I have the interface name included in there as well. Let me know if I should remove that or how you recommend I modify what I currently have. Thanks

Code: 
export jail_plexJail=ip="re1|192.168.0.104"
 
I would be curious if this works first. This is basically what is in the handbook.
ifconfig lo1 create

Along with the ezjail config in /usr/local/etc/ezjail/<jailname>.
Code: 
export jail_plexJail_ip="re1|192.168.0.104,lo1|127.0.2.1"

If you are getting the exact same error as right now, than you can try with 127.0.0.1 as it looks like they did in the PCBSD bug. The interface, pipe, IP syntax makes it assign and remove the IP when the jail starts and stops so you just just leave off the lo interface have a 127.0.0.1.
 
Yeah still not working...a few things of interest here though:
- tried both 127.0.0.1 and 127.0.2.1;
- when trying to run ifconfig lo1 create it returns:
Code: 
ifconfig: create: bad value
.

I assume I should leave my lo0 interface cloned in my /etc/rc.conf correct?

A few things I read about this error/issue seem to be resolved when people fix their subnet so they are on the same subnet. I think I am though...but maybe not the loopback. My PC and the host server are on the same LAN, same switch etc. and my PC's subnet mask is 255.255.255.0. Below is my updated output from ifconfig(8)

Code: 
brad@mercury:/home/brad$ sudo ifconfig lo1 create
Password:
ifconfig: create: bad value
brad@mercury:/home/brad$ ifconfig
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether b8:97:5a:23:26:32
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (none)
  status: no carrier
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether 00:14:d1:2b:9c:b5
  inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
  inet 192.168.0.104 netmask 0xffffffff broadcast 192.168.0.104
  inet 192.168.0.103 netmask 0xffffffff broadcast 192.168.0.103
  inet 192.168.0.102 netmask 0xffffffff broadcast 192.168.0.102
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet 127.0.0.1 netmask 0xffffffff
  nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
brad@mercury:/home/brad$

and from within the jail

Code: 
# ifconfig
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether b8:97:5a:23:26:32
  media: Ethernet autoselect (none)
  status: no carrier
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
  ether 00:14:d1:2b:9c:b5
  inet 192.168.0.104 netmask 0xffffffff broadcast 192.168.0.104
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet 127.0.0.1 netmask 0xff000000
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet 127.0.0.1 netmask 0xffffffff
 
I think I may just delete the jail and start from scratch... I used portdowngrade and tried several different versions, none of which are working like they used to. I am just wondering if anything in the jail got messed up per that DNS thread I posted.

Might be a good idea to just dump it and rebuild. If it doesn't work though, I will be completely out of ideas and will start to question the configuration of my jails and host system. All other networking services and jails work fine though, it is just plex that is being a pain.
 
Interesting... I cannot tell though if that affects me or not. I do not have multiple subnets, but do have multiple cards (although only one is configured/up) and there are numerous jails. Plex runs in its own jail so not sure where I fall into that or not.

That bug looks like it has been around for a while... I am running 10.0 currently.

The only thing I will say is that everything worked fine on the older version of Plex and before I started tinkering around with stuff per that DNS thread (see first post). I downgraded Plex and still no dice. I almost want to say it has to do with the DNS changes, or perhaps packages that were required to be updated in the process.

I am so confused any more though lol... just want the stupid thing to work. VERY tempted to start a new jail and see what happens. I have a feeling it won't work though. I am a pessimist and just have that feeling something is broken.
 
ondra_knezour said:
You have a loopback interface in the jail, only there is no address configured. It is the lo0 entry.

There are several solutions which come on my mind:
- Set an additional alias for the lo0 interface like ifconfig lo0 alias 127.0.0.2/32
- Clone new loopback for given jail, see begining of https://www.freebsd.org/doc/handbook/jails-ezjail.html
- Check Plex docs/config, what is loopback used for and if another address/interface may be used
- Try this https://lists.freebsd.org/pipermail/freebsd-jail/2013-June/002326.html
- Go with the VIMAGE (network virtualization, still experimental)
Click to expand...

Looks like I may need to try VIMAGE after all... A Plex developer has recommended it for FreeBSD jails running Plex.

Does anyone have any thoughts on that as it is experimental? I guess it would be worth a shot. I just have never used it and want to make sure it wont cause other issues or be difficult to secure. Also, I would need to recompile my host kernel and THEN set up all of my jails again too wouldn't I? If so that may not be worth the hassle.
 
You must log in or register to reply here.
Back
Top