jails Unexpected behavior with Jail on Plex

M

mbze430

I am on 13.2 FreeBSD. I use Bastille to create the jail. I used ix1 as my interface. ix0 is the 'LAN' host interface.

I can reach the http://<ip>:32400/web interface and configure all that. However, when I play something it doesn't seem to play. From what I gather is that it might be accepting inbound traffic on ix1, but it's sending outbound traffic on ix0 instead.

Anyone else ran into anything like this? I pretty sure I haven't configured the network jail part correctly, as I am newb with FreeBSD's jail.
 
Are ix0 and ix1 on the same network? That's going to cause issues with routing. If you want the plex jail available on the LAN just bind it to ix0.
 
Ahh okay! I was planning to put ix1 port on another VLAN but I wanted to test it before I moved it to the VLAN it was intended for. I will do that and give it a try and report back.
 
how do I assign a gateway to the interface in the jail? I changed it to the vlan it supposed to be. Again, I can access the http://<new ip>:32400/web it is now on VLAN 253 and my main VLAN is 1000.
 
In your jail's /etc/rc.conf, you can add the following line to add a gateway:

Code: 
defaultrouter="192.168.0.1"

Is your jail using VNET?
 
Yeah I have been reading about FIB so that each "interface"/subnet has a different separate "table". However, since I am using XigmaNAS.... I haven't been able to figure it out, and there is zero help from their forums.


So I am trying to re-created what the guys are doing on the FreeBSD forum....but I am not sure what is the right syntax....

Following this thread:

cloned_interfaces="lagg0 vlan100 vlan60"
Click to expand...

Is the above line necessary, minus the lagg/lacp? (since I am not creating a portchannel)

ifconfig_vlan100="vlan 100 vlandev lagg0"
ifconfig_vlan60="vlan 60 vlandev lagg0"
Click to expand...

The sample uses the above syntax to create the vlan and bind it to the interface.

However, in the XigmaNAS 'rc.conf' it uses:
Code: 
vlans_em0="myvlan"
create_args_myvlan="vlan 102"

do I use the line below to create and bind in XigmaNAS?
Code: 
ifconfig_myvlan="vlan <id> vlandev <interface>" ??
 
Why do you think you need more/separate routing tables? I don't think I ever needed more than the 'standard' single routing table. I've only had some edge cases that required some creative routing, but most of the time things work fine with a single table on the host.

Big question however is, are those VNET jails or "plain" jails? There is a difference in how they are set up and who does the actual routing.
 
Well, our file server, it has 2x 1Gb NIC and 4x 10Gb NICs and 2x 50Gb NIC.

the 1Gb is on our management VLAN (254) 10.0.254.235/24. This is how we get into the XigmaNAS management webgui. There is also a IPMI/LoM as well on the second 1Gb.

2x 10Gb interface we are trying to use it with SMB and NFS. SMB VLAN (80) 10.0.80.235/24 We have other Windows Server within that same VLAN serving files. Same goes for NFS VLAN (90).... etc... for our Linux workstations.

We have the 2x 50Gb as iSCSI path.

That's the high level of what the box is like.

What we have found out is that outbound traffic is ALWAYS being sent out of the 1Gb, so none of the clients are getting the 10Gb. Also the 50Gb iSCSI path... it will corrupt the datastore (VMWare).

The whole jail thing.... I was told by the people at XigmaNAS that it would "fix my issue" or... "please do a jail to do that". So out of curiosity I created a jail running plex on it... but even then all the traffic will come out of the 1Gb.

SAwJPfq.png
 
mbze430 said:
What we have found out is that outbound traffic is ALWAYS being sent out of the 1Gb
Click to expand...
That depends on where the default gateway is set to. If your default gateway is an address on the 10.0.254.0/24 network, then yes, that's expected. With a single routing table there can only be ONE default gateway. But with some cleaver static routes you might be able to change this, there's no need for more than one default. On the other hand, I've had a similar issue with a Juniper firewall, and had to resort to creating two routing instances. One routing instance was for the firewalling (it excluded the management interface), and one routing instance was specifically for the management interface.

mbze430 said:
2x 10Gb interface we are trying to use it with SMB and NFS. SMB VLAN (80) 10.0.80.235/24 We have other Windows Server within that same VLAN serving files. Same goes for NFS VLAN (90).... etc... for our Linux workstations.
Click to expand...
It begs the question though, what takes care of routing on your network? Because there's connectivity between your clients (I assume they are on a different VLAN too) and the SMB/NFS VLANs.

mbze430 said:
Also the 50Gb iSCSI path...
Click to expand...
Are source and destination on the same VLAN? If they are on the same 'directly connected' network then your default gateway isn't important here, they'll be able to directly communicate with each other.
 
Yes the 10.0.254.0 is using the .1 as gateway.

All the vlan are on the core switch SVI table/database

As for the iscsi path... I think the VMware side asks for a gateway... not 100% sure.
 
You may need to configure a gateway, just as a typical network configuration. But if both sides are on the same network (i.e. broadcast domain) then that gateway is never actually used. Each interface always creates an implied route.
 
so my question is....
XigmaNAS uses a webGUI and if you don't use it it doesn't stay persistent...(technically speaking you can edit the config.xml file)

When I create a vlan it adds something like this to the rc.conf
ifconfig_ix0="inet 10.0.253.237/24 -wol"
ifconfig_ix1="inet 10.0.80.237/24 -wol"
vlans_ix1="vln80"
create_args_vln80="vlan 80 vlanproto 802.1Q"


If I were to follow that one guide on separate FIB
it uses something like this:
vlans_em0="101"
ifconfig_em0_101="inet 192.0.2.1/24"

There is a part of XigmaNAS that i can manually put in variables to the rc.conf that will stay persistent.
However, I don't know what is right way
would I be using ifconfg_mylvan="" context?
 
This is what I used in the past in my /etc/rc.conf:

Code: 
cloned_interfaces="igb0_vlan80"
ifconfig_igb0_vlan60="inet 192.168.80.2/24 vlan 80 vlandev re0"
ifconfig_igb0="up"

Is this what you mean?
 
Yeah I wasn't sure if the vlan created by XigmaNAS's way will translate. I see the context in the manual for rc.conf and it show both way of creating a VLAN.

To create a vlan device named em0.101 on em0 with the vlan
tag 101 and the optional the IPv4 address 192.0.2.1/24:

vlans_em0="101"
ifconfig_em0_101="inet 192.0.2.1/24"

To create a vlan device named myvlan on em0 with the vlan
tag 102:

vlans_em0="myvlan"
create_args_myvlan="vlan 102"
Click to expand...

vlans_ix1="vln80"
create_args_vln80="vlan 80 vlanproto 802.1Q"

just making sure that when I create the ifconfig_vln80 will be a correct syntax.

If not I guess I can remove what XigmaNAS created for the vlan in the gui and just throw it all in the rc.conf part.
system_advanced_rc.conf.png
 
I finally got some time to play around with this over the weekend. But still nothing. Apparently net.fibs is now in sysctl.conf and not in loader.conf

here is what I have in the rc.conf now:
Code: 
cloned_interfaces="vlan80"
ifconfig_vlan80="inet 10.0.80.235/27 vlan 80 vlandev cxl1"
static_routes="smbnet smbgate"
route_smbnet="-net 10.0.80.224/27 -interface vlan80 -fib 1"
route_smbgate="default 10.0.80.225 -fib 1"

but when I do a route show 10.0.80.224, it show fib:0
Code: 
datanas: ~# route show 10.0.80.224
   route to: 10.0.80.224
destination: 10.0.80.224
       mask: 255.255.255.224
        fib: 0
  interface: cxl1
      flags: <UP,DONE,PINNED>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0
datanas: ~# route show 10.0.80.225
   route to: 10.0.80.225
destination: 10.0.80.224
       mask: 255.255.255.224
        fib: 0
  interface: cxl1
      flags: <UP,DONE,PINNED>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0

Code: 
datanas: ~# netstat -nrF1
Routing tables (fib: 1)


Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.80.225        UGS      vlan80
10.0.80.224/27     link#8             US       vlan80
127.0.0.1          link#7             UHS         lo0


Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           URS         lo0
::1                               link#7                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           URS         lo0
fe80::/10                         ::1                           URS         lo0
ff02::/16                         ::1                           URS         lo0
datanas: ~# netstat -nrF0
Routing tables


Internet:
Destination        Gateway            Flags     Netif Expire
default            10.0.253.1         UGS        cxl0
10.0.80.224/27     link#4             U          cxl1
10.0.80.235        link#4             UHS         lo0
10.0.253.0/24      link#3             U          cxl0
10.0.253.235       link#3             UHS         lo0
127.0.0.1          link#7             UH          lo0


Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           URS         lo0
::1                               link#7                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           URS         lo0
fe80::/10                         ::1                           URS         lo0
fe80::%lo0/64                     link#7                        U           lo0
fe80::1%lo0                       link#7                        UHS         lo0
ff02::/16                         ::1                           URS         lo0

it looks like FIB 0 also contains the same routes. How can I stop it from putting it in FIB 0?
 
If I understand correctly, you have a Plex jail on the 10.0.80.0/24 network with a 10G interface, and your default gateway is on the 10.0.253.0/24 network, but with a 1G interface.
All your Plex client traffic will go through the 1G interface because that is where the default gateway is.

I'm guessing that your clients are not on the 10.0.80.0/24 network, but on some other network that communicates with your router?
If this is the case, you could create a VLAN of your client network on a 10G interface, put a bridge on it, and then connect your Plex jail to it via VNET (epair).

That way, your jail would have its own routing table, and traffic wouldn't have to be routed over the 1G link.
 
This originally wasn't a Plex or Jail thing. I was told by the guys on XigmaNAS what I was trying to accomplished was through a jail... but I couldn't find a way to run a SMB or NFS in a jail setting, so I decided to do Plex to see what the result was when it is ran in a jail.

My original and primary objective is to have SMB in VLAN80 and NFS on VLAN90. That will send out and receive on VLAN 80. I only did the Plex exercise to see if that was the case.

Where I work we do a LOT of video editing, and all the storage is on a 10Gb or 25Gb or 40Gb connections. All the video workstations are on 10Gb at least 90% of them.

We have about over 50+ editing stations.

Anyway I thought I was able to go the "cheap" route, but this is not panning out the way it supposed to.
 
You must log in or register to reply here.
Back
Top