Example jail configuration causes host to lose network

Wout

New Member


Messages: 11

Using FreeBSD 10-RELEASE, when following the jail(8) jail.conf example, it causes the host to lose network when starting the jail.

Code:
testjail {
   path	= /tmp/jail/testjail;
   mount.devfs;
   host.hostname = testhostname;
   ip4.addr = 192.0.2.100;
   interface = ed0;
   exec.start =	"/bin/sh /etc/rc";
   exec.stop = "/bin/sh	/etc/rc.shutdown";
}
The host is using one static IP address on the same interface as is passed to the jail.

After removing the interface option, everything works fine. It does not seem to be required.
Is this expected behaviour? If so, shouldn't this be removed from the example to avoid mistakes?

First time I try out the jail.conf method, luckily on a local machine :pP
 

wblock@

Beastie Himself
Developer

Reaction score: 3,680
Messages: 13,851

Does the host actually have an ed0 interface?
 
OP
Wout

Wout

New Member


Messages: 11

No, but the above example is just a copy from jail(8). My version has path, host.hostname, ip4.addr and interface options changed.

To be complete, my jail.conf contains:
Code:
testjail {
	path = /usr/jails/test;
	mount.devfs;
	host.hostname = testhostname;
	ip4.addr = 10.0.2.15;
	interface = em0;
	exec.start = "/bin/sh /etc/rc";
	exec.stop = "/bin/sh /etc/rc.shutdown";
}
And my rc.conf contains:
Code:
ifconfig_em0="inet 10.0.2.15 netmask 255.255.255.0"
defaultrouter="10.0.2.2"
jail_enable="YES"
 

wblock@

Beastie Himself
Developer

Reaction score: 3,680
Messages: 13,851

So you mean that merely setting the interface in jail.conf causes the host's interface to go down?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,119
Messages: 33,688

Not surprisingly as the jail and the host want the same IP address. You have to set a different IP address on the jail. Think of the jail as a separate machine, you can't have two machines on the network with the same IP address.
 
OP
Wout

Wout

New Member


Messages: 11

@wblock@: Yes, it seems so. Without the interface option, everything works as expected. Please, try it, but not on a remote machine :)

@SirDice: I do not believe that to be true. A jail and host may share the same IP address. It is even required as the host has to have this IP assigned to a interface.
 
Last edited by a moderator:

usdmatt

Daemon

Reaction score: 551
Messages: 1,465

@SirDice: I do not believe that to be true. A jail and host may share the same IP address. It is even required as the host has to have this IP assigned to a interface.
You're wrong on the "required" front. The host and jail ideally need different IP addresses. There is no problem with assigning multiple IP addresses to one interface, which is what will happen. The interface will accept traffic to both addresses. Packets going to the IP assigned to the host will be delivered to the host, and packets received by the interface for the jail's IP will be passed to the jail.

Edit: Clarification needed - It does appear possible to run a jail on the same IP as the host, but I suspect you have to be very careful about the services running in the jail to make sure none overlap services running on the host. It makes sense to use different IP addresses in most cases.

Edit 2: You also need to make sure that any network services running on the host are restricted to just the host's IP. Looking on my machine for example it looks like ssh defaults to accepting connections on all addresses (*.22) which would need to be changed to only accept on the host's IP. Otherwise running ssh on both host and jail would cause problems.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,119
Messages: 33,688

Wout said:
A jail and host may share the same IP address.
Multiple jails can share the same IP address, the host cannot.

It is even required as the host has to have this IP assigned to a interface.
A jail can have zero or more IP addresses. It's not required to set an IP address.
 
OP
Wout

Wout

New Member


Messages: 11

I agree it is uncommon, and probably bad practice. Normally I also assign IP aliases to my jails. But it surely is possible. If the host and jail would use the same port on a shared IP address, the host will have preference. I do not find anything in jail(8), nor in the Handbook, that says it is not allowed to use the host's primary IP address.

Just found out that when I add a IP alias to the host, there is no problem using the interface option in jail.conf. But it does not matter which IP is assigned to the jail.

So it seems when using the interface option in jail.conf, it is required to have at least one alias assigned to that interface, or the interface goes down on jail start.

Edit: Typo
 
OP
Wout

Wout

New Member


Messages: 11

SirDice said:
Multiple jails can share the same IP address, the host cannot.
I am running a system doing the very same thing right now. Even without talking about the "primary" IP address, when a jail uses an IP address, it must the assigned to the host as well (as alias or primary). So they always share IP addresses. To use this IP for services on the host, is probably not a good idea.

SirDice said:
A jail can have zero or more IP addresses. It's not required to set an IP address.
I agree on this one.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,119
Messages: 33,688

Wout said:
SirDice said:
Multiple jails can share the same IP address, the host cannot.
I am running a system doing the very same thing right now. Even without talking about the "primary" IP address, when a jail uses an IP address, it must the assigned to the host as well (as alias or primary). So they always share IP addresses. To use this IP for services on the host, is probably not a good idea.
Having thought it through a bit more, you are absolutely right.
 
OP
Wout

Wout

New Member


Messages: 11

SirDice said:
Having thought it through a bit more, you are absolutely right.
Thanks. You do have a valid point. When using aliases, the interface problem does not occur.

I do not see any reason why the interface option is in the example, and as it causes the interface to go down when not having aliases, maybe it is best to have it removed from the jail(8) example?

Should this be reported to the jail maintainers? I believe this to be pretty dangerous §jr

Edit: Trying to follow those proper formatti rules.
 

kpa

Beastie's Twin

Reaction score: 1,822
Messages: 6,318

The example should have a clarification that the address used should be an interface alias if shared with the real host. Many parts of the handbook are unfortunately written in style and with the expectations that the reader knows his/her TCP/IP, not a good assumption in a handbook meant for beginners.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,119
Messages: 33,688

Wout said:
Should this be reported to the jail maintainers? I believe this to be pretty dangerous §jr
It won't hurt to report it. Looking at the bug reports I think you should file it under "Documentation". I see a lot more requests for changes in manual pages there.
 

nakal

Active Member

Reaction score: 37
Messages: 245

An interface alias will be automatically generated by the jail initialization routine. A jail start with an assigned IP will setup the interface new and cause all connections dropped on this IP. This is a well-known behavior. Not sure if it is a bug, because jails are contained within an assigned IP for a reason. Assigning the IP of the host will probably cause much more problems than just dropped connections. If this works at all, it will also be perhaps bad for security (haven't thought about this, because I don't run jails like this).

But I have the feeling that the OP has got a different problem, which he tries to solve by re-using an IP. (S)he hasn't told us what he is trying to do.
 

kpa

Beastie's Twin

Reaction score: 1,822
Messages: 6,318

nakal said:
An interface alias will be automatically generated by the jail initialization routine.
There are actually two syntaxes for specifying the address. This form will not create the interface alias automatically:

Code:
ip4.addr = 192.0.2.100;
This one will create the alias automatically on jail start up:

Code:
ip4.addr = ed0|192.0.2.100;
 
OP
Wout

Wout

New Member


Messages: 11

@nakal: No problem here, I was just trying out the jail.conf way of starting jails. I was doing this on a local machine, which does not have any aliases.

So if I understand correctly, when using the interface option, you do not need to have aliases assigned to the host before starting the jail. As they will be created by the jail command.
If you do not set the interface option, you need to create those aliases yourself.

This is all good, but the strange part is when you assign the primary address to the jail, the interface goes down on jail start except when you have assigned a random alias to the interface before the jail command runs. In this case, I can use the primary address on the jail.

Bad practice or not, the interface option is a little dangerous.
 
Last edited by a moderator:

wblock@

Beastie Himself
Developer

Reaction score: 3,680
Messages: 13,851

Trying to address several questions:

Setting interface and address separately behaves differently than using the "interface|address" form. Right? The entries on ip4.address and ip4.saddrsel imply there is some automatic behavior going on that might complicate this.

The Handbook tries to walk the line of not talking down to the reader while still being basic enough to cover most questions. It's not easy, and much of the time some knowledge has to be assumed. Because jails are not truly separate machines, networking is a little trickier there anyway. And if we repeat the warning about not using duplicate IP addresses everywhere it might apply, it will be repeated all throughout the Handbook.

Yes, corrections to man pages should be filed as documentation bugs.
 
OP
Wout

Wout

New Member


Messages: 11

wblock@ said:
Setting interface and address separately behaves differently than using the "interface|address" form. Right? The entries on ip4.address and ip4.saddrsel imply there is some automatic behavior going on that might complicate this.
No, they behave the same. Adding the interface by either interface option or interface|address syntax in ip4.address results in automatic creating and destroying of IP aliases.

To problem lies when you (either by mistake, or on purpose) assign the host's primary IP address to the jail. Than the automatic behavior results in the interface going down. While this is certainly not recommended, it is a valid thing to do without using "automatic IP aliasing" by jail.

wblock@ said:
The Handbook tries to walk the line of not talking down to the reader while still being basic enough to cover most questions. It's not easy, and much of the time some knowledge has to be assumed. Because jails are not truly separate machines, networking is a little trickier there anyway. And if we repeat the warning about not using duplicate IP addresses everywhere it might apply, it will be repeated all throughout the Handbook.
The Handbook does not suggest using jail.conf, nor the interface option, only jail(8) does. But it might do so in the future.

wblock@ said:
Yes, corrections to man pages should be filed as documentation bugs.
I shall report it, and refer to this topic.
 
Top