Network unreachable inside jail

matthewp

matthewp

I have a jail which can't connect to the internet for some reason. Another jail is fine. Their config seems identical to me.
Code: 
# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
127.0.1.2          link#3             UH          lo1
192.168.1.51       link#1             UHS         lo0
Additionally this seems weird:
Code: 
# service routing stop
route: writing to routing socket: Operation not permitted
route: writing to routing socket: Operation not permitted
route: writing to routing socket: Operation not permitted
route: writing to routing socket: Operation not permitted
ifconfig: socket(family 28,SOCK_DGRAM: Protocol not supported
 
The jail's config ip has to match the alias ip in the base system. This is set in the alias line in the base system's rc.conf.

Inside the jail check that you have /etc/resolv.conf and /etc/hosts matching what you have in the other jail and basesystem.

Check the config file /usr/local/etc/jails/jailnamefile with the one in your other jail: this is where much of the jail's options are set.

According to the error message above see:
jail(8) jail.conf settings
Code: 
Jail Parameters
allow.*
  allow.raw_sockets
  The jail root is allowed to create raw sockets.  Setting
  this parameter allows utilities like ping(8) and
  traceroute(8) to operate inside the jail.  If this is
  set, the source IP addresses are enforced to comply with
  the IP address bound to the jail, regardless of whether
  or not the IP_HDRINCL flag has been set on the socket.
  Since raw sockets can be used to configure and interact
  with various network subsystems, extra caution should be
  used where privileged access to jails is given out to
  untrusted parties.
 
matthewp: When you say your jail cannot connect to the Internet, what have you tried? What did you expect and what did you get? Have you configured name resolution in your jail (/etc/resolv.conf)?

matthewp said:
Additionally this seems weird.
Code: 
# service routing stop
Click to expand...
Were you running that command inside the jail? Unless you are using a vnet jail (you will know if you are, otherwise you aren't), routing is managed on the host system.
 
asteriskRoss said:
matthewp: When you say your jail cannot connect to the Internet, what have you tried? What did you expect and what did you get? Have you configured name resolution in your jail (/etc/resolv.conf)?
Click to expand...

I'm trying to pkg install stuff. I do have /etc/resolv.conf that makes the other jail (and the host).


Were you running that command inside the jail? Unless you are using a vnet jail (you will know if you are, otherwise you aren't), routing is managed on the host system.
Click to expand...

I ran it inside the jail but I don't know if it should work or not, it's just something I tried, let's ignore that one for now.
 
Thanks everyone, I'm testing with host google.com. It does work, strangely. But nothing else seems to work. When I try to pkg install I get:
Code: 
# pkg install git
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/latest, please wait...
pkg: Error fetching http://pkg.FreeBSD.org/freebsd:10:x86:64/latest/Latest/pkg.txz: Network is unreachable
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.
 
Here is my host's rc.conf:
Code: 
hostname="webserver"
# ifconfig_em0="inet 192.168.1.150 netmask 255.255.255.0"
# ifconfig_em0_ipv6="inet6 accept_rtadv"
ifconfig_re0="inet 192.168.1.152 netmask 255.255.255.0"
ifconfig_re0_ipv6="inet6 accept_rtadv"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
# ezjail_enable="YES"
jail_enable="YES"
cloned_interfaces="${cloned_interfaces} lo1"
 
sidetone said:
The jail's config ip has to match the alias ip in the base system. This is set in the alias line in the base system's rc.conf.
Click to expand...

I don't have an alias stuff in my host rc.conf, is there docs on this?

Inside the jail check that you have /etc/resolv.conf and /etc/hosts matching what you have in the other jail and basesystem.
Click to expand...

I did check those, they are the same.

Check the config file /usr/local/etc/jails/jailnamefile with the one in your other jail: this is where much of the jail's options are set.
Click to expand...

I don't have a /usr/local/etc/jails folder at all. I initially setup my jails with ezjail but am not using that to manage them now. I am using an /etc/jail.conf file.
 
As far as documentation goes, there is the chapter on jails in the FreeBSD handbook, and the man pages for jail(8), jail.conf(5) and rc.conf(5).

Your migration from sysutils/ezjail to managing those jails outside the ezjail framework is a possible source of issues. ezjail sets things up in a particular way, using nullfs(5) to share the guts of what is needed for a jail between all jails and sticking the unique bits on top. It also manages much of the configuration for you, like assigning the appropriate aliases to the desired network interfaces. I suspect it is this configuration that ezjail was handling for you that you are now missing. Can you run ifconfig in both the host and the jail and post the outputs?

If you understood exactly how it all worked when you changed to using /etc/jail.conf then it could work, but in my view would be a strange choice. Section 15.5 of the FreeBSD handbook describes how to set something up by hand that is a bit like the ezjail configuration and might be a better place to start if that is what you want.

sysutils/ezjail is excellent and if you're just starting out with jails I would really recommend using it.
 
Ok, I'll try using ezjail. I originally migrated away from it because it hasn't been updated in over a year and a half, but maybe I should use it anyways.
 
Ok, deleted and recreated the bad jail with ezjail and see the same issue.

So now I find out that the problem is the host! I'm not able to update pkg there. However one of my jails is serving a website so I know there is some network. Here's what happens when I try to install curl from the host.

Code: 
root@webserver:~ # pkg install curl
Updating FreeBSD repository catalogue...
pkg: http://pkg.FreeBSD.org/FreeBSD:10:amd64/latest/meta.txz: Network is unreachable
pkg: repository FreeBSD has no meta file, using default settings
pkg: http://pkg.FreeBSD.org/FreeBSD:10:amd64/latest/packagesite.txz: Network is unreachable
pkg: Unable to update repository FreeBSD
All repositories are up-to-date.
The following 2 packages will be affected (of 0 checked):

New packages to be INSTALLED:
    curl: 7.41.0
    ca_root_nss: 3.17.4_1

The process will require 5 MiB more space.
2 MiB to be downloaded.

Proceed with this action? [y/N]: y
pkg: http://pkg.FreeBSD.org/FreeBSD:10:amd64/latest/All/curl-7.41.0.txz: Network is unreachable
 
The problem is in /etc/resolv.conf or /etc/hosts. Also, check that your gateway is listed in /etc/rc.conf. If it's wireless and not this, it could be wpa_supplicant's configuration, which I doubt.

Ping different things, from your gateway to outside the gateway with both the numerical ip and web address.
 
Ah, I didn't have a default gateway. Adding that seems to have fixed everything.... knock on wood. Thanks everyone for being patient and helping me.
 
You must log in or register to reply here.
Back
Top