1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

can't ping inside of jail

Discussion in 'Web and Network Services' started by kr651129, Nov 26, 2012.

  1. kr651129

    kr651129 New Member

    Messages:
    327
    Likes Received:
    0
    I think I'm missing something in my jail. The setup went fine but I don't have inet access inside my jail. I can ping the jail from the host system.

    Code:
    # jls
    JID   IP Address    Hostname    Path
     2    10.10.10.101  webserver   /usr/jails/webserver
    
    # ifconfig
    re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
            ether bc:5f:f4:59:97:a9
            inet6 fe80::be5f:f4ff:fe59:97a9%re0 prefixlen 64 scopeid 0x7
            inet 192.168.15.26 netmask 0xffffff00 broadcast 192.168.15.255
            inet 10.10.10.101 netmask 0xffffffff broadcast 10.10.10.101
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
            options=3<RXCSUM,TXCSUM>
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
            inet 127.0.0.1 netmask 0xff000000
            nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    
    # jexec 2 csh
    
    webserver# ifconfig
    re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
            ether bc:5f:f4:59:97:a9
            inet 10.10.10.101 netmask 0xffffffff broadcast 10.10.10.101
    ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
            options=3<RXCSUM,TXCSUM>
    ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
    
    webserver# ping www.google.com
    ping: cannot resolve www.google.com: Host name lookup failure
    
    Is there something I've missed?
     
  2. mamalos

    mamalos New Member

    Messages:
    427
    Likes Received:
    0
    Most probably you're not NAT-ing on your external interface, so the jail tries to ping using IPv4 address 10.10.10.101, the packet leaves your host's external interface and cannot be routed back, since nobody knows how to reply to 10.10.10.101 (except if your host machine is the gateway for the machine you're trying to ping, or the machine knows a route to 10.10.10.101/32, which I doubt). In short: Do you NAT on your external interface? If not, you may find difficulties pinging from inside the jail.
     
  3. kr651129

    kr651129 New Member

    Messages:
    327
    Likes Received:
    0
    I'm going to assume I don't NAT on the external interface since sysinstall setup the network for me when I first installed the system.
     
  4. chatwizrd

    chatwizrd New Member

    Messages:
    205
    Likes Received:
    0
    If you cant resolve the hostname then do you have your nameservers in /etc/resolv.conf
     
  5. kr651129

    kr651129 New Member

    Messages:
    327
    Likes Received:
    0
    I have nameservers in /etc/resolv.conf on the host, but they are the defaults that were setup at the first install.
     
  6. jaxxed

    jaxxed New Member

    Messages:
    10
    Likes Received:
    0
    This is a common networking problem in jails.

    From inside the host, what do you get from this:
    $/> netstan -rn

    You should get a list of all the routing rules for your system, including the IP on the jail.is there a routing rule that passes the jail net traffice to your network?

    A common implementation is to NAT your jails to you external IP, or to pick jail ips that make more sense on your host systen network. Many of us are using the loopback approach, where we creat a loopbakc device lo1 just for jails, put all the jails the itheir own subnet(s) on the loopback intercae and then NAT them to the external. Additionally you can alias any jail on your external interface. This approach gives lt's of firewalling/routing options inside the jail setup, jail to host and jail to network. It also works on laptops, with changing ips.
     
  7. kr651129

    kr651129 New Member

    Messages:
    327
    Likes Received:
    0
    jaxxed,
    Thanks for the reply. Here's my output

    Code:
    > netstat -rn
    Routing tables
    
    Internet:
    Destination        Gateway            Flags    Refs      Use  Netif Expire
    default            192.168.15.1       UGS         0   219743    re0
    10.10.10.101       link#7             UHS         0      254    lo0 =>
    10.10.10.101/32    link#7             U           0        0    re0
    127.0.0.1          link#10            UH          0       75    lo0
    192.168.15.0/24    link#7             U           0    67276    re0
    192.168.15.26      link#7             UHS         0        0    lo0
    
    Internet6:
    Destination                       Gateway                       Flags      Netif Expire
    ::/96                             ::1                           UGRS        lo0
    ::1                               ::1                           UH          lo0
    ::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
    fe80::/10                         ::1                           UGRS        lo0
    fe80::%re0/64                     link#7                        U           re0
    fe80::be5f:f4ff:fe59:97a9%re0     link#7                        UHS         lo0
    fe80::%lo0/64                     link#10                       U           lo0
    fe80::1%lo0                       link#10                       UHS         lo0
    ff01::%re0/32                     fe80::be5f:f4ff:fe59:97a9%re0 U           re0
    ff01::%lo0/32                     ::1                           U           lo0
    ff02::/16                         ::1                           UGRS        lo0
    ff02::%re0/32                     fe80::be5f:f4ff:fe59:97a9%re0 U           re0
    ff02::%lo0/32                     ::1                           U           lo0
    
    Do you have any good documentation/howto's that you can link me to for any of implementations you suggested?
     
  8. jaxxed

    jaxxed New Member

    Messages:
    10
    Likes Received:
    0
    I guess it depends on what jail system you're using. I'm using the PCBSD warden system, but you're likely using the EZJail system.

    Check this link:
    http://kbeezie.com/freebsd-jail-single-ip/

    The nice thing about this approach is that once you have it set up, it works for all jails automatically, even with network changes etc. It becomes low hassle when adding/removing jails.

    I treat all of my jails as a part of a host only internal network on lo1, and I nat that traffic out to the internet through my main interface. If I want a jail to have an external interface, then I manually pick an external interface IP and modify the jail to add the interface:
    $/> jail -m jid=# ip4.addr="{internal_ip}" ip4.addr="{new_external_ip}
    (I actually have modified the warden scripts for this)

    One note is that I don't think that you need to manually specifiy your external IP in the pf.conf, as you can specify the interface and use ${ext_if}:network instead. This allows me to move my machine to new networks without worrying about having to adapt my firewall/router everytime.

    The current weakness of my approach are:
    zeroconf, which doesn't behave well. Firstly it binds to the most recent IP/alias of an interface, so adding IPs throws everything off, and secondly only one instance of avahi-daemon can bind to an interface, so either the host, or one jail can do it externally, and internally.
    Hmmm - there was one other that I can't remember now.

    I use this scenario on a dell inspiron notebook, that I use for web developemnt. I have individual jails set up for client web server development environments. I have additionally one jail as a mysql server, that the others can see. I manually bind a client jail to my external interface if I want to share the server with other developers.
    I map my user home folders over (one set of source on all machines) to the jails, an then symlink source folders into /usr/local/www ... folders, but could use homedir stuff. Because of this I maintain a user uid map between the jails (which warden does automatically.)

    In a less personal space I would consider using the same approach, perhaps without the portjail concept, but sticking to internal network for internal space (connecting web server to db server) and then either RDR external server traffic, or map external IPS. I like the network separation, and control. It maps really well will network control and monitoring.
     
  9. mamalos

    mamalos New Member

    Messages:
    427
    Likes Received:
    0
    Guys, I really don't understand why you are stretching this thread...

    @OP: NAT!!!! Keep the rest of your configuration as is and add the following to the configuration files mentioned:

    In your /etc/rc.conf:
    Code:
    pf_enable="YES"
    pflog_enable="YES"
    
    Create /etc/pf.conf with your "favorite editor", set its permissions 0700 and write these lines inside it:

    Code:
    ext_if="re0"
    ext_ip="192.168.15.26"
    jail_ip="10.10.10.101"
    
    set skip on lo0
    
    nat pass on $ext_if -> $ext_ip
    
    pass out quick on $ext_if inet from { $ext_ip, $jail_ip } to any keep state
    
    
    And either restart your machine or give:

    # /etc/rc.d/pf start && /etc/rc.d/pflog start

    on your command line. This way you will enable a firewall that allows ONLY all traffic originating from either one of your local IPv4 addresses (your host's and your jail's IPv4 addresses) that is destined to any machine outside of your external interface, statefully. For more information on how to build a firewall with PF read this and consult this forum's firewall section.

    As mentioned previously, make sure that your jail's /etc/resolv.conf contains your reachable namservers. If the above contents don't work (I haven't tested them...), replace the last line with this one:
    Code:
    pass all
    CAUTION: this will make your firewall to accept all traffic! Of course, if you haven't setup a firewall on your box, then this is already your case :).
     
  10. gqgunhed

    gqgunhed New Member

    Messages:
    92
    Likes Received:
    0
    some more things to consider:

    1.) Inside the jail, check the contents of your /etc/resolv.conf. Please add any nameservers here if needed, see your /etc/resolv.conf in the host-system (outside of the jail). Caution: These may change if set by your ISP via DHCP.

    2.) Ping is not allowed AFAIK because raw sockets are disabled for security reason by default. Check this via:

    # sysctl security.jail.allow_raw_sockets

    If you want to test connectivity of the jail by pinging, you must allow_raw_sockets. But try dig() or fetch(1)() to test connectivity (by IP and DNS-names) and isolate the cause.
     
  11. kr651129

    kr651129 New Member

    Messages:
    327
    Likes Received:
    0
    Thanks! I'd like to take this approach and I followed the instructions. I can ping the jail now (192.168.0.1) but I still don't have any external access.

    Code:
    # jls
       JID  IP Address      Hostname                      Path
         1  192.168.0.1     apache                        /usr/jails/apache
    
    /etc/resolv.conf (from inside the jail)
    Code:
    nameserver 8.8.8.8
    nameserver 8.8.4.4
    
    /etc/pf.conf (from the host)
    Code:
    IP_PUB="192.168.15.26"
    IP_JAIL="192.168.0.2"
    NET_JAIL="192.168.0.0/24"
    PORT_JAIL="{80,443,2020}"
    scrub in all
    nat pass on re1 from $NET_JAIL to any -> $IP_PUB
    rdr pass on re1 proto tcp from any to $IP_PUB port $PORT_JAIL -> $IP_JAIL
    
    /etc/rc.conf (host)
    Code:
    # -- sysinstall generated deltas -- # Sat Dec  1 10:36:25 2012
    ifconfig_re0="DHCP"
    ipv6_activate_all_interfaces="YES"
    hostname="host"
    
    sshd_enable="YES"
    ezjail_enable="YES"
    
    cloned_interfaces="lo1"
    ipv4_addrs_lo1="192.168.0.1-9/29"
    
    ifconfig (from jail)
    Code:
    apache# ifconfig
    re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
            options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
            ether bc:5f:f4:59:97:a9
    ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
            media: Ethernet autoselect (100baseTX <full-duplex>)
            status: active
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
            options=3<RXCSUM,TXCSUM>
    ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
    lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
            options=3<RXCSUM,TXCSUM>
            inet 192.168.0.2 netmask 0xffffffff
    ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
    
    Edit:
    I can ping any networked device except for the router 192.168.15.1 which is what the host's default name server is.
     
  12. SirDice

    SirDice Moderator Staff Member Moderator

    Messages:
    17,365
    Likes Received:
    0
    You're NAT'ing on re1, which doesn't exist on your system. You do have an re0.
     
  13. mamalos

    mamalos New Member

    Messages:
    427
    Likes Received:
    0
    Bravo and thanx SirDice,

    I've been trying to tell this to the OP from day 1 and included the code to do this on day 2...then I stopped believing there is a point in that :)