How to connect a FreeBSD jail to the internet? (Amazon EC2)

phil5

New Member


Messages: 15

I am running a FreeBSD EC2 instance with a single jail.

In a regular homebrew setup, I would create a jail in "shared IP" (iocage) mode, and configure my router to forward various ports to the jail. In EC2, I am not sure whether to share the public IP or assign a new one.

I created a new Network Interface and Elastic IP, but could not get the jail to ping or telnet anything, not even the host. Which makes me suspect that the jail's internal config is wrong. The host could however ping the jail (which still does not mean that the Elastic IP is the right approach...)

Any ideas? (I posted the above in the AWS forums.)


In addition, I wonder if the jail approach is necessary in EC2? The instance's sole purpose is a webserver for Wordpress.

In my present setup (which I am attempting to migrate to AWS), the host is useful for running various cron jobs which implement my backup solution.

Any suggestions on both issues is greatly appreciated.
 
OP
OP
P

phil5

New Member


Messages: 15

I let go of the added Network Interface/Elastic IP approach and added a secondary IP (same subnet) to my instance.

With a vanilla iocage jail I can now resolve addresses but get no response. I think I need to configure NAT on the host to enable port forwarding?

I still would like to have separate public IPs for the host and jail, however.
 

alexseitsinger

Member

Reaction score: 19
Messages: 94

With sysutils/ezjail all that's needed is:

/etc/rc.conf
ifconfig_<nic>_alias="<address>/<subnet>"
(Making sure that the IP is in the subnet provided by the router.)

cp /etc/resolv.conf /usr/jails/<jail_name>/etc/resolv.conf
(Copying the official resolv.conf to the jail)

/usr/local/etc/ezjail/<jail_name>
export jail_<jail_name>_parameters="allow.raw_sockets=1"
(Allow things like ping)

And then I'm able to use the internet from the jail.
 

alexseitsinger

Member

Reaction score: 19
Messages: 94

Since youre using a virtual server, I think you will need to add a non-routable address to the jail, and then NAT the traffic from it using PF.
 
OP
OP
P

phil5

New Member


Messages: 15

Thanks. I finally got it working, sort of. My pf rule was wrong.

I tested 3 ways:
  1. With a standard cloned loopback interface with pf rule (not particular to EC2/AWS)
  2. By assigning a Secondary private IPv4 IP to my instance's network interface and substituting that IP into the pf rule
  3. Same as (2.) but with Elastic IP associated to the secondary IP

However, I still cannot connect my jail to the internet thru a second network interface (xn1):

# ifconfig
Code:
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9001
        options=503<RXCSUM,TXCSUM,TSO4,LRO>
        ether 12:ae:db:a2:0c:68
        inet6 fe80::10ae:dbff:fea2:c68%xn0 prefixlen 64 scopeid 0x2
        inet 172.31.91.191 netmask 0xfffff000 broadcast 172.31.95.255
        media: Ethernet manual
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
xn1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9001
        options=503<RXCSUM,TXCSUM,TSO4,LRO>
        ether 12:d2:14:4a:a7:58
        inet6 fe80::10d2:14ff:fe4a:a758%xn1 prefixlen 64 scopeid 0x3
        inet 172.31.81.75 netmask 0xffffffff broadcast 172.31.81.75
        media: Ethernet manual
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
I tried the following rule (does not work):

# cat /etc/pf.conf
Code:
ext_if="xn1"
jail_net="172.31.81.75/32"

nat pass on $ext_if from $jail_net to any -> $ext_if

pass out
pass in
Which is based on my original rule for the loopback (127.1.0.0/24 on lo1, not shown in ifconfig above):

# cat /etc/pf.conf
Code:
ext_if="xn0"
jail_net="127.1.0.0/24"

nat pass on $ext_if from $jail_net to any -> $ext_if

pass out
pass in
 

alexseitsinger

Member

Reaction score: 19
Messages: 94

nat pass on $ext_if from $jail_net to any -> $ext_if
I might be wrong about this, but I believe the syntax is:
Code:
nat on $ext_if from $jail_net to any -> ($ext_if)
 
OP
OP
P

phil5

New Member


Messages: 15

I might be wrong about this, but I believe the syntax is:
Code:
nat on $ext_if from $jail_net to any -> ($ext_if)
I'm not sure either, but from the handbook:
The parentheses surrounding the last part of the nat rule ($ext_if) is included when the IP address of the external interface is dynamically assigned.
All IPs are static in this case (I believe).

Although, I suspect the rule is wrong since NAT shouldn't be necessary-- the interface (xn1) is associated to a public IP.
 
OP
OP
P

phil5

New Member


Messages: 15

Same problem: Multiple public network interfaces not accessable on Amazon EC2/VPC

TL;DR: Issue stems from both network interfaces sharing the same subnet.

Workaround (linked at the bottom): Get Two Public IPs on an Amazon EC2 Instance for Free

This workaround is the same as no. 3 mentioned in post #5 of this thread.

Not really a solution in this case, since the goal is to set different security groups for each IP, hence the second network interface.

Ref:
Elastic IP Addresses https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/MultipleIP.html#working-with-multiple-ipv4
Elastic Network Interfaces https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html
 

alexseitsinger

Member

Reaction score: 19
Messages: 94

It sounds like the server might have it's outbound connections restricted to a single specific interface/NIC/IP and all other connection attempts are dropped by default. I've known other PaaS providers to do something like this. Maybe contact their support team to confirm this to be sure.
 
Top