FreeBSD 12.1 + EZ-Jail (Web Jail Configuration Problem!) #2

StreetDancer

StreetDancer

Hey there again! After getting the basic installation requirements for rc.conf with SirDice's assistance I proceeded to install my new jail to a STATIC IP (Public IP) and ran into a couple problems.

Code: 
# ezjail-admin create apache1SYSorgPHP56 'lo1|127.0.0.1,em7|104.36.17.19'
Warning: Some services already seem to be listening on IP 127.0.0.1
  This may cause some confusion, here they are:
www      php-fpm    88773 6  tcp4   127.0.0.1:9001        *:*
mysql    mysqld     72587 22 tcp4   127.0.0.1:3306        *:*
www      php-fpm    3289  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    3175  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    3093  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    2907  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    2717  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    2350  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    2037  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    1863  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    1778  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    1544  6  tcp4   127.0.0.1:9006        *:*
www      php-fpm    1208  6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    906   6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    556   6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    529   6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    179   6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99951 6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99810 6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99717 6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99404 6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99351 6  tcp4   127.0.0.1:9002        *:*
www      php-fpm    99011 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    98754 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    98681 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    98404 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    98091 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97973 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97924 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97773 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97466 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97350 6  tcp4   127.0.0.1:9005        *:*
www      php-fpm    97154 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96901 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96671 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96586 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96457 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96360 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    96241 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    95887 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    95631 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    95415 6  tcp4   127.0.0.1:9004        *:*
www      php-fpm    95052 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    95026 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    94904 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    94830 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    94467 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    94205 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    93887 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    93547 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    93380 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    92993 6  tcp4   127.0.0.1:9003        *:*
www      php-fpm    92808 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    92668 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    92436 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    92048 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    92047 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    91874 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    91800 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    91647 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    91328 6  tcp4   127.0.0.1:9001        *:*
www      php-fpm    90967 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    90932 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    90808 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    90770 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    90556 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    90174 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    89872 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    89523 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    89445 6  tcp4   127.0.0.1:9000        *:*
www      php-fpm    89233 6  tcp4   127.0.0.1:9000        *:*
root     php-fpm    89144 8  tcp4   127.0.0.1:9000        *:*
root     php-fpm    89144 9  tcp4   127.0.0.1:9001        *:*
root     php-fpm    89144 10 tcp4   127.0.0.1:9003        *:*
root     php-fpm    89144 11 tcp4   127.0.0.1:9004        *:*
root     php-fpm    89144 12 tcp4   127.0.0.1:9005        *:*
root     php-fpm    89144 13 tcp4   127.0.0.1:9002        *:*
root     php-fpm    89144 14 tcp4   127.0.0.1:9006        *:*
bind     named      64900 24 tcp4   127.0.0.1:53          *:*
bind     named      64900 25 tcp4   127.0.0.1:953         *:*
bind     named      64900 515 udp4  127.0.0.1:53          *:*
Warning: Some services already seem to be listening on all IP, (including 127.0.0.1)
  This may cause some confusion, here they are:
www      nginx      4571  9  tcp4   *:81                  *:*
root     nginx      4164  9  tcp4   *:81                  *:*
bind     named      64900 21 tcp6   *:53                  *:*
bind     named      64900 512 udp6  *:53                  *:*
Warning: Some services already seem to be listening on all IP, (including 104.36.17.19)
  This may cause some confusion, here they are:
www      nginx      4571  9  tcp4   *:81                  *:*
root     nginx      4164  9  tcp4   *:81                  *:*
bind     named      64900 21 tcp6   *:53                  *:*
bind     named      64900 512 udp6  *:53                  *:*
# ls
apache1SYSorgPHP56    basejail        flavours        newjail

The jail list with ezjail-admin shows the following:

Code: 
# ezjail-admin list
STA JID  IP              Hostname                       Root Directory
--- ---- --------------- ------------------------------ ------------------------
ZS  N/A  127.0.0.1       apache1SYSorgPHP56             /usr/jails/apache1SYSorgPHP56
    N/A  em7|104.36.17.19

Manual Starting of Jail attempt:

Code: 
# ezjail-admin start apache1SYSorgPHP56
Starting jails: cannot start jail  "apache1SYSorgPHP56":
jail: apache1SYSorgPHP56: chdir /root: No such file or directory
jail: apache1SYSorgPHP56: /sbin/ifconfig lo1 inet 127.0.0.1/32 alias: failed
.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
Error: Could not start apache1SYSorgPHP56.
  You need to start it by hand.
# ezjail-admin start apache1SYSorgPHP56
Starting jails: cannot start jail  "apache1SYSorgPHP56":
jail: apache1SYSorgPHP56: chdir /root: No such file or directory
jail: apache1SYSorgPHP56: /sbin/ifconfig lo1 inet 127.0.0.1/32 alias: failed
.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
Error: Could not start apache1SYSorgPHP56.
  You need to start it by hand.
#

Thank you for a shuv in the right direction! :)
 
Hi
note that in the handbook the create jail command uses 127.0.1.1 and not 127.0.0.1 that you have used. That is what is causing the warnings about something else listening on the 127.0.0.1 addresses etc.
Code: 
ezjail-admin create jailname 'lo1|127.0.1.1,em0|192.168.1.50'

The ultimate aim of jailing services/processes is to move all those services that are currently running on the Host into separate jails. The easiest way is to note that jails are typically created on private/internal network addresses. So 10.x.x.x or 192.168.x.x would be more typical ranges to use. You then use the host firewall to forward requests from the internet to the relevant internal jail IP address ranges for the services there to process.

So when creating jails this is typically done like this and avoids the issues with Host and Jails services listening on the same ip addr / port
Code: 
# ezjail-admin create jailname1 'lo1|127.0.1.1,vtnet0|10.0.0.1'
# ezjail-admin create jailname2 'lo1|127.0.2.1,vtnet0|10.0.0.2'
etc.
which gives
Code: 
# ezjail-admin list
DR  4    127.0.2.1       jailname2                      /usr/jails/jailname2
    4    vtnet0|10.0.0.2
DR  3  127.0.1.1       jailname1                      /usr/jails/jailname1
    3  vtnet0|10.0.0.1

This has a nice tutorial for jails to get you more familiar : https://www.cyberciti.biz/faq/howto-setup-freebsd-jail-with-ezjail/
 
anlashok said:
Hi
note that in the handbook the create jail command uses 127.0.1.1 and not 127.0.0.1 that you have used. That is what is causing the warnings about something else listening on the 127.0.0.1 addresses etc.
Code: 
ezjail-admin create jailname 'lo1|127.0.1.1,em0|192.168.1.50'

The ultimate aim of jailing services/processes is to move all those services that are currently running on the Host into separate jails. The easiest way is to note that jails are typically created on private/internal network addresses. So 10.x.x.x or 192.168.x.x would be more typical ranges to use. You then use the host firewall to forward requests from the internet to the relevant internal jail IP address ranges for the services there to process.

So when creating jails this is typically done like this and avoids the issues with Host and Jails services listening on the same ip addr / port
Code: 
# ezjail-admin create jailname1 'lo1|127.0.1.1,vtnet0|10.0.0.1'
# ezjail-admin create jailname2 'lo1|127.0.2.1,vtnet0|10.0.0.2'
etc.
which gives
Code: 
# ezjail-admin list
DR  4    127.0.2.1       jailname2                      /usr/jails/jailname2
    4    vtnet0|10.0.0.2
DR  3  127.0.1.1       jailname1                      /usr/jails/jailname1
    3  vtnet0|10.0.0.1

This has a nice tutorial for jails to get you more familiar : https://www.cyberciti.biz/faq/howto-setup-freebsd-jail-with-ezjail/
Click to expand...
anlashok,

Thank you for the clarification and examples. Very wonderful and makes a whole lot more sense to me!

Here I created the jail with the correct "127.0.1.1" and external NIC em7 w/ STATIC. This time it outputted nginx and named using all ports.

I disabled the NGINX and retried to get only named. I went into BIND 9.9 entries in /usr/local/etc/namedb/ and commented out all entries that pertained to 104.36.17.19

I hope this will mitigate the difficulty in the starting of this jail after a server reboot.

Best Regards,
 
Code: 
# ezjail-admin create apache1SYSorgPHP56 'lo1|127.0.1.1,em7|104.36.17.19'
Warning: Some services already seem to be listening on all IP, (including 127.0.1.1)
  This may cause some confusion, here they are:
www      nginx      4571  9  tcp4   *:81                  *:*
root     nginx      4164  9  tcp4   *:81                  *:*
bind     named      64900 21 tcp6   *:53                  *:*
bind     named      64900 512 udp6  *:53
 
Here is the current output at the moment before server reboot:

Code: 
# ezjail-admin create apache1SYSorgPHP56 'lo1|127.0.1.1,em7|104.36.17.19'
Warning: Some services already seem to be listening on all IP, (including 127.0.1.1)
  This may cause some confusion, here they are:
bind     named      64900 21 tcp6   *:53                  *:*
bind     named      64900 512 udp6  *:53                  *:*
Warning: Some services already seem to be listening on all IP, (including 104.36.17.19)
  This may cause some confusion, here they are:
bind     named      64900 21 tcp6   *:53                  *:*
bind     named      64900 512 udp6  *:53
 
Code: 
# cd /usr/jails
# ls
apache1SYSorgPHP56    basejail        flavours        newjail
# ezjail-admin start apache1SYSorgPHP56
Starting jails: cannot start jail  "apache1SYSorgPHP56":
jail: apache1SYSorgPHP56: chdir /root: No such file or directory
jail: apache1SYSorgPHP56: /sbin/ifconfig lo1 inet 127.0.1.1/32 alias: failed
.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
Error: Could not start apache1SYSorgPHP56.
  You need to start it by hand.
#
 
StreetDancer said:
Code: 
# cd /usr/jails
# ls
apache1SYSorgPHP56    basejail        flavours        newjail
# ezjail-admin start apache1SYSorgPHP56
Starting jails: cannot start jail  "apache1SYSorgPHP56":
jail: apache1SYSorgPHP56: chdir /root: No such file or directory
jail: apache1SYSorgPHP56: /sbin/ifconfig lo1 inet 127.0.1.1/32 alias: failed
.
/etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables  is obsolete.  Please consider migrating to /etc/jail.conf.
Error: Could not start apache1SYSorgPHP56.
  You need to start it by hand.
#
Click to expand...
Note this:
So 10.x.x.x or 192.168.x.x would be more typical ranges to use. You then use the host firewall to forward requests from the internet to the relevant internal jail IP address ranges for the services there to process.

Use a private IP address in place of the public IP address here:
ezjail-admin create apache1SYSorgPHP56 'lo1|127.0.1.1,em7|104.36.17.19'

So delete and recreate the jail. Or go edit its config file in /use/local/etc/ezjail/apache1SYSorgPHP56.conf.
Get it running first before worrying about other services - bind etc.
 
I would suggest making sure that services running on the host are bound to the host's IP addresses specifically. The same should be done for any services running in jails. This will prevent a service from trying to grab all available IP addresses to bind on.
 
SirDice said:
I would suggest making sure that services running on the host are bound to the host's IP addresses specifically. The same should be done for any services running in jails. This will prevent a service from trying to grab all available IP addresses to bind on.
Click to expand...
SirDice,

Thanks for that! I agree. I tried finding NGINX's; however it doesn't utilize an asterix.

NGINX --> nginx.conf displays:

Code: 
   server {
        listen       81;
        server_name  sharpenyoursword.org;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;
        access_log logs/sharpenyoursword.org.access.log;
        location / {
           #root   /usr/local/www/nginx;
            root   /usr/local/www/nginx/sysorgmain/;
            index index.php index.html index.htm;
        }

and it's custom configuration file for the VHOST is 001_sharpenyoursword.org.conf which is on an include line on nginx.conf listening lines are:

Code: 
server {

  listen 81;
  server_name sharpenyoursword.org; # Replace with your IP or hostname
  root /usr/local/www/nginx/sysorgmain/;
  index index.php index.html index.htm;

  location / {
    try_files $uri $uri/ =404;
  }

  location ~ \.php$ {
    fastcgi_pass 127.0.0.1:9001;
    fastcgi_index index.php;
#    fastcgi_param SCRIPT_FILENAME $document_root               $fastcgi_script_name;
    include fastcgi_params;
  }

}

I am not sure how NGINX is grabbing all IP's with these 2 configuration files setup like they are. hmm... tough one!

I also do not know where named (BIND 9.9) is grabbing all IP's... those are also specifically set.

/usr/local/etc/namedb/named.conf listen lines are the following:

// If named is being used only as a local resolver, this is a safe default.
// For named to be accessible to the network, comment this option, specify
// the proper IP address, or delete this option.
listen-on { 127.0.0.1; 104.36.16.185; 104.36.16.37;};

// If you have IPv6 enabled on this system, uncomment this option for
// use as a local resolver. To give access to the network, specify
// an IPv6 address, or the keyword "any".
// listen-on-v6 { ::1; };


Could DNS entries attribute to binding? I don't see how though; it doesn't seem to make sense to me that a DNS entry for any domain on the server would attribute a binding variable.

NGINX and BIND were the only services that were being prompted when installing my first jail with ezjail-admin

Best Regards,
 
If you don't specify an IP address in the listen definition it listens on all available addresses. applies to any service really, some use the *, some don't. Define both the ip address and the port to constrain it onto a single ip : port

nginx
Code: 
server {
        listen       81;
        server_name  sharpenyoursword.org;
is interpreted as listen *:81 in Apache speak, eg listen on every IP address port 81.

You want to use
Code: 
server {
        listen      104.36.17.19:81;
        server_name  sharpenyoursword.org;
i.e. nginx is only listening on 104.36.17.19 and only to port 81
 
anlashok said:
If you don't specify an IP address in the listen definition it listens on all available addresses. applies to any service really, some use the *, some don't. Define both the ip address and the port to constrain it onto a single ip : port

nginx
Code: 
server {
        listen       81;
        server_name  sharpenyoursword.org;
is interpreted as listen *:81 in Apache speak, eg listen on every IP address port 81.

You want to use
Code: 
server {
        listen      104.36.17.19:81;
        server_name  sharpenyoursword.org;
i.e. nginx is only listening on 104.36.17.19 and only to port 81
Click to expand...

anlashok,

ahhh. That's how it's done. I was wondering where to define the static IP.

:)

Best Regards,
 
You must log in or register to reply here.
Back
Top