How to populate jails?

[max21]

I just build five jails on my home desktop. Maybe all five will eventually land on a hosting VPS, minus the desktop; so security is of great concern. Maybe jail-1 will be the router, gateway, security center or whatever else is possible to server the rest of the jails. How would you do this?

Now that I’m near finish, I just realized, I really need to know what each jail should include and what each jail should not include, and why. I done ran out of guesses and now I feel very foolish for not asking about this in the first place when it came to mind before getting started. Any and all suggestions would be helpful. Even a bad idea could generate better idea's. What I’m posting below is what I have done so far. I just don’t know what’s missing but I do know they should stay basic and secure as possible.

This is about manual-jails only. So far, all five jails have the most minimum FreeBSD 10.1 base-system. Each has sshguard and portmaster installed as the bottom-line format for all jails. Are there other ports that should be included within this bottom-line format, and why?

Are these jails complete?

Are there any other most basic applications required to make the main application complete and more secure?

Should any of these jails need to have their own php etc., included or can they use them from Jail-5 without lost of performance?

Jail-1)   Router + sshguard + pf + snort + __
Jail-2)   Nginx + sshguard + __
Jail-3)   Apache + sshguard + __
Jail-4)   MySQL + sshguard + c-connector + __
Jail-5)   php + perl + java + sshguard + __

I have not tested anything. Learning how to network jails and use SQL and the others will follows. But for now, I need these jails in place to reach that point. Thanks in advance.

 

[getopt]

The jails share the kernel with the jail host. A feature of jails is that jails have a reduced /dev to the minimum.
If you read pf(4) the first sentence you can read is

DESCRIPTION
Packet filtering takes place in the kernel. A pseudo-device, /dev/pf, allows userland processes to control the behavior of the packet filter through an ioctl(2) interface.

If you look in a jail's /dev you won't find a /dev/pf there for a very good reason.

Regarding what to install in a jail. If you want to run a webserver in a jail just add those packages that are required to run this - nothing more. If you want your webserver use i.e. PHP, install it in the webserver's jail. The concept of jails is minimalism.

 

[storvi_net]

If you don't have any reason for it (for example different administrators / groups) I would not start several instances of the ssh-daemon.

Just use the jails for the services and configure them through the host.
Regards
Markus

 

[max21]

The jails share the kernel with the jail host. A feature of jails is that jails have a reduced /dev to the minimum.
If you read pf(4) the first sentence you can read is

If you look in a jail's /dev you won't find a /dev/pf there for a very good reason.

Regarding what to install in a jail. If you want to run a webserver in a jail just add those packages that are required to run this - nothing more. If you want your webserver use i.e. PHP, install it in the webserver's jail. The concept of jails is minimalism.

Thanks for shedding light on these facts in such a unique way. It clears a lot of things up for me, in a flash.

That’s all I need to know. Everywhere I turn there a guide showing how to squeeze three web applications in to one jail, with 44 programming languages, and a beep. When FreeBSD and it’s history teaches us … one processes per jail… I did not understand that it meant “one FULL process.” The application and all the tools to help it run in one place, for quicker access. The other application goes into its own jail to keep the confusions down. I started out on the right track, than
I was about to spread thing out a bit too thin. Thank you

 

[max21]

If you don't have any reason for it (for example different administrators / groups) I would not start several instances of the ssh-daemon.

Just use the jails for the services and configure them through the host.
Regards
Markus

SSHGUARD is not only for shared systems. The advantage of having sshguard at the door of each cell is to serve as warning magnesium. The overhead is near to nothing, and it can be configured to send a message to your host that someone is knocking at your door. There is no better enhancement for jails IMHO. This kind of service is priceless.

I don’t think I ever want to be an administrator dealing with groups on shared systems; it would drive me nuts in no time. What kind of protection could I provide went governments and banking systems super-systems are getting rip off daily.

 

[storvi_net]

There is no reason to open the door and put a guard in front, if it is not neccesary to open the door in the first place...

Use the jail to put the applications on their own and then use pf(4) / ipfw(8) to do the filtering on the host. There is no reason to not put an IDS on the host, but keep the jails simple.

On one host I use a single jail as reverse proxy for several web aps, which are handled in separate jails. The reverse proxy is the only jail with a connection to the outside.

Regards
Markus

 

[abishai]

Usage of ssh guard and is not very useful. The reason that your server will always be scanned and your ssh daemon will always be bruted. So the best practice is to hide ssh daemon version, use RSA-keys for auth set on firewall policy to ban IPs, hammering your server to reduce log size.
If you prefer pf, you can mess a little with packet normalizer to confuse OS fingerprint.

Also, I noticed sshguard on jails. You definitely don't want to expose them to the internet. Make a lo1 pseudo interface and bind your jails to it. Use a firewall for packet redirection if needed.

 

[max21]

Thanks guys. Here are my five new jails.

Jail-1)  pound - - - for reverse proxy
Jail-2)  Nginx - - - to handle static pages
Jail-3)  Apache
Jail-4)  Wordpress
Jail-5)  MySQL

Through your responses I concluded there must be three to four basic rc.conf need to force the host and web-jails to interact with one another. Perhaps a few members could post relevant parts their rc.conf for one or more web-jail types.

The file below is what I presently use on my host, but I stripped it down the best I could figure for use in aps-jails. I’m sure it don’t qualify for jail-1, reverse proxy type. I been trying to figure what else to remove or add to it to get it right. Could someone post corrections of this file for aps-jails? Also, is order important in a rc.conf?

The possible four needed rc.conf for web-jails:

Jail-1)  pound - - - reverse proxy rc.conf - 1
Jail-2)  Nginx - - - web app rc.conf -2
Jail-3)  Apache - - web app rc.conf - 2
Jail-4)  Wordpress - - web app rc.conf -2
Jail-5)  MySQL - - backend app rc.conf -3
HOST)  watch jail logs, & stuff  rc.conf -4

my aps-jail rc.conf:

# #################################
#  web-jail web aps  ....  rc.conf
# #################################

network_interfaces=""

inetd_enable="NO"
hostname="ws.jail-1.local"
ifconfig_re0="inet 80.0.0.0 netmask 0xffffffff broadcast 80.0.0.0"
defaultrouter="192.168.1.79"

sshd_enable="YES"
clear_tmp_enable="NO"

syslogd_flags="-ss"
syslogd_enable="NO"
cron_flags="$cron_flag -J 15"

rpcbind_enable="NO"
mountd_enable="NO"
mountd_flags=""
nfs_server_enable="NO"
nfs_client_enable="NO"
nfs_reserved_port_only="YES"
rpcbind_statd_enable="NO"
rpcbind_lockd_enable="NO"

sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

postfix_enable-"NO"

ipv6_activate_all_interface="NO"
virecover_enable="NO"
update_motd="NO"

tcp_drop_synfin="YES"
icmp_drop_redirect="YES"
icmp_log_redirect="NO"
portmap_enable="NO"
update_motd="NO"
fsck_y_enable="YES"

# ...............................
# to be used ...... rc.d  order .
# ...............................
pound_enable="NO"     #  *  .. reverse proxy
nginx_enable="NO"     #  *  .. static pages
apache22_enable="NO"   #  *  .. heavy duty
wordpress_enable="NO"   #  *  .. CMS
mysql_enable="NO"     #  *  .. backend db

 

[storvi_net]

I don't really get what you want.

Probably you start with a single jail - for example the nginx jail to get used to it.
Start with the goal to serve a static page out of your nginx-jail!

Do you use some jail-Framework like sysutils/qjail / sysutils/ezjail?

Which host-firewall do you use for forwarding the packets? (pf(4) / ipfw(8))?

Regards
Markus

 

[max21]

I don't really get what you want.

Sorry for the confusion. I always over do it than loss focus myself I posted what I create to be my jail /etc/rc.conf. It might be an over-kill or missing something. So I was hoping that someone could modify it to what it should be for jails. I just want to be sure I’m not missing anything as I move forward. I been taking my time googling as I go so to get it right the first time.

Just use the jails for the services and configure them through the host.

storvi_net, when I asked to review experienced jailer /etc/rc.conf I also wanted to look for clues to what you mean by this. Google turn-up nothing about *jails as a service*. I know this is kind of lame not to understand these simple words that could be pointing to a common meaning, but I haven’t gain a real clue yet and I was too embarrass to bring it up again. Could you please demonstrate to me what this means?

Do you use some jail-Framework like sysutils/qjail / sysutils/ezjail?

No. I created all jails manually by the handbook and a few how-2’s.

Which host-firewall do you use for forwarding the packets? (pf(4) / ipfw(8))?

I use pf and a very special Thanks for the following two replies. Other than pf rules on the host these two replies are more than enough certification for me that additional security will be added for applications in web type jails. I been doing lots of house cleaning on my desktop system as I go, so that I can easily transfer the jails and config files to a pure FreeBSD server system, when need be. That’s why I’m so slow to get .. :

storvi_net said GET TO WORK!: Probably you start with a single jail - for example the nginx jail to get used to it. Start with the goal to serve a static page out of your nginx-jail!

OK

storvi_net said GET TO WORK!: On one host I use a single jail as reverse proxy for several web aps, which are handled in separate jails. The reverse proxy is the only jail with a connection to the outside.

OK


This is unbelievable! elm just posted: https://forums.freebsd.org/threads/rc-d-script-for-perl-service.50557/

... or Perl (jails) as a service: Can’t get closer than that I guest. I’m going to read about rc.d(8), rc.subr(8) documents to learn how these things work. After you guys proved how simple even jailing can be this can’t be as hard as I once thought either.

Thanks again storvi_net.

 

[SirDice]

Usage of ssh guard and is not very useful. The reason that your server will always be scanned and your ssh daemon will always be bruted.

I have to disagree. The big difference is one client doing hundreds of attempts, flooding your logs or just a couple of tries. The problem with brute-force attempts is that more they can try the more likely it is they'll get in. That's the big difference.

So the best practice is to hide ssh daemon version, use RSA-keys for auth set on firewall policy to ban IPs, hammering your server to reduce log size.

This is called security-through-obscurity. There's nothing stopping an attacker from scanning your box and finding the alternate port. Granted, most attacks are simple scripts that don't look beyond a couple of ports.

If you prefer pf, you can mess a little with packet normalizer to confuse OS fingerprint.

I very rarely find attackers actually fingerprinting a host before they attack. Most attacks (I'd say about 90-95% of them) are automated, there's nobody watching and it's just a script that scans hosts. If it finds an open port it'll simply try without looking what it actually is.

Also, I noticed sshguard on jails. You definitely don't want to expose them to the internet. Make a lo1 pseudo interface and bind your jails to it. Use a firewall for packet redirection if needed.

Packet redirection would expose your host to the internet. If it has an open port it will get scanned, it doesn't matter if it's a jail, a VPS or a physical machine. Even redirecting through one or more firewalls doesn't matter. Opening a port exposes it to the internet. At some point you will have to expose something, a completely closed off host is rather useless as a server.

As for running sshguard, I highly recommend it. Run it on the host though, the jails can't manipulate the firewall. But it's easy to configure sshguard to monitor multiple logs. Just add this to rc.conf:

sshguard_watch_logs="/var/log/auth.log:/jails/jail1/var/log/auth.log:/jails/jail2/var/log/auth.log"

Although the documentation states you can also use sshguard for a few other services (like exim) I have never been able to get those working. If you have other services to protect I'd recommend security/py-fail2ban as it's easier to add checks, even customized ones.

 

[kpa]

Usage of ssh guard and is not very useful. The reason that your server will always be scanned and your ssh daemon will always be bruted. So the best practice is to hide ssh daemon version, use RSA-keys for auth set on firewall policy to ban IPs, hammering your server to reduce log size.
If you prefer pf, you can mess a little with packet normalizer to confuse OS fingerprint.

I beg to differ here in a very big way. If you have a good enough password no one this planet has enough time and resources to brute force your password *). Brute force attacks work only on weak passwords that do not have enough entropy and can be cracked with an automated method. Of course a good enough password that can not be brute forced tend to be rather long and/or hard to remember and that's why use of RSA or DSA public key authentication is recommended over passwords.

*) Brute force the password below. Explain how much effort the attacker would have to go trough to find out that I was using this particular scheme for my passwords and how much effort he would have to go trough to actually crack the password.

freebsd4forum3password2kpa1<my social security number>0

I'm not using this particular scheme anywhere, I just cooked it up for this example. Note how easy this would be to remember, it's just lot to type.

 

[abishai]

This is called security-through-obscurity.

Scripts are hammering well known ports, so using another port will decrease scan attempts drastically. Also, you can set up a firewall policy to ban attackers quite effective. Both of methods will keep logs cool.

Packet redirection would expose your host to the internet.

It will expose daemons of your choose. sshguard on every jail means ssh is exposed and who knows what else can listen all interfaces. ssh is rarely needed there as jails can be administrated from the host.

If you have a good enough password

I'm speaking from the log size perspective mostly. Good passwords or keys with good passwords are obvious.

 

[max21]

I'm finally at step-2 which seems to involve hooking up the host pf and sshd to interact privately with any jail, and allowing the pound jail as the only way to provide internet access for other jails.

What are the NAT rules to make this happen?

Below are my most relevant rules for my workstation. Could someone add to or modify this file so that will work for what I am after? I think I got a half of clue listed in this file under NAT.

# Standard rules for my Gnome Workstation
# to be rewritten for a pure FreeBSD web-server
# This file is based on unixguide-FreeBSD 8.2
# and other ideas found while googling in 2010

ext_if = "re0"
int_if = "re1"
dns1 = "192.168.1.254"
dhcp = "192.168.1.254"

jail1 = "80.0.100.1"  #  pound - my guest.
jail2 = "80.0.100.2"  #  nginx - my guest.
jail3 = "80.0.100.3"  #  apache - my guest.
jail4 = "80.0.100.4"  #  mysql - my guest.
jail5 = "80.0.100.5"  #  home  - control center
##### = "##########"  #  never to use host unless

tcp_services = "{22, 80, 100, 3389}"
ob_state = "flags S/SA modulate state"
ib_state = "flags S/SA synproxy state"

# [TABLES]

table <sshguard> persist

# [OPTIONS]

#set block-policy drop
set block-policy return
set state-policy if-bound
set loginterface $ext_if

# [TRAFFIC NORMALIZATION]

set skip on lo0
scrub in all
scrub reassemble tcp

# [QUEUEING]


# [NAT - TRANLATION]

# newly found minutes ago .. I'm guesting .................................................
nat on $ext_if from !($ext_if:network) to any -> ($ext_if:0)
rdr pass out on $ext_if inet proto tcp from any to ($ext_if:0) port 100 -> 80.0.0.1 port 22

# .........................................................................................
# ok .. back to what I have:

nat on $ext_if from !($ext_if) -> ($ext_if:0)
pass quick on $int_if all
pass quick on lo0 all

pass in on $int_if inet proto tcp from any to 80.0.0.1 port 8880 keep state
pass out on $ext_if inet proto tcp from any to any port www keep state
pass out on $int_if inet proto tcp from any to 192.168.1.254 port 3389

# [PACKET FILTERING]

pass out quick on $ext_if proto tcp from any to $dns1 port 53 $ob_state
pass out quick on $ext_if proto udp from any to $dns1 port 53 keep state
pass out quick on $ext_if proto udp from any to $dhcp port 67 keep state
pass out quick on $ext_if proto tcp from any to any port 80 $ob_state
pass out quick on $ext_if proto tcp from any to any port 443 $ob_state

# [MORE THAN JUST PORT 22]

block in quick on $ext_if from <sshguard> label "ssh bruteforce"

 

[max21]

I read enough confusing documents to drive a networking expert insane. Beyond how to build a jail to network with the host who has users, to ssh into your own jail, or to connect to a site to play a game; nearly all others most important jail related threads ends with no solutions whatsoever. My connection activity charts at this forum and Google will prove that I been searching everywhere, here and at Google, all day and all nights, ever since I started this thread … and well before; and; this thread is the closest thread that can be found in the universe in relation to what I’m after:

https://forums.freebsd.org/threads/pf-nat-rdr-rules-for-jailed-proxy-services.43840/

I bet everyone knows this already, but surely there got to be one jailing expert that has done it all to the letter. Would you mind putting your degree to work, right here? It obvious, I tried hard not to call on you by requesting only to see a few sample file build for jails… The rc.conf, pf.conf. For me and anybody else, that would be the road-map, but as you see, even that hasn’t happen. If this thread is to die like the rest, hopefully new members to come will gain answers, or the truth about these most important questions concerning jails. This should be as simple as they say about building jails it-self.

The real learning doesn’t even begin until after perfectly securing connections of all jails. This should be common knowledge and well documented for every possible scenario just like everything else about FreeBSD which serves as a stepping stone to get you at the point of where you need to be, at minimum. But as you see by my search, and many others, it doesn’t even come close. It doesn’t matter.

Have a nice day.

 

[gkontos]

What exactly are you looking for? Have you read the PF documentation?

If you expect other people to "code" for you, then I am afraid that you will be disappointed. Like others told you, start simple with one jail. Use a virtual environment to experiment like emulators/virtualbox-ose .

Learn how to troubleshoot using tcpdump(). PF Logging will also assist you in troubleshooting your network issues....

 

[max21]

I been at it like a dog, I got so sick of searching last night… I forgot about what wblock@ told me last year … *if you can’t find any information on the internet concerning your problem, it could be that others never had the same issue.

gkontos said: Like others told you, start simple with one jail. Use a virtual environment to experiment like emulators/virtualbox-ose .

I don’t expect anyone to code it for me. Coding is my specialty for FreeBSD when I know what the pointers do or how they work, but in this case, I don't. But it’s all good. I pick up lots useful tips along the way and a half of ton of new commands. Sleep did me well. I’m human again. Yes, the guys did suggest that I take one step at a time. I guess I have to because I can’t see no way to start in the middle to do something useful as I figure out how to begin. I once brought a AB Dick printing press just to look it over, than afterwards I taught myself how to run it. It makes you more productive and wiser as you go. Anywho, I’ll figure it out soon.

Thank gkontos.

 

[gkontos]

It is not bad to have high expectations but you need to understand that the only person that you can rely upon is you.

 

[sidetone]

Are you trying to open up /dev files to the jail? I use ezjail, it is similar enough to the complicated jail. There's a custom jail configuration file in /usr/local/etc/ (jail or ezjail directory) , option that points to a section in /etc/defaults/devfs.rules. I think the override file is supposed to go to /etc/devfs.rules, since /usr/local/etc/ didn't work for me.

Some lines have to be added and/or set to unhide /etc/defaults/devfs.rules

[devfsrules_jail=4]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path zfs unhide

My jail configuration files are in /usr/local/etc/ezjail/myfile, which should be your local etc jail/myfile location. I don't know if this helps. Can you be more clear in what you are asking?

You'll probably have to pick up a book if your answer is very difficult to find online, and the most of the only ones available are for older FreeBSD versions.

 

[max21]

Thanks sidetone, but I got all of that down to a science. I done actually build my own framework for jails, almost down to the T to meant all of what I know so far. That’s how I been learning how to deal with jails, ever since the first reply on this thread by getopt. I seen it all… I flashed the entire system. You should have seen it! It took the system out and everything all because of one simple mistake. I learn the lesson I will never forget and I understand if strange things ever happen. I even found a better dd .. rm -rP

I finally just directly connected the Apache jail to internet since I could not avoid doing so if I wanted to get anywhere. Now I need Apache to talk to the back-end database, MySQL. I’ll deal with pound to server as a load-balancer or proxy later for the thrill of it all. Don't worry about it. I’ll figure it out.

Anywho, this is the way I use devfs(8) manually, and in a script sidetone. I would recommend that anyone new to jail to build manually before getting spoiled. At least until near perfection, and then use sysutils/qjail or sysutils/ezjail when needed or ready.

mount -t devfs devfs /Alcatraz/apache/dev
devfs -m /Alcatraz/apache/dev rule -s 4 applyset
devfs -m /Alcatraz/apache/dev rule apply path extra-device1 unhide
devfs -m /Alcatraz/apache/dev rule apply path extra-device2 unhide

FreeBSD really offer a lot of amazing things that Windows or Linux can't match. It’s the same thing most jail frameworks would do, but it so smooth and fun to feel it when you do it all a few times for yourself.