SSH login to jail without public IP through master machine

[ondra_knezour]

Hello all,

I'm looking for solution to following scenario:

There is one server with one IP address accessible from internet. This machine is running jails, which uses private IPs, so they are not accessible from internet. I would like to allow users to login via SSH to their jails, but I'm not sure how to do that.

Would you consider using ChrootDirectory in sshd_config of "master machine" set to / of user's jail secure enough to separate ssh session of jail user from system running on "master machine"? If yes, is there posible to include additional files to "main" /etc/ssh/sshd_config, which will allow me to keep config files for particular user in common place for such user? Didn't find any usefull information sources for this.

Is there in the universe something like ssh proxy available?

Any other ideas?

 

[anomie]

Would you consider using ChrootDirectory in sshd_config of "master machine" set to / of user's jail secure enough to separate ssh session of jail user from system running on "master machine"?

That's an unusual use of jails (and I wouldn't personally recommend it). It may or may not work the way you're expecting.

How many jails are you managing? The best option would be to acquire more web-routable IP addresses - one for each jail - and do this properly. If that is not possible, you could probably work some magic with NAT where you pass requests to certain host system ports on to the jail(s).

 

[ondra_knezour]

How many jails are you managing? The best option would be to acquire more web-routable IP addresses - one for each jail - and do this properly.

Expecting dozens in couple of first weeks, going to hundreds quickly. I don't expect any request to RIPE like "Give us please 4 C IP blocks for our new AS" to be successfull.

If that is not possible, you could probably work some magic with NAT where you pass requests to certain host system ports on to the jail(s).

Only one "magic" I can imagine is specifying port range on "master machine" and force users to connect in way like % ssh -p [i]port_number[/i] [email]user_registered_under_example.net@example.com[/email]

But I want to allow them to connect "directly" to their machine in form % ssh [email]user@example.net[/email] where example.com is "master machine" and example.net is virtual machine used by given user.

Additionally, virtual machines will be "travelling" between couple of real hardware servers in something like let say cluster, with all login and configuration settings (for everything, not even user accounts) in LDAP, which makes configuration of any magical redirection of users connections request almost unmanamable, at least from my point of view.

 

[SirDice]

Forward port 2222 to your jail's port 22. Now your users can connect directly to the jail.

 

[ondra_knezour]

Forward port 2222 to your jail's port 22.

I know about this possibility, but I want to avoid it, as I wrote in my post.

force users to connect in way like % ssh [b]-p [i]port_number[/i][/b] [email]user_registered_under_example.net@example.com[/email]

There are two reasons why I don't want implement solution like this. Objective - I don't want administer such monster (consider hundreds of jails available to world this way) and subjective - I don't want to force users remember ports which belongs right to their jails, not mentioning ugly and unprofessional look of such solution.

 

[kdemidofff]

u need something like apache virtual hosts but for jail ssh

 

[anomie]

Right, the Apache virtualhost analogy is more or less apt. (The problem is I doubt that it exists in the openssh world. It would need to decrypt packets and analyze host names.)

If it would be impractical to have a hostort -> jail:ssh mapping for each jail, and it would be difficult to procure a web-routable IP for each jail, then is it possible jails are not the solution you need?

Would your users be just as well served by a basic openssh chrooted environment? (You alluded to that in your first post.) That seems trivial enough to implement for sftp usage. Complexity grows as shell needs grow.

If none of this is going to cut it, you may be hosting a lot of shell users on your host system, with other precautions in place to lock things down.

-------

I'd add: if you can get your hands on one more IP address, you could at least keep your shell users confined to a single jail (and out of the host system).

 

[kdemidofff]

one solution is to use VPN server to assign private IPs to clients matching their jail IP
also u can make apache virtual hosts with data directories in jails for web services but u need some kind of centralized admin panel

 

[ondra_knezour]

Right, the Apache virtualhost analogy is more or less apt. (The problem is I doubt that it exists in the openssh world. It would need to decrypt packets and analyze host names.)

It is something I meant under "ssh proxy" in my question, but as far as I know, there is no such solution in universe

[...]then is it possible jails are not the solution you need?

I want to prepare jail running apache with php, python and RoR, several sql databases etc. for each user. May be done without jails, but I want data of each single user as separated from others as possible.

Would your users be just as well served by a basic openssh chrooted environment? (You alluded to that in your first post.)

As for now, I'm considering this enough, just asking forum and myself about level of security and separation. What is my standard user expected to do using shell? Moving files, editing them on machine, mostly by midnight commander, restarting services running in his jail. As far as I understand solution provided by OpenSSH chrooted enviroment, it can by done this way securely.

if you can get your hands on one more IP address, you could at least keep your shell users confined to a single jail (and out of the host system).

I hope to have several IPs to separate most important services in their own jails, including one for this multihosting.

 

[ondra_knezour]

one solution is to use VPN server to assign private IPs to clients matching their jail IP[...]

Not applicable in this situation, mostly for two reasons. Most of users will consider VPN to be a rude word , not service and user should be able to connect to his data and services anywhere, just knowing his username and password. I know, that looking for internet cafe, which will allow you to use SSH wouln't be easy, but looking for some, which will allow you to establish own VPN connection and having all required keys/certs by yourself is mission impossible.

But anyway, thanks for your input.

 

[SirDice]

VPN don't require the use of certificates. It can work perfectly fine with just username/password.

 

[nORKy]

don't have any ipv6 range ?

 

[kdemidofff]

well other option is to use ssh ForceCommand with appropriate mapping to jexec (jailme maybe)
and also check sysutils/jailme
remember to disallow loopback connections from inside ssh
Also look this at serverfault, try to ask there too

 

[ondra_knezour]

VPN don't require the use of certificates. It can work perfectly fine with just username/password.

Didn't know that, we are using certificates everywhere, so this was unknown for me. But still consider VPN too beyond skills of my ordinary user.

don't have any ipv6 range ?

I do, but many of my clients doesn't.

well other option is to use ssh ForceCommand with appropriate mapping to jexec[...][/URL]

Such solution didn't ever come on my mind, will try it.

Thanks to all for input.

 

[allanjude@]

It is something I meant under "ssh proxy" in my question, but as far as I know, there is no such solution in universe

On top of the issues with this (which SSH finger print do you send when someone connects? how can a user be sure their connection is not being intercepted behind the proxy?), ssh doesn't send the equivalent to the HTTP Host header, so even if you had this, there would be no way to tell which internal ip/jail the user wants to be routed to.

I want to prepare jail running apache with php, python and RoR, several sql databases etc. for each user. May be done without jails, but I want data of each single user as separated from others as possible.

Running entirely separate daemons for 100+ jails seems like a lot of overkill. Database daemons are going to be particularly troublesome, you are going to either have to tune them to have very small buffers/caches so you don't run our of memory (this is wasteful of unused cache memory, for example if each db has 4mb of cache, instead of one big SQL server with 400mb of cache, if one db is not busy, and only using 2mb of its cache, and another is busy and could benefit from the additional 2mb of cache, this is lost). The same goes for the web server, like apache, if you are using mpm_prefork then you're going to have a bunch of idle apache processes sitting around not serving anything, whereas if they were all pooled, as with a regular virtual hosting environment then you make much more efficient use of the resources on your system. This doesn't seem like it will scale very well.

As for now, I'm considering this enough, just asking forum and myself about level of security and separation. What is my standard user expected to do using shell? Moving files, editing them on machine, mostly by midnight commander, restarting services running in his jail. As far as I understand solution provided by OpenSSH chrooted enviroment, it can by done this way securely.

If you could describe in a bit more detail what you are trying to accomplish, and why a typical shared web hosting environment is insufficient, we may be better able to assist you.

 

[kdemidofff]

On top of the issues with this (which SSH finger print do you send when someone connects? how can a user be sure their connection is not being intercepted behind the proxy?), ssh doesn't send the equivalent to the HTTP Host header, so even if you had this, there would be no way to tell which internal ip/jail the user wants to be routed to.

Running entirely separate daemons for 100+ jails seems like a lot of overkill. Database ..

1. look at previous posts about jexec
2. well u missed efficiency with SLAs

if i want private jail (lol) and make my set of services its my right.. with programs that i need and versions i need

VPS just provide CPU / MEM /FS not very efficient shared database or efficient shared web server.

It up to VPS provider to provide resources. Clients don't care much about other VPS using w/e web servers or databases... If their free resources get low they can just change provider...

Nowadays servers with 4-16 cores can provide platform for many VPSes just think how many P4x2Hz equals with Core 2 Quad or i7 and still P4 can serve many jails (web server + database forum + scripts) alone...

Of course there shall be policy not to run game hogs and so on because it really easy saturate CPU/RAM/FS etc.

But with just apache and reasonable database will not be problem - u just need to set RAM/CPU/FS limit reasonable (look what others providers put on it)... they put guranteed RAM and no guaranteed CPU details sample don't think u can saturate 20GB ram with given 256-768 mb limit

also on stackoverflow

 

[FryShadow]

just an option

create the user on your host that they can ssh, and put some code in .bash_profile to ssh again to your jails. I don't know if this is a good practice in term of security or else, but this what I do for my vps (xen) using shellinabox as the ssh tools

 

[allanjude@]

1. look at previous posts about jexec
2. well u missed efficiency with SLAs

if i want private jail (lol) and make my set of services its my right.. with programs that i need and versions i need

VPS just provide CPU / MEM /FS not very efficient shared database or efficient shared web server.

It up to VPS provider to provide resources. Clients don't care much about other VPS using w/e web servers or databases... If their free resources get low they can just change provider...

Nowadays servers with 4-16 cores can provide platform for many VPSes just think how many P4x2Hz equals with Core 2 Quad or i7 and still P4 can serve many jails (web server + database forum + scripts) alone...

Of course there shall be policy not to run game hogs and so on because it really easy saturate CPU/RAM/FS etc.

But with just apache and reasonable database will not be problem - u just need to set RAM/CPU/FS limit reasonable (look what others providers put on it)... they put guranteed RAM and no guaranteed CPU details sample don't think u can saturate 20GB ram with given 256-768 mb limit

also on stackoverflow

Unlike a typical VPS solutions, Jails were not ment to do that, and have no way to limit the CPU, RAM and FS (except a separate partition for each FS).

With a typical VPS, you can limit how much ram the processes in each container can have access to, this is not so with Jails. There is no current implementation of a way to limit how much ram a Jail can use.

Most Virtualization software also has the limitation that the entire VPS runs as a single thread on the host, and therefore can not make sure of more than 100% of a single CPU Core. Some offer the ability to virtualize multiple cores, but the best you can do is a 1:1 mapping to physical cores. This provides at least some level of control to keep a single VPS from using 100% of all CPU cores, Jails do not have this ability. Recently a feature was introduced to allow Jail's to be bound to a specific CPUSet, which does help a bit.

 

[kdemidofff]

my bad i missed VPS with jails xD