HTTPS certificates for local jailed services?

C

ccammack

I would like to configure https for some jailed services on a home server and am curious about my options. Here's what I have considered so far:
  1. Self-signed certificates
  2. Run a cron job in each jail that uses a letsencrypt ACME DNS-01 script and a DNS update script to keep the certs updated. This would require me to hardcode the DNS credentials in all of the scripts.
  3. Run #2 for all of the jails on the host or in a dedicated cert-management jail and copy the output certificates to the other jails using ssh. This would protect the DNS credentials a little better.
Are there any other options?
 
I have a HAProxy running on my firewall/router that connects to the internet. I've setup letsencrypt certificates for some local websites. This has three major advantages, first is that I access those local websites from anywhere on the internet (only have 1 IP address on my home connection). The second reason is that I can off-load the SSL/HTTPS on HAProxy (and refresh the certificates there). Third reason is that I can access those internal websites through their external DNS entry without having to hairpin NAT or some split-horizon DNS.
 
Thanks, VladiBG, I had not heard of Webroot before. I think CloudFlare Tunnel might do something similar. Do you renew your certificates using a script or is that included with Webroot?

Thanks, SirDice and sko. I have some experience with Caddy on a VPS and your approaches sound similar.

My router is a consumer-level Ubiquiti ER-X. I'm not sure I want to install additional software on there in case I have trouble and need to undo it. To experiment with reverse proxies, would it be reasonable to try them out in a dedicated jail and just port-forward the router to the jail? That, way I can just close the port(s) if things go wrong.

Does anyone bother to use HTTPS for services that live entirely behind the firewall and are not exposed to the world? If so, is there an easy/centralized way to manage and renew their certificates?
 
When you need only a single certificate for a host and not a wildcard certificate you can use webroot plugin which will create a temporary acme challenge file inside your web directory which is then used for verify the ownership of the domain. This work only for a simple single host certificate.

For example you need certificate for "www" host on "example.com" domain. (https://www.example.com). Using certbot webroot plugin will create a file in http://www.example.com/.well-known/acme-challenge which is verified by letsencrypt and they can issue a single CN certificate to www.example.com

It will also work for multiple hosts like "ftp.example.com" "mail.example.com" but if you have many of them it's better to get a single wildcard certificate *.example.com using DNS plugin instead of a single server certificate using webroot. The difference is that in the DNS based verification you create a TXT DNS record "_acme-challenge.example.com" in your DNS zone which is then used for verification of ownership of that domain "example.com" So this creation of the DNS record is a bit tricky as not all DNS hosting providers have a API to modify the DNS records. Some of them give only web based access to edit your DNS zone and it will be pain to renew the wildcard certificate every time by hand it's better to change your DNS provider to someone who have programable interface and provide a way to create a DNS record using script instead of modify it manual.

For single subdomain certificate

For wildcard certificate

Official guide:

Certbot Instructions

Tagline
certbot.eff.org certbot.eff.org
 
ccammack said:
My router is a consumer-level Ubiquiti ER-X. I'm not sure I want to install additional software on there in case I have trouble and need to undo it. To experiment with reverse proxies, would it be reasonable to try them out in a dedicated jail and just port-forward the router to the jail? That, way I can just close the port(s) if things go wrong.
Click to expand...
I'm not running anything else than routing/firewalling and at most some basic services like DHCP and DNS on my home routers (at work they *only* do routing and peering); so at home with a single IP anything else is simply forwarded to a dedicated jail on some server. This way you can also do e.g. major upgrades by simply building a new jail and then changing the forward rule.
But TBH - acme.sh is simply a shell script and has zero dependencies, so this won't hurt to run on a small router system (but its impractical if you then have to get the certs off that host). certbot is another beast, as it drags in ~130MB of dependencies (python...), for the simple task of just talking to some REST API, i.e. send some http POSTs and GETs.

(PS: I used an Ubiqiti ER-4 for a while at home running OpenBSD. IIRC most of the ERs can run OpenBSD or even FreeBSD, so you get a proper OS with PF and whatnot)

ccammack said:
Does anyone bother to use HTTPS for services that live entirely behind the firewall and are not exposed to the world? If so, is there an easy/centralized way to manage and renew their certificates?
Click to expand...
That is my main goal with that setup - I only want a single point for certificate management on a system. This might be the reverse-proxy jail or, if I have several jails that need certificates (e.g. a mailserver), acme.sh runs on the host itself and the certs are simply copied into the jails. Because jailhosts are usually never directly reachable from the outside, this variant even adds another safety layer to the certificate management.
I'm only using DNSAPI-mode, because it is 100% reliable and can run on any host or jail that has outgoing internet access. The various webserver- or standalone modes are dependent on various things, especially that the host that runs acme also needs to be accessible from the outside world.
 
VladiBG said:
When you need only a single certificate for a host and not a wildcard certificate you can use webroot plugin which will create a temporary acme challenge file inside your web directory which is then used for verify the ownership of the domain. This work only for a simple single host certificate.

It will also work for multiple hosts like "ftp.example.com" "mail.example.com" but if you have many of them it's better to get a single wildcard certificate *.example.com using DNS plugin instead of a single server certificate using webroot.
Click to expand...
Thank you for explaining the details, VladiBG. I think the DNS approach will work well in my case to generate wildcards and keep them updated with scripts even for hosts that are not exposed to the world.
 
sko said:
I'm not running anything else than routing/firewalling and at most some basic services like DHCP and DNS on my home routers (at work they *only* do routing and peering); so at home with a single IP anything else is simply forwarded to a dedicated jail on some server. This way you can also do e.g. major upgrades by simply building a new jail and then changing the forward rule.
Click to expand...
I understand now; forwarding to an internal jail seems like a good way to get started with this. I don't mind the extra routing for the sake of simplicity. Eventually I'll want to try this with a jailed WireGuard client and close those ports again (or a jailed WireGuard server and leave them open).

sko said:
certbot is another beast, as it drags in ~130MB of dependencies (python...), for the simple task of just talking to some REST API, i.e. send some http POSTs and GETs.

(PS: I used an Ubiqiti ER-4 for a while at home running OpenBSD. IIRC most of the ERs can run OpenBSD or even FreeBSD, so you get a proper OS with PF and whatnot)
Click to expand...
I have seen similar complaints about Caddy on the ER-X. These devices don't have much room to spare. Next time I upgrade I'll look for something that can run a BSD.

sko said:
if I have several jails that need certificates (e.g. a mailserver), acme.sh runs on the host itself and the certs are simply copied into the jails
Click to expand...
Thanks, that's what I was wondering about with option #3: manage the certs in one place and copy them out where needed. Do you copy the certs with shell scripts or Ansible or some other way?

I find myself creating dedicated users to run a small number of admin commands over ssh, wondering if there's a better/centralized way to manage them. I'm not an admin, so I don't know what the pros and semi-pros use for this kind of thing that might also work at home.
 
ccammack said:
Thanks, that's what I was wondering about with option #3: manage the certs in one place and copy them out where needed. Do you copy the certs with shell scripts or Ansible or some other way?
Click to expand...
acme.sh handles this all; you simply specify a path for each cert/key/fullchain file and a 'reloadcmd' when first issueing the cert, and this is saved in a config file and used during all certificate renewals. Anything that needs manual intervention is worthless, as it will be forgotten or you are on holiday/sick/abducted by aliens when the renewal is due.

I usually use one cert per jail (which translates to one per earch service) and save that (sometimes rather long) one-liner in a shellscript file named after the jail (service), so I can easily review or change the options, like e.g. adding other alias domains to the cert (often needed for a reverse-proxy jail).

e.g. this is the script I used on one of my hosts for the initial cert request for the nginx jail:
Code: 
#!/bin/sh
export BUNNY_API_KEY='MYSECRETAPIKEY'
acme.sh --force --issue --dns dns_bunny --dnssleep 20 --server letsencrypt\
    -d webmail.mydomain.tld\
    -d groupoffice.mydomain.tld\
    -d nginx.mail.mydomain.tld\
    -d groupoffice.mail.mydomain.tld\
    --keylength ec-384\
    --cert-file "/iocell/jails/ef06ba7a-c5a1-11ee-8adb-002590fc5b0a/root/usr/local/etc/nginx/ssl/cert"\
    --key-file "/iocell/jails/ef06ba7a-c5a1-11ee-8adb-002590fc5b0a/root/usr/local/etc/nginx/ssl/key"\
    --fullchain-file "/iocell/jails/ef06ba7a-c5a1-11ee-8adb-002590fc5b0a/root/usr/local/etc/nginx/ssl/fullchain"\
    --reloadcmd "service -j ioc-ef06ba7a-c5a1-11ee-8adb-002590fc5b0a nginx restart"
You only have to run this once; its pretty much 'fire and forget' - just add the cronjob as per the pkg message and acme.sh will check daily for any cert that needs to be renewed.
 
I see. One cert per service, and you just write the certs directly into the jail on the host and restart. I'm still using iocage, so this will adapt nicely. This is great. Thank you for showing me the details.
 
You must log in or register to reply here.
Back
Top