Musings of a noob as I migrate from Windows to FreeBSD in my homelab

Service deployment update: OpenLDAP + Kerberos is up and running (4 of 4)​


Problems I encountered (or created for myself) getting surfrock’s prescription for a KDC+LDAP up and running​

In no particular order (at least not chronological), here’s a sampling of what went wrong during this adventure:
  • The #1 issue I had was typos, of course. I think I set a personal best (worst?) for fat-fingered mistakes. It doesn’t help that the Rx for my glasses is long-overdue for an update, making it easier to misread the config I’m working on (multiple times!) and search elsewhere for a fix, when the real fix was staring me right in the face.
    Example: My first attempt at parsing the initial .ldif into a starter database puked. So did my 2nd, and even my 3rd. The reason? Stupid things like mis-typing “olcDatabase” as “oldDatabase”. I was using vim to edit, so I can’t blame autocorrect… eventually I got rid of the gross typos and was left with more subtle ones (like not being consistent on how I specified the DNs for users)​
    Example #2: # service slapd start failed the first time with the error error: grep: /usr/local/etc/openldap/slapd.d/cn=config/olcDatabase=*: No such file or directory. Another stupid compound typo… when I created the folder /usr/local/etc/openldap/slapd.d/, I mis-named it slap.d/. Not only that, but my slapadd command to parse the ldif file into the database folder had the same typo, so it worked (the folder existed). Of course, rc went looking for the correct folder name and couldn’t find it…​
  • TLS didn’t work out of the box, for several reasons:
    1. I forgot to import my CA certificates into the LDAP jail. Easily fixed.
    2. I followed the FreeBSD Handbook’s instructions for hashing the server certificate (found in the Chapter 32.5 discussion of OpenLDAP), which it turns out is out of date. The command c_rehash . has been depricated and removed from the openssl package. The correct command to use is openssl rehash .
    3. I followed JamieLinux’s advice on creating a single .pem with the complete CA chain: intermediate CA cert listed first, followed by the root CA. OpenLDAP apparently can’t read a .pem with more than a single certificate in it. So, I split the CA chain cert into separate certs for each CA and only listed the intermediate CA in the olcTLSCACertificateFile: attibute in slapd.ldif. Seems to work fine… both CAs have their certs in /usr/share/certs/trusted/, and I made sure those were recognized by the system using certctl rehash.
    4. The FreeBSD handbook section on OpenLDAP doesn’t mention that TLS needs to be explicitly enabled in /etc/rc.conf. Specifically, slapd_flags needs to also have an entry ldaps://0.0.0.0/. If that’s not there, rc won’t launch the service with TLS enabled and there’s no possiblity of getting it to work. This was the last piece I needed to get secure TLS communication working.
  • The first time I tried to modify the database using ldapadd -Y EXTERNAL -H ldapi:/// -f 01.config.InstallMemberOf.ldif it failed with an insufficient access error. This one took me a bit to figure out, and I learned something along the way (bonus!). My first slapd.ldif was based on the example provided with the FreeBSD openldap26-server pkg install. That example does not allow this command to run out of the box.
    Explanation: using the -Y EXTERNAL flag tells ldapadd (or ldapmodify) to use the SASL layer to map the root user (uid=0,gid=0) to an LDAP-formated DN, and then asks LDAP to run the command using that DN for credentials. The DN format from SASL doesn’t match the global config administrator (cn=admin,cn=config), so the command is rejected.​
    Solution: I went off to do quite a bit of reading, which eventually led me to taking a look at the template slapd.ldif provided by Ubuntu. They include an ACL that explicitly allows the SASL-formatted DN to make management-level changes: olcAccess: to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break. Applying this to both the frontend and config database sections solved the issue. Given that the Ubuntu template seems a little more modern and full-featured than the one that FreeBSD ships, I switched my entire template over to the Ubuntu version and customized from there.​
  • The kadmind service failed to start on the first try with the error kadmind: LDAP bind dn value missing while initializing, aborting.
    • This was by far the most insideous problem I had to solve, and it took me a little over a week (I have a day job) to get past it. The issue wasn’t anything to do with Ubuntu vs FreeBSD minutae, typos, incomplete / out of date FreeBSD Handbook entries, or anything along those lines. What I finally tracked down makes me wonder about how package maintainers make their decisions on which options to include, however.
      The core issue: Take a look at surfrock66’s definitions for his user (and service) account DNs. You’ll notice that all his DNs have cn=somename,… However, if you read the OpenLDAP section on SASL authentication with GSSAPI, it explicitly says that the form of the DNs being used by SASL is uid=somename,…​
      Somehow, surfrock66’s setup is working using a nonstandard DN format.
      I went through and modified all the ldif files to use the uid convention for account DNs – making sure to explicitly change not just the accounts but all the ACLs that have them hardcoded in. Once I did this (basically required a rebuild), kadmind started without issue.​
      Conclusion? I can’t say for sure, but one explanation that comes to mind is that there’s a strict name-checking option enabled in the FreeBSD version of either cyrus-sasl-gssapi or openldap26-server that’s not enabled in the Ubuntu versions. Either that or I went completely crazy, because switching from the cn= to uid= version of account DNs was the only change I made, and kadmind went from not working to working. I’m glad it works for surfrock66, but if you want to copy his setup on FreeBSD you’ll need to do what I did and switch to uid=.​
    • Of course, there were the expected Ubuntu vs FreeBSD differences in paths for executables, logs, config files, along with differences in how flags get passed to services during startup. That’s all straightforward to figure out, if a little tedious.
    So, that’s the flavor of what I went through… to anyone looking to do this yourself, good luck! Take your time and be prepared to learn more than you’d like to. Not much of this is very well documented. I found myself re-reading the same man pages multiple times over different days to try and suss out how things should be working.

    Next up: enable pf on the LDAP/Kerberos jail (it’s wide open atm). Once that’s tightened up I’ll create a backup LDAP service in a jail on my other box and get 1-way replication working. Finally, a secondary KDC goes in that jail, and my setup will be done.

    My testing laptop running 15.0-RELEASE is reading from the directory and I can log in using my “firstuser” account on the laptop. All directory-specified items work: home directory, login shell, access to sudo privilages, etc.
 
Last edited:
Squaring the circle, because I feel a bit guilty about not posting a client config. Here's what got my test laptop integrated with the Kerberos+LDAP system:
  • # pkg install openldap26-client cyrus-sasl-gssapi krb5 pam_ldap sssd2 sudo-sssd
  • Copy your CA certificate(s) into /usr/share/certs/trusted/ and run # certctl rehash
  • Edit the following files:
    • /usr/local/etc/openldap/ldap.conf
      TLS_CACERTDIR /usr/share/certs/trusted/
      TLS_CACERT /usr/share/certs/trusted/intermediateca.mydomain.com.cert.pem
    • /etc/krb5.conf
      [logging]
      default = FILE:/var/log/krb5.log​
      [libdefaults]
      default_realm = MYDOMAIN.COM
      dns_lookup_realm = true
      dns_lookup_kdc = true
      kdc_timesync = 1
      ccache_type = 4
      forwardable = true
      proxiable = true
      fcc-mit-ticketflags = true​
      [realms]
      MYDOMAIN.COM = {​
      kdc = ldapjail.mydomain.com
      admin_server = ldapjail.mydomain.com
      default_domain = mydomain.com​
      }​
      [domain_realm]
      .mydomain.com = MYDOMAIN.COM
      mydomain.com = MYDOMAIN.COM​
    • /etc/nsswitch.conf
      group: files sss
      hosts: files dns
      netgroup: files sss
      networks: files
      passwd: files sss
      shells: files sss
      services: files sss
      protocols: files
      rpc: files
      sudoers: files sss
    • /usr/local/etc/sssd/sssd.conf
      [sssd]
      services= nss, pam, sudo
      config_file_version = 2
      domains = MYDOMAIN.COM
      debug_level=4

      [nss]
      shell_fallback=/bin/sh

      [sudo]
      debug_level=4

      [domain/MYDOMAIN.COM]
      debug_level=4
      cache_credentials = False
      enumerate = True
      id_provider = ldap
      sudo_provider = ldap
      ldap_uri = ldap://ldapjail.mydomain.com
      ldap_search_base = dc=mydomain,dc=com
      ldap_group_search_base = ou=groups,dc=mydomain,dc=com
      ldap_sudo_search_base = ou=SUDOers,dc=mydomain,dc=com
      ldap_sudo_full_refresh_interval = 86400
      ldap_sudo_smart_refresh = 3600
      ldap_schema = rfc2307bis
      ldap_user_object_class = domainAccount
      ldap_group_object_class = domainGroup
      ldap_group_member = member
      ldap_default_bind_dn = uid=ldapbinduser,ou=accounts,dc=mydomain,dc=com
      ldap_default_authtok = Plain_text_password_here
      ldap_id_use_start_tls = True
      ldap_tls_reqcert = demand
      ldap_tls_cacert = /usr/share/certs/trusted/intermediateca.mydomain.com.cert.pem
      ldap_tls_cacertdir = /usr/share/certs/trusted
      auth_provider = krb5
      chpass_provider = krb5
      krb5_server = ldapjail.mydomain.com
      krb5_kpasswd = ldapjail.mydomain.com
      krb5_realm = MYDOMAIN.COM
      create_homedir = False

      [pam]
      pam_passkey_auth = False
    • /etc/pam.d/sudo
      auth sufficient pam_unix.so no_warn
      auth sufficient pam_sss.so use_first_pass

      account required pam_unix.so no_warn use_first_pass

      session required pam_permit.so
    • /etc/pam.d/sddm
      auth sufficient pam_sss.so try_first_pass
      auth required pam_unix.so no_warn try_first_pass

      account required pam_unix.so
      account sufficient pam_sss.so

      password required pam_unix.so no_warn try_firts_pass
      password sufficient pam_sss.so

      session required pam_permit.so
      session optional pam_sss.so
    • Add sssd_enable="YES" to /etc/rc.conf
  • Reboot
I'm using SDDM, so obviously swap this out and modify as appropriate for whatever display manager you're using in your setup. Also, note that I've set create_homedir=False in /usr/local/etc/sssd/sssd.conf. I did this because I didn't have time to play around with the pam_zfs_key.so module that supports creation of encrypted /home/$user datasets on first login. Given the extremely low number of people using my systems (4 total, and 2 of those almost never), I'm fine with manually setting up home datasets for family before they can log on to a host. There really won't be that many hosts that allow local logon... if the situation ever changes I'll dig in and look at enabling automatic home folder / dataset creation, but for now I'll skip that potential headache.

Happy holidays!
 
Last edited:

Interim update: phpldapadmin is working​

No progress on getting the secondary LDAP + Kerberos jail up on my junky backup services box... I got sidetracked trying to get phpldapadmin working. It's a long boring tale of incompetence, punctuated by bouts of apathy and a holiday break that saw me distracted by family obligations during what could have been "me time."

I realized shortly after getting my LDAP+Kerberos setup running that it's going to be a nightmare to administer if I can't visualize the directory tree. I looked high and low for good tools, and unfortunately the only ones that support custom schema also charge a hefty subscription fee, save one: phpLDAPadmin. The phpLDAPadmin port is very much long in the tooth - the last update was in Feb '24 and it's only at version 1.2.6.7. The current version of phpLDAPadmin has changed rev #s and now sits at version 2.3.8 as of 2 weeks ago. Since it appears the FreeBSD port is well on its way to becoming abandoned, I decided to strike out on my own and run the current version from sources. I had a spare jail already set up - this jail was going to be a standalone KDC, but since I sucked that role into my LDAP jail I didn't have a purpose for this one.

So... off I went. I followed the instructions on the phpLDAPadmin github, installing php84, composer, npm, etc and attempted to get the source to compile. Maybe it did? I don't really know... I always saw a slew of warnings flash by during the process, but it completed. I guess that could be ok... the real problem is that my web server admin chops are LONG GONE. I struggled and failed to get php working properly in either nginx or apache (tried one and then the other in this jail). Some of it was undoubtedly not having my environment variables for phpldapadmin set properly, so the php side was erroring out trying to talk to my openldap server. Regardless, I came to the realization that I just didn't care enough about the FreeBSD purity of my homelab setup to keep going down the roll-your-own route for phpldapadmin. The truth is that I'll use it almost never - once user accounts are in place (I only have 3 to go), the only times I'll ever need it is when adding a new host - I don't expect that to happen often. I had a spare rpi3B in a drawer, so I cheated and went the Docker on raspbian route.

This was my first experience with Docker, and there's some things I like and things I hate already. Overall I'd say I prefer vnet jails hands down for containerization. However, getting phpldapadmin working was easy (someone did the work for me), and after some struggles I managed to get an nginx reverse proxy to sit in front so that the web site is secured by SSL. That's important because username and password are being sent through the login web form, and I don't want those clear text on the wire. phpldapadmin on its own only provides http. The biggest problem I had was getting the backend LDAPS communication working - in order for that to work properly the phpldapadmin container needed to trust the cert being offered up by my openLDAP jail, and it wasn't as easy as I'd thought it should be to get my root CA certificates into the phpldapadmin container. At any rate, all is working now, so I can get back to working on my core setup.

Call this my first real fail, in that I didn't manage to do what I want with FreeBSD. The only consolation is that now I know a bit about Docker, and it's reinforced my conviction that FreeBSD is the saner path for my core services deployments. Also, this will almost never get used, so I feel ok about cheating a bit. Slippery slope, you say? Not really... my experience with Docker so far hasn't exactly impressed me, and I doubt I'll dabble again unless I encounter a similar ultra-low-use, throwaway but nice-to-have thing I want.
 
This comment is so the OP knows that this person that happens to be me finds this thread very original and interesting, that I think that the OP writes very well, and that I'm following the OP's future installments even though I might not understand every technical detail.
 
Long pause, but I'm getting back into my setup. I had to divert my attention to dealing with some home automation issues... I'd been using a "beta" all-in-one appliance that integrated Zigbee2MQTT, Zwave-JS UI, Mosquitto, Node-Red, and some other quality-of-life packages on a little SBC running podman on Manjaro. It was a nice idea, but it became abandonware when the developers all peeled off before finishing. Promises that an update and/or instructions would be provided to decouple the platform from the update server weren't kept, and the inevitable happened. One of the packages (Zwave-JS UI) partially failed. It still spoke to devices in the mesh and communicated to MQTT, but the GUI wouldn't work and that became a deal breaker. So... I doubled down on my docker adventure and built a new HA device on an rpi with my 4 containers (z2m, zwave, MQTT, Node-Red) running on it. The painful part was moving over all my Zigbee and Zwave devices... took forever. I could have done it faster if I hadn't gotten sucked in to re-watching Breaking Bad and Better Call Saul... I'm not great with priorities.

At any rate, the home automation problem is fixed (abandonware box has been relegated to my e-waste pile), so I started thinking about my homelab setup again. During the time I was working on the HA upgrade I also came to the realization that forcing my laptop to run FreeBSD was the wrong choice. Matlab performance the way I'd kludged it together was just not acceptable, and this machine has to be able to run that software. So, I went for CachyOS (Arch-based distro) because it has root-on-ZFS built in to the installer. While I'm willing to roll up my sleeves and work for it with FreeBSD, when it comes to a Linux client my attitude is that I shouldn't have to pay as much attention or focus on becoming expert in every aspect of the system. There are enough good options out there that I should be able to just pick one that offers what I want without needing to jump through hoops and work hard to maintain a custom setup. What do I want? Decent compute and graphics performance for Matlab, root-on-ZFS so I can easily automate system backups with ZFS send/receive (and also protect me from myself using snapshots), and out-of-the-box desktop that doesn't need me to make endless tweaks to get a usable experience. CachyOS was the closest I could find to fitting those requirements... the only thing that bugs me is their installer is extremely limited in how datasets are set up. Good enough for now

The "fun" came when I tried to integrate the CachyOS client with my LDAP/Kerberos setup. I think I stepped on every rake and fell face first into every cow pie... it did not go well. I blame systemd, for no reason in particular... actually I have a reason. Take a look at nsswitch.conf as it ships with CachyOS and tell me you're not confused. It's way more complex than it needs to be. Deciding to simplify is a great way to make sure you can't log in to the system at all... I had to boot into a live USB many times to fix one blunder after another. In the end, getting my CachyOS client to work was simpler than I thought - the key was to make minimal changes (duh).

This being a FreeBSD forum, I won't bother with a detailed recreation of the setup for a CachyOS/Arch LDAP/KRB5 client. Highlights are:
  • Install openldap, sssd, and krb5 packages
  • Install CA certificate(s) and update the system cache
  • Use https://forums.freebsd.org/threads/...ws-to-freebsd-in-my-homelab.98860/post-734041 as a starting point.
    • ldap.conf, sssd.conf, and krb5.conf are identical as the previous post. Note I did change to create_homedir = True in sssd.conf for this client.
    • for nssswitch.conf, just add "sss" (no quotes) at the end of the entries for passwd, group, shadow, services, and netgroup. Also add a new line with sudoers: files sss
  • PAM only requires edits in /etc/pam.d/system-auth. This gets included in everything else so by making the modifications here you won't have to touch any other PAM config files.
    • I've attached the file (with a txt extension so it'd attach) - ymmv. Use at your own risk.
  • Enable sssd
Going back to my earliest posts, it was always my intention to have one or more Linux clients, so I'm glad I have this worked out. Next up: this weekend I'm going to force myself to finish getting the secondary LDAP/Kerberos jail set up so that I can move on and restore my network shares and files before finally moving my wife's primary PC over to the new setup.
 

Attachments

Back
Top