Musings of a noob as I migrate from Windows to FreeBSD in my homelab

Service deployment update: OpenLDAP + Kerberos is up and running (4 of 4)​


Problems I encountered (or created for myself) getting surfrock’s prescription for a KDC+LDAP up and running​

In no particular order (at least not chronological), here’s a sampling of what went wrong during this adventure:
  • The #1 issue I had was typos, of course. I think I set a personal best (worst?) for fat-fingered mistakes. It doesn’t help that the Rx for my glasses is long-overdue for an update, making it easier to misread the config I’m working on (multiple times!) and search elsewhere for a fix, when the real fix was staring me right in the face.
    Example: My first attempt at parsing the initial .ldif into a starter database puked. So did my 2nd, and even my 3rd. The reason? Stupid things like mis-typing “olcDatabase” as “oldDatabase”. I was using vim to edit, so I can’t blame autocorrect… eventually I got rid of the gross typos and was left with more subtle ones (like not being consistent on how I specified the DNs for users)​
    Example #2: # service slapd start failed the first time with the error error: grep: /usr/local/etc/openldap/slapd.d/cn=config/olcDatabase=*: No such file or directory. Another stupid compound typo… when I created the folder /usr/local/etc/openldap/slapd.d/, I mis-named it slap.d/. Not only that, but my slapadd command to parse the ldif file into the database folder had the same typo, so it worked (the folder existed). Of course, rc went looking for the correct folder name and couldn’t find it…​
  • TLS didn’t work out of the box, for several reasons:
    1. I forgot to import my CA certificates into the LDAP jail. Easily fixed.
    2. I followed the FreeBSD Handbook’s instructions for hashing the server certificate (found in the Chapter 32.5 discussion of OpenLDAP), which it turns out is out of date. The command c_rehash . has been depricated and removed from the openssl package. The correct command to use is openssl rehash .
    3. I followed JamieLinux’s advice on creating a single .pem with the complete CA chain: intermediate CA cert listed first, followed by the root CA. OpenLDAP apparently can’t read a .pem with more than a single certificate in it. So, I split the CA chain cert into separate certs for each CA and only listed the intermediate CA in the olcTLSCACertificateFile: attibute in slapd.ldif. Seems to work fine… both CAs have their certs in /usr/share/certs/trusted/, and I made sure those were recognized by the system using certctl rehash.
    4. The FreeBSD handbook section on OpenLDAP doesn’t mention that TLS needs to be explicitly enabled in /etc/rc.conf. Specifically, slapd_flags needs to also have an entry ldaps://0.0.0.0/. If that’s not there, rc won’t launch the service with TLS enabled and there’s no possiblity of getting it to work. This was the last piece I needed to get secure TLS communication working.
  • The first time I tried to modify the database using ldapadd -Y EXTERNAL -H ldapi:/// -f 01.config.InstallMemberOf.ldif it failed with an insufficient access error. This one took me a bit to figure out, and I learned something along the way (bonus!). My first slapd.ldif was based on the example provided with the FreeBSD openldap26-server pkg install. That example does not allow this command to run out of the box.
    Explanation: using the -Y EXTERNAL flag tells ldapadd (or ldapmodify) to use the SASL layer to map the root user (uid=0,gid=0) to an LDAP-formated DN, and then asks LDAP to run the command using that DN for credentials. The DN format from SASL doesn’t match the global config administrator (cn=admin,cn=config), so the command is rejected.​
    Solution: I went off to do quite a bit of reading, which eventually led me to taking a look at the template slapd.ldif provided by Ubuntu. They include an ACL that explicitly allows the SASL-formatted DN to make management-level changes: olcAccess: to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break. Applying this to both the frontend and config database sections solved the issue. Given that the Ubuntu template seems a little more modern and full-featured than the one that FreeBSD ships, I switched my entire template over to the Ubuntu version and customized from there.​
  • The kadmind service failed to start on the first try with the error kadmind: LDAP bind dn value missing while initializing, aborting.
    • This was by far the most insideous problem I had to solve, and it took me a little over a week (I have a day job) to get past it. The issue wasn’t anything to do with Ubuntu vs FreeBSD minutae, typos, incomplete / out of date FreeBSD Handbook entries, or anything along those lines. What I finally tracked down makes me wonder about how package maintainers make their decisions on which options to include, however.
      The core issue: Take a look at surfrock66’s definitions for his user (and service) account DNs. You’ll notice that all his DNs have cn=somename,… However, if you read the OpenLDAP section on SASL authentication with GSSAPI, it explicitly says that the form of the DNs being used by SASL is uid=somename,…​
      Somehow, surfrock66’s setup is working using a nonstandard DN format.​
      I went through and modified all the ldif files to use the uid convention for account DNs – making sure to explicitly change not just the accounts but all the ACLs that have them hardcoded in. Once I did this (basically required a rebuild), kadmind started without issue.​
      Conclusion? I can’t say for sure, but one explanation that comes to mind is that there’s a strict name-checking option enabled in the FreeBSD version of either cyrus-sasl-gssapi or openldap26-server that’s not enabled in the Ubuntu versions. Either that or I went completely crazy, because switching from the cn= to uid= version of account DNs was the only change I made, and kadmind went from not working to working. I’m glad it works for surfrock66, but if you want to copy his setup on FreeBSD you’ll need to do what I did and switch to uid=.​
    • Of course, there were the expected Ubuntu vs FreeBSD differences in paths for executables, logs, config files, along with differences in how flags get passed to services during startup. That’s all straightforward to figure out, if a little tedious.
    So, that’s the flavor of what I went through… to anyone looking to do this yourself, good luck! Take your time and be prepared to learn more than you’d like to. Not much of this is very well documented. I found myself re-reading the same man pages multiple times over different days to try and suss out how things should be working.

    Next up: enable pf on the LDAP/Kerberos jail (it’s wide open atm). Once that’s tightened up I’ll create a backup LDAP service in a jail on my other box and get 1-way replication working. Finally, a secondary KDC goes in that jail, and my setup will be done.

    My testing laptop running 15.0-RELEASE is reading from the directory and I can log in using my “firstuser” account on the laptop. All directory-specified items work: home directory, login shell, access to sudo privilages, etc.
 
Last edited:
Squaring the circle, because I feel a bit guilty about not posting a client config. Here's what got my test laptop integrated with the Kerberos+LDAP system:
  • # pkg install openldap26-client cyrus-sasl-gssapi krb5 pam_ldap sssd2 sudo-sssd
  • Copy your CA certificate(s) into /usr/share/certs/trusted/ and run # certctl rehash
  • Edit the following files:
    • /usr/local/etc/openldap/ldap.conf
      TLS_CACERTDIR /usr/share/certs/trusted/
      TLS_CACERT /usr/share/certs/trusted/intermediateca.mydomain.com.cert.pem
      BASE dc=mydomain,dc=com
      URI ldap://ldapjail.mydomain.com
    • /etc/krb5.conf
      [logging]
      default = FILE:/var/log/krb5.log​
      [libdefaults]
      default_realm = MYDOMAIN.COM
      dns_lookup_realm = true
      dns_lookup_kdc = true
      kdc_timesync = 1
      ccache_type = 4
      forwardable = true
      proxiable = true
      fcc-mit-ticketflags = true​
      [realms]
      MYDOMAIN.COM = {​
      kdc = ldapjail.mydomain.com
      admin_server = ldapjail.mydomain.com
      default_domain = mydomain.com​
      }​
      [domain_realm]
      .mydomain.com = MYDOMAIN.COM
      mydomain.com = MYDOMAIN.COM​
    • /etc/nsswitch.conf
      group: files sss
      hosts: files dns
      netgroup: files sss
      networks: files
      passwd: files sss
      shells: files sss
      services: files sss
      protocols: files
      rpc: files
      sudoers: files sss
    • /usr/local/etc/sssd/sssd.conf
      [sssd]
      services= nss, pam, sudo
      config_file_version = 2
      domains = MYDOMAIN.COM
      debug_level=4

      [nss]
      shell_fallback=/bin/sh

      [sudo]
      debug_level=4

      [domain/MYDOMAIN.COM]
      debug_level=4
      cache_credentials = False
      enumerate = True
      id_provider = ldap
      sudo_provider = ldap
      ldap_uri = ldap://ldapjail.mydomain.com
      ldap_search_base = dc=mydomain,dc=com
      ldap_group_search_base = ou=groups,dc=mydomain,dc=com
      ldap_sudo_search_base = ou=SUDOers,dc=mydomain,dc=com
      ldap_sudo_full_refresh_interval = 86400
      ldap_sudo_smart_refresh = 3600
      ldap_schema = rfc2307bis
      ldap_user_object_class = domainAccount
      ldap_group_object_class = domainGroup
      ldap_group_member = member
      ldap_default_bind_dn = uid=ldapbinduser,ou=accounts,dc=mydomain,dc=com
      ldap_default_authtok = Plain_text_password_here
      ldap_id_use_start_tls = True
      ldap_tls_reqcert = demand
      ldap_tls_cacert = /usr/share/certs/trusted/intermediateca.mydomain.com.cert.pem
      ldap_tls_cacertdir = /usr/share/certs/trusted
      auth_provider = krb5
      chpass_provider = krb5
      krb5_server = ldapjail.mydomain.com
      krb5_kpasswd = ldapjail.mydomain.com
      krb5_realm = MYDOMAIN.COM
      create_homedir = False

      [pam]
      pam_passkey_auth = False
    • /etc/pam.d/sudo
      auth sufficient pam_unix.so no_warn
      auth sufficient pam_sss.so use_first_pass

      account required pam_unix.so no_warn use_first_pass

      session required pam_permit.so
    • /etc/pam.d/sddm
      auth sufficient pam_sss.so try_first_pass
      auth required pam_unix.so no_warn try_first_pass

      account required pam_unix.so
      account sufficient pam_sss.so

      password required pam_unix.so no_warn try_firts_pass
      password sufficient pam_sss.so

      session required pam_permit.so
      session optional pam_sss.so
    • Add sssd_enable="YES" to /etc/rc.conf
  • Reboot
I'm using SDDM, so obviously swap this out and modify as appropriate for whatever display manager you're using in your setup. Also, note that I've set create_homedir=False in /usr/local/etc/sssd/sssd.conf. I did this because I didn't have time to play around with the pam_zfs_key.so module that supports creation of encrypted /home/$user datasets on first login. Given the extremely low number of people using my systems (4 total, and 2 of those almost never), I'm fine with manually setting up home datasets for family before they can log on to a host. There really won't be that many hosts that allow local logon... if the situation ever changes I'll dig in and look at enabling automatic home folder / dataset creation, but for now I'll skip that potential headache.

Happy holidays!
 
Last edited:

Interim update: phpldapadmin is working​

No progress on getting the secondary LDAP + Kerberos jail up on my junky backup services box... I got sidetracked trying to get phpldapadmin working. It's a long boring tale of incompetence, punctuated by bouts of apathy and a holiday break that saw me distracted by family obligations during what could have been "me time."

I realized shortly after getting my LDAP+Kerberos setup running that it's going to be a nightmare to administer if I can't visualize the directory tree. I looked high and low for good tools, and unfortunately the only ones that support custom schema also charge a hefty subscription fee, save one: phpLDAPadmin. The phpLDAPadmin port is very much long in the tooth - the last update was in Feb '24 and it's only at version 1.2.6.7. The current version of phpLDAPadmin has changed rev #s and now sits at version 2.3.8 as of 2 weeks ago. Since it appears the FreeBSD port is well on its way to becoming abandoned, I decided to strike out on my own and run the current version from sources. I had a spare jail already set up - this jail was going to be a standalone KDC, but since I sucked that role into my LDAP jail I didn't have a purpose for this one.

So... off I went. I followed the instructions on the phpLDAPadmin github, installing php84, composer, npm, etc and attempted to get the source to compile. Maybe it did? I don't really know... I always saw a slew of warnings flash by during the process, but it completed. I guess that could be ok... the real problem is that my web server admin chops are LONG GONE. I struggled and failed to get php working properly in either nginx or apache (tried one and then the other in this jail). Some of it was undoubtedly not having my environment variables for phpldapadmin set properly, so the php side was erroring out trying to talk to my openldap server. Regardless, I came to the realization that I just didn't care enough about the FreeBSD purity of my homelab setup to keep going down the roll-your-own route for phpldapadmin. The truth is that I'll use it almost never - once user accounts are in place (I only have 3 to go), the only times I'll ever need it is when adding a new host - I don't expect that to happen often. I had a spare rpi3B in a drawer, so I cheated and went the Docker on raspbian route.

This was my first experience with Docker, and there's some things I like and things I hate already. Overall I'd say I prefer vnet jails hands down for containerization. However, getting phpldapadmin working was easy (someone did the work for me), and after some struggles I managed to get an nginx reverse proxy to sit in front so that the web site is secured by SSL. That's important because username and password are being sent through the login web form, and I don't want those clear text on the wire. phpldapadmin on its own only provides http. The biggest problem I had was getting the backend LDAPS communication working - in order for that to work properly the phpldapadmin container needed to trust the cert being offered up by my openLDAP jail, and it wasn't as easy as I'd thought it should be to get my root CA certificates into the phpldapadmin container. At any rate, all is working now, so I can get back to working on my core setup.

Call this my first real fail, in that I didn't manage to do what I want with FreeBSD. The only consolation is that now I know a bit about Docker, and it's reinforced my conviction that FreeBSD is the saner path for my core services deployments. Also, this will almost never get used, so I feel ok about cheating a bit. Slippery slope, you say? Not really... my experience with Docker so far hasn't exactly impressed me, and I doubt I'll dabble again unless I encounter a similar ultra-low-use, throwaway but nice-to-have thing I want.
 
This comment is so the OP knows that this person that happens to be me finds this thread very original and interesting, that I think that the OP writes very well, and that I'm following the OP's future installments even though I might not understand every technical detail.
 
Long pause, but I'm getting back into my setup. I had to divert my attention to dealing with some home automation issues... I'd been using a "beta" all-in-one appliance that integrated Zigbee2MQTT, Zwave-JS UI, Mosquitto, Node-Red, and some other quality-of-life packages on a little SBC running podman on Manjaro. It was a nice idea, but it became abandonware when the developers all peeled off before finishing. Promises that an update and/or instructions would be provided to decouple the platform from the update server weren't kept, and the inevitable happened. One of the packages (Zwave-JS UI) partially failed. It still spoke to devices in the mesh and communicated to MQTT, but the GUI wouldn't work and that became a deal breaker. So... I doubled down on my docker adventure and built a new HA device on an rpi with my 4 containers (z2m, zwave, MQTT, Node-Red) running on it. The painful part was moving over all my Zigbee and Zwave devices... took forever. I could have done it faster if I hadn't gotten sucked in to re-watching Breaking Bad and Better Call Saul... I'm not great with priorities.

At any rate, the home automation problem is fixed (abandonware box has been relegated to my e-waste pile), so I started thinking about my homelab setup again. During the time I was working on the HA upgrade I also came to the realization that forcing my laptop to run FreeBSD was the wrong choice. Matlab performance the way I'd kludged it together was just not acceptable, and this machine has to be able to run that software. So, I went for CachyOS (Arch-based distro) because it has root-on-ZFS built in to the installer. While I'm willing to roll up my sleeves and work for it with FreeBSD, when it comes to a Linux client my attitude is that I shouldn't have to pay as much attention or focus on becoming expert in every aspect of the system. There are enough good options out there that I should be able to just pick one that offers what I want without needing to jump through hoops and work hard to maintain a custom setup. What do I want? Decent compute and graphics performance for Matlab, root-on-ZFS so I can easily automate system backups with ZFS send/receive (and also protect me from myself using snapshots), and out-of-the-box desktop that doesn't need me to make endless tweaks to get a usable experience. CachyOS was the closest I could find to fitting those requirements... the only thing that bugs me is their installer is extremely limited in how datasets are set up. Good enough for now

The "fun" came when I tried to integrate the CachyOS client with my LDAP/Kerberos setup. I think I stepped on every rake and fell face first into every cow pie... it did not go well. I blame systemd, for no reason in particular... actually I have a reason. Take a look at nsswitch.conf as it ships with CachyOS and tell me you're not confused. It's way more complex than it needs to be. Deciding to simplify is a great way to make sure you can't log in to the system at all... I had to boot into a live USB many times to fix one blunder after another. In the end, getting my CachyOS client to work was simpler than I thought - the key was to make minimal changes (duh).

This being a FreeBSD forum, I won't bother with a detailed recreation of the setup for a CachyOS/Arch LDAP/KRB5 client. Highlights are:
  • Install openldap, sssd, and krb5 packages
  • Install CA certificate(s) and update the system cache
  • Use https://forums.freebsd.org/threads/...ws-to-freebsd-in-my-homelab.98860/post-734041 as a starting point.
    • ldap.conf, sssd.conf, and krb5.conf are identical as the previous post. Note I did change to create_homedir = True in sssd.conf for this client.
    • for nssswitch.conf, just add "sss" (no quotes) at the end of the entries for passwd, group, shadow, services, and netgroup. Also add a new line with sudoers: files sss
  • PAM only requires edits in /etc/pam.d/system-auth. This gets included in everything else so by making the modifications here you won't have to touch any other PAM config files.
    • I've attached the file (with a txt extension so it'd attach) - ymmv. Use at your own risk.
  • Enable sssd
Going back to my earliest posts, it was always my intention to have one or more Linux clients, so I'm glad I have this worked out. Next up: this weekend I'm going to force myself to finish getting the secondary LDAP/Kerberos jail set up so that I can move on and restore my network shares and files before finally moving my wife's primary PC over to the new setup.
 

Attachments

So this is another one of those "oops" posts... it turns out I wasn't quite ready yet to implement a backup LDAP/Kerberos jail. While putting together my plan for implementing the secondary jail I stumbled across a couple of things that had slipped through the cracks that needed tending to.

First up, I forgot to create a group to go along with my firstuser LDAP account. When running any version of # ls with flags that show ownership, I never looked hard enough to notice that the results were returning without any GID to name translation, e.g. -rw-r--r-- 1 firstuser 10002 208 Mar 16 20:10 somefile.
  • The fix for this was obvious enough - I already created a domainGroup type when setting up the LDAP schema, and I also created ou=groups,dc=mydomain,dc=com to hold all my group info... the stage was set, I just failed to execute.
  • I used my handy dandy phpLDAPAdmin instance to create a new firstuser group with GID 10002 using the domainGroup template, and voila! GID to name translation works, and my previous example now returns e.g. -rw-r--r-- 1 firstuser firstuser 208 Mar 16 20:10 somefile.
  • For good measure, I added dn=uid=firstuser,ou=accounts,dc=mydomain,dc=com as a member of the new firstuser group. I'm not sure if I'll ever have any software that reads membership from my directory, but if I do... best to be prepared.
The other item I forgot was to set up entries for my hosts and devices. ou=hosts,dc=mydomain,dc=com was created during LDAP install, but never populated. The core schema has a "device" objectClass that looks pretty good for things like printers and scanners, but for hosts it's not exactly what I want. So, I took a page out of surfrock66's book and rolled my own domainHost objectClass. Some attributes I want simply don't exist at all in the standard public schema, so I created those, too. I've attached the .ldif that extends the schema with my new domainHost type - this is very much personal preference and if you're following along I'd suggest you look carefully at it and make appropriate modifications for your needs. In the attached file, I replaced my OID with XXXXX to anonymize it. Replace with your own OID before attempting to use this... I'm not sure what happens if you have letters instead of numbers, but I'm guessing it'll fail.
  • As before, schema additions are made using ldapadd, but I ran into an issue.
    • Once LDAP/Kerberos/SASL was set up and working, we had: mech_list: GSSAPI PLAINas the first line of /usr/local/lib/sasl2/slapd.conf. This was too restrictive, in that it disallows the system root user to LDAP admin translation we took advantage of during setup.
      • The fix is to append EXTERNAL to the end of this line... once done, ldapadd worked as expected
  • # ldapadd -y EXTERNAL -H ldapi:/// -f domainHost.ldif
With the domainHost template installed, I used phpLDAPAdmin to add a host entry for my laptop: cn=laptop1,ou=hosts,dc=mydomain,dc=com. I then deleted the existing host/laptop1 Kerberos pricipal and created a new one:
# kadmin.local​
> addprinc -x dn=cn=laptop1,ou=hosts,dc=mydomain,dc=com -randkey host/laptop1.mydomain.com (the -x flag tells kadmin to attach the new principal to the specified LDAP entry, instead of just placing it in the default kerberos subtree of dc=mydomain,dc=com)​
> ktadd -k /root/laptop1.keytab host/laptop1.mydomain.com​
> exit​
This is nit-picking, but I wanted to have Kerberos data stored alongside the LDAP host entries, because OCD. There's nothing I can do about the primary LDAP/Kerberos jail - that key is already in place and I don't know enough to risk breaking everything and having to start from scratch. It'll just stay where it is, untethered to an LDAP domainHost entry. So, now I just copy /root/laptop1.keytab over to my laptop, renaming it /etc/krb5.keytab. Lastly, I make sure Kerberos is up to speed on the laptop by running (as root) # kinit -k -t /etc/krb5.keytab host/laptop1.mydomain.com.

My last little test was to create an LDAP account for my wife, Kerberize it by adding the matching principal, and then log into my laptop with the new network username and password. For a change, it actually worked on the first try! So I have hosts and groups working, I have fully half of the user accounts I'll ever create in place, and I think I'm good... NOW I'll move on and implement my plan for a backup LDAP/KDC jail.
 

Attachments

Service deployment update: OpenLDAP + Kerberos backup services are running​

WOW. That was way harder than I thought it'd be​

After muddling my way through the installation and configuration of my primary LDAP + KDC setup, I'll admit it, I got cocky. I thought getting the read-only LDAP + KDC backup running would be easy. Part of this was that there's a really nice Ubuntu writeup on how to set up OpenLDAP replication that makes it look straightforward. The other part? Hubris... I'm just as egotistical as anyone else. I really thought I had it. Slam dunk, just an afternoon of light work...

I chose to use a simple publisher/subscriber model, so my backup LDAP jail is read-only. One side benefit of using LDAP as the back end storage for Kerberos is that once LDAP replication is working, I get Kerberos replication for free. So, here's how I managed to pull this off, merging the Ubuntu replication doc with my own prior posts to this forum.
On the primary jail, make the following modifications to allow LDAP replication:
  1. If you haven't stumbled across this error and fixed it yet, make sure the first line of /usr/local/lib/sasl2/slapd.conf reads mech_list: GSSAPI PLAIN EXTERNAL. See one of my previous posts (somewhere) where I discovered that not having the EXTERNAL keyword was causing issues. Although, if I'm too lazy to search for the reference, I guess it's ok if you don't bother. Just add EXTERNAL and be happy. Edit: actually it's the post right before this.
  2. Create a hashed password for the replication user account using: # slappasswd -o module-load=pw-sha2 -h '{SSHA512}' -s 'your-password-here'
  3. Create a replication user - I chose to use an ldif here, but I could have used phpLDAPadmin... add replicator.ldif using # ldapadd -x -D 'cn=admin,dc=mydomain,dc=com' -W -H ldapi:/// -f replicator.ldif
    Contents of replicator.ldif:
    Code: 
    dn: uid=replicator,ou=accounts,dc=mydomain,dc=com
objectClass: domainAccount
objectClass: simpleSecurityObject
cn: replicator
name: replicator
sn: replicator
uid: replicator
userPassword: {SSHA512}++Redacted==
description: Replication user
  4. The replication user needs to have read access to the base DN and all subtrees. I initially followed the Ubuntu doc and just inserted a new read-only rule + unlimited olcLimits modification for the replicator user, but my execution was poor. Supposedly rule insertion should bump all rules down 1 slot automatically, but I screwed it up somehow and ended up with the replicator account having no access. My solution was to re-write the ACLs completely, which resulted in acl_update.ldif. Apply using
    # ldapmodify -Q -Y EXTERNAL -H ldapi:/// -f acl_update.ldif
    Contents of acl_update.ldif:
    Code: 
    dn: olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcAccess
olcAccess: {0}to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by dn="cn=admin,dc=mydomain,dc=com" manage by dn="uid=firstuser,ou=accounts,dc=mydomain,dc=com" manage by dn="uid=replicator,ou=accounts,dc=mydomain,dc=com" read by dn="uid=ldapbinduser,ou=accounts,dc=mydomain,dc=com" read by dn="uid=kdc-service,ou=accounts,dc=mydomain,dc=com" read by dn="uid=kadmin-service,ou=accounts,dc=mydomain,dc=com" write by * break
olcAccess: {1}to dn.children="ou=accounts,dc=mydomain,dc=com" attrs=userPassword,shadowExpire,shadowInactive,shadowLastChange,shadowMax,shadowMin,shadowWarning by self write by anonymous auth
olcAccess: {2}to dn.subtree="dc=mydomain,dc=com" by self read
olcAccess: {3}to attrs=krbPrincipalKey by anonymous auth by dn.exact="cn=kdc-service,ou=accounts,dc=mydomain,dc=com" read by dn.exact="cn=kadmin-service,ou=accounts,dc=mydomain,dc=com" write by self write by * none
olcAccess: {4}to dn.subtree="cn=krbContainer,dc=mydomain,dc=com" by dn.exact="cn=kdc-service,ou=accounts,dc=mydomain,dc=com" read by dn.exact="cn=kadmin-service,ou=accounts,dc=mydomain,dc=com" write by * none
  5. Now we need to set up the primary LDAP jail to act as a simple sync provider. We just need the syncprov overlay to make this work. Create provider_simple_sync.lidf and apply using
    # ldapmodify -Q -Y EXTERNAL -H ldapi:/// -f provider_simple_sync.ldif
    Contents of provider_simple_sync.ldif:
    Code: 
    # Add indexes to the frontend db
dn: olcDatabase={1}mdb,cn=config
changetype: modify
add: olcDbIndex
olcDbIndex: entryCSN eq
-
add: olcDbIndex
olcDbIndex: entryUUID eq

# Load the syncprov module
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: syncprov

# syncrepl Provider for primary db
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: syncprov
olcSpCheckpoint: 100 10
olcSpSessionLog: 100
  6. Deviating from the Ubuntu prescription... it turns out that the memberOf overlay that's part of my custom schema wreaks havoc on the replication process. I could watch replication start, but as soon as objects were sent they'd get deleted from the secondary. This took forever to track down, but there is a solution: we need to set olcMemberOfDangling: ignore on the master LDAP instance. Create fix_memberOf.ldif and apply with
    # ldapmodify -Q -Y EXTERNAL -H ldapi:/// -f fix_memberOf.ldif
    Contents of fix_memberOf.ldif:
    Code: 
    dn: olcOverlay={0}memberof,olcDatabase={1}mdb,cn=config
changetype: modify
replace: olcMemberOfDangling
olcMemberOfDangling: ignore

And that's all for the primary LDAP jail - it should be set to replicate once a subscriber authenticates.

Secondary jail setup:​

  1. Follow the prep steps in this post to get the jail ready for use:
    1. create and sign a server certificate and make sure the jail trusts the root CAs
    2. run the prep steps and steps 0-6 from this post to get the basic schema set up
    3. run a modified step #7 from this post, creating only the top level dc=mydomain,dc=com DN. (delete the accounts, groups, and hosts OU creation). Replication won't work if the top-level DN for the directory isn't in place... the Ubuntu docs don't say one way or the other, so I tried without this first. No go... you need to create the O.
    4. In place of step 12, use the final form of the ACLs from step #5 above.. just copy over acl_update.ldif and apply it.
    5. run steps 13,14,16,17,22 from this post, making sure to change the olcSaslHost in step 17 to use the FQDN of the backup LDAP jail.
    6. Add the syncprov overlay to make this jail an LDAP consumer using
      # ldapmodify -Q -Y EXTERNAL -H ldapi:/// -f consumer_simple_repl.ldif
      Contents of consumer_simple_repl.ldif:
      Code: 
      dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: syncprov

dn: olcDatabase={1}mdb,cn=config
changetype: modify
add: olcDbIndex
olcDbIndex: entryUUID eq
-
add: olcSyncrepl
olcSyncrepl: rid=0
  provider=ldap://ldapjail.mydomain.com
  bindmethod=simple
  binddn=uid=replicator,ou=accounts,dc=mydomain,dc=com credentials=<actual password here (not the hash)>
  searchbase=dc=mydomain,dc=com
  schemachecking=on
  type=refreshAndPersist retry=”60 10 300 +”
  starttls=critical tls_reqcert=demand
-
add: olcUpdateRef
olcUpdateRef: ldap://ldapjail.mydomain.com
    7. I forgot this step when originally posting: also apply the fix_memberOf.ldif fix mentioned in step 6 for the primary LDAP section to make sure the schemas are identical. I suspect this step isn't necessary, but I did it anyway out of an abundance of caution.
Two notes/changes from the prescription for the secondary LDAP setup: Ubuntu docs had quotes around the binddn and searchbase - this doesn't work. Get rid of the quotes. Also, the Ubuntu docs had retry="60 +", which doesn't work. Being more explicit and saying "try every 60 seconds 10 times, then try every 300 seconds for infinity" seems to work.
That should get LDAP replication up and running. You can test by querying the contextCSN on both the primary and secondary LDAP jails: # ldapsearch -Y EXTERNAL -H ldapi:/// -b "dc=mydomain,dc=com" -s base contextCSN. Once both numbers are the same you know that replication has completed and you have a full copy on your secondary LDAP jail. Try querying the secondary from a client machine - it should give you good results, now.
The final task is to set up another KDC - we won't enable kadmind... this will be a read-only backup KDC. Not much has to happen, since its data already came over via LDAP replication. To get this going:
  1. On the secondary jail, run the prep steps and steps 1-4,6,7,9 from the first spoiler of this post, with modifications:
    1. in /etc/krb5.conf, swap the order of kdc = ldapjail.mydomain.com and kdc = ldapjail2.mydomain.com in the MYDOMAIN.COM section of the [realms] heading. ldapjail2 should be first so that the local Kerberos queries the local LDAP.
    2. as before, make sure you use mech_list: GSSAPI PLAIN EXTERNAL as the first line of /usr/local/lib/sasl2/slapd.conf
    3. in /etc/rc.conf do not enable kadmind. Instead, set kadmind_enable="NO"
  2. Copy the following files from your primary jail to the same name/location in your secondary jail, and make sure ownership and permissions match:
    • /usr/local/var/krb5kdc/service.keyfile
    • /usr/local/var/krb5kdc/stash
    • /usr/local/var/krb5kdc/.k5.MYDOMAIN.COM
  3. Head over to the primary jail and create host/ and ldap/ principals, then export those to a keytab:
    # kadmin.local
    > addprinc -randkey host/ldapjail2.mydomain.com
    > addprinc -randkey ldap/ldapjail2.mydomain.com
    > ktadd -k /root/ldapjail2.keytab host/ldapjail2.mydomain.com ldap/ldapjail2.mydomain.com
    > exit
  4. Copy /root/ldapjail2.keytab over to the secondary jail, renaming it /etc/krb5.keytab
  5. Set permissions of /etc/krb5.keytab to 660 with ownership root:ldap.
  6. Start the KDC and SASL and restart LDAP for good measure:
    # service slapd restart
    # service kdc start
    # service saslauthd start
  7. Run kinit to validate the keytab:
    # kinit -k -t /etc/krb5.keytab host/ldapjail2.mydomain.com
That should do it! Last step is client config... that's pretty easy:
  1. make sure you have 2 entries in krb5.conf starting with kdc = in the [realms] MYDOMAIN = { section... one for ldapjail.mydomain.com and one for ldapjail2.mydomain.com.
  2. in sssd.conf update the ldap_uri and krb5_server lines to include the secondary. These are comma separated entries, so just append ,ldapjail2.mydomain.com.
  3. OS dependent - restart sssd and clear the SSS cache.
I tested the secondary setup the easy way - I went and disabled slapd and kdc on the primary jail and then tried using my client machine to get a Kerberos ticket and run some directory queries. Both worked flawlessly.

FINALLY, I can put a bow on this and move on.

Next I'll be heading into the subjective guesswork of trying to figure out in advance how I should set up my small block cache limit to best make use of the special vdev that I have attached to my big pool slated for general user storage. I have a 2nd client working (another CachyOS machine - I'm enjoying it)... it looks like I'm actually within shouting distance of pulling the plug on my Microsoft VMs completely. Let's see how long it takes to get my storage sorted out and NFS up and running...
 
You must log in or register to reply here.
Back
Top