Centralized management of dotfiles

Hello,

I am relatively new to Unix administration, and I cannot find any software for version-controlling and backing up dotfiles that are scattered throughout the file system tree and across several local machines. I've seen software like Restic, BackupPC, Etckeeper, and rcs, but I'm not sure I understand how they work, or they don't meet my criteria for : centralization + version control + backup, without duplicating code through copies like I might do with a Fossil repo. Although, once centralized, backups are easy.

I also considered using shell scripts or a database, but that seems overly complicated for such a small gain. In large infrastructures, this isn't done haphazardly, so how do you manage it?
 
CVS or git over ssh, with an install script in that directory.

You dotfiles should not be scattered to many places. What files are you thinking of that are not in your homedir?
 
Emacs




you can use tramp to tangle the files to a remote machine as well
 
poor mans answer.
I make a directory /home/x/Conf
Then i take the .conf files which are very relevant and put them in Conf and put a softlink.

But .dotfiles are problematic. Eg do rm -vfR /home/me/.cache , and your fontcache is gone.
 
the advantage of emacs is you have all you dotfiles in one file
which you then tangle ( write out ) into place

rather than opening files one by one and editing them
much faster
 
These days I don't bother worrying too much about it. But, I used to just have directories for it in a folder mounted under / to hold the originals and simlink those to the relevant places. That directory itself is easy enough to use your favorite revision control system to handle. Technically, revision control and not source control is what you want, but I don't know that the difference is really that relevant to what's likely to be a small collection of files.

Where I personally have the most problems is remembering to use the version tracking. If I had the interest, I'd probably just write a script to do it the way that there's a script for things like dealing with crontab and the adding/editing users. One script to get the file, open it in your favorite editor and update ask to install/update it in the repo.
 
not really a dot file user here, but lately I have been trying to use mise dot track file dont like much that each time you track something new there is a new commit even if you forgot to undo something, so you get extra "steps" in the history... hehehe... also not many things build for freebsd so mise kind of fails installing sometimes developer tools and things like that...
 
First off, dot files shouldn't change that frequently so version control seems like overkill. However you probly DO want to back up your .mozilla and .thunderbird, etc directories. In my backups the apropos files and directories get renamed as "dot-whatever-YYMMDD"
 
My dotfiles change constantly. Don't forget that one tree is shared across all Unix like OSes.

Backing up my browser profiles is something I don't do, my settings are in google and mozilla.org
 
I use Stow (mentioned above by froggit9000) not just for many of my dotfiles, but for all sorts of configuration too.

I'm very happy with my system; it has made my configuration management much simpler and less error-prone than it had been before I came up with it. Whenever I've described it here before, a lot of people have thought I'm crazy, and that my system is total overkill. For my part, I agree with the "I'm crazy", but it's perfectly OK with me if someone thinks it's "overkill" (I'm like, OK, that's cool, so just don't use it yourself, dude).

If you're interested, here's a thread that I made a year or two ago that was not directly about my system, but somehow at least partially devolved into people telling me I'm crazy and my system is overkill. I put some detail about the system (including the Stow-related parts) in there as the thread progressed. If you wind up with any questions about it -- preferably but not necessarily beyond "Are you crazy?" :p -- please feel free to ask.

Oh, and yeah, regarding other things mentioned in your question: I keep all of it (and far more stuff) in source control using Subversion, and back everything up in a variety of ways (including using Restic).
 
I store my personal dotfiles in ~/dotfiles under version control (devel/git], and I install them with a simple Makefile.

I also store system-wide configuration files (well, they are not dotfiles, but the idea is almost the same) in ~/sysfiles and use the same kind of a file structure that is used in my dotfiles repository I linked above, and install them with Makefile as well. E.g. there I will have etc/rc.conf, usr/local/etc/tmux.conf, and so on, and I can do make etc to install everything under etc into the same sub-location in /etc, and make /usr/local to do the same for usr/local, or just make to install everything I have. I especially like this because this way I can edit system files without a need to escalate privileges to root, and then just once execute make(1) command as root to install them.
 
First off, dot files shouldn't change that frequently so version control seems like overkill. However you probly DO want to back up your .mozilla and .thunderbird, etc directories. In my backups the apropos files and directories get renamed as "dot-whatever-YYMMDD"
That will depend, the main time that it's super worth it to use revision control is for when you're working out configuration. Personally though, the not dotfiles in places like /etc are more likely to be an issue.
 
Version control and the ability to go back to previous versions is essential for multi-machine dotfile keeping. It is way too easy to do something for a new emacs that breaks when logging in to a machine with the old emacs.
 
At least in terms of my shell I've settled on having a .bash_profile and a .bashrc that is the same across all systems and then source a .bash_profile_local and a .bashrc_local. There's surprising little that has to be the in the _local files. Other dot files I generally don't worry about.
 
I keep personal dot-files in Git and install these with make(1).

As of system-wide configuration files, these are also in Git but managed with Ansible.
 
I'm lost in a dark and cold dotfiles forest, I can't find my way, oh wait there is a path ... I'm following it ... I'm already at the end point, there is a big stone down the ground, I kick off the green moss on it, there is a text carved on : "If you don't find what you are looking for, then [broken text]" 🪨😢

As of system-wide configuration files, these are also in Git but managed with Ansible.
Ansible, that interests me, how do you do it?

Thanks for the other suggestions, but I can't find a comprehensive solution; I still feel like everything is separate: user/system, versioning/backup, and there's no GUI to see everything at a glance.
(Please avoid Emacs, I do not wish to use it)
 
I'm lost in a dark and cold dotfiles forest, I can't find my way, oh wait there is a path ... I'm following it ... I'm already at the end point, there is a big stone down the ground, I kick off the green moss on it, there is a text carved on : "If you don't find what you are looking for, then [broken text]" 🪨😢


Ansible, that interests me, how do you do it?

Thanks for the other suggestions, but I can't find a comprehensive solution; I still feel like everything is separate: user/system, versioning/backup, and there's no GUI to see everything at a glance.
(Please avoid Emacs, I do not wish to use it)
Keep it simple, unless you've got a specific need, you don't really require a special tool, just create a directory to hold the files you're editing and make a hard link from the file to put it into that directory and back that folder up. And just create a new copy everytime you edit it.

Or you can put those on a different filesystem and use soft links, just make sure to account for when filesystems come online.

It's not the "best" way, but it does work and if you start to find that that's failing in some way, the failure should help identify what you need in a solution. There's a fair number of solutions, and they're not all going to be right for one particular environment.
 
Hello,

I am relatively new to Unix administration, and I cannot find any software for version-controlling and backing up dotfiles that are scattered throughout the file system tree and across several local machines. I've seen software like Restic, BackupPC, Etckeeper, and rcs, but I'm not sure I understand how they work, or they don't meet my criteria for : centralization + version control + backup, without duplicating code through copies like I might do with a Fossil repo. Although, once centralized, backups are easy.

I also considered using shell scripts or a database, but that seems overly complicated for such a small gain. In large infrastructures, this isn't done haphazardly, so how do you manage it?

There are a lot of tools available to manage dot files, /etc files and other configuration files. Old school uses rcs as a VCS and rdist to distribute them to your network. A slightly more more modern approach might be to use a shell script with scp or rsync. This is still pretty hoaky but it can do the job.

More modern approaches include using git as a VCS on your management workstation, optionally pushing your config files to a local forgejo repo. But never to github on the internet. That would map out your configuration and network for others to see and exploit.

To push out your configurations you could use cfengine, salt, puppet, chef, ansible, or some other tool. I currently use ansible, while git maintains the master repo, storing a copy on my forgejo server (no one has access to) on my network. In addition to maintaining configuration files it also handles reboots, lists of packages on each machine (so I can easily rebuild each machine). You can use chef, puppet or salt instead.

At $JOB we maintain our 1400 Linux server configuration in git but instead of forgejo they host the git repo on gitub enterprise, a github we have purchased for use within the company. We use ansible through Ansible Automation Platform (AAP), a GUI to run our ansible jobs.

But for home use or for use in a small business, using ansible from the command line works just as well.

git, ansible, puppet, salt, chef, and cfengine are FreeBSD ports and packages. We don't have an AWX port -- AWX is the open source AAP. I haven't found it worth my while to port it to FreeBSD.
 
Back
Top