But not all things you might do with a dotfile (or, more generally, per-user customization) are just replacing files. Things like cronjobs, brew installs, `defaults` in MacOS, etc. Viewing dotfile-based customization as strictly files to obliterate with pre-existing files is needlessly myopic.
For this broader problem, there are other more complete solutions that are more robust and flexible. Personally I like dotbot (https://github.com/anishathalye/dotbot) as a balance between power and simplicity, particularly when managing files across multiple OS homedirs (e.g. linux server, macos laptop).
That's provisioning, not dotfiles management. My dotfiles only includes config files. I'd just use the package manager to install packages and I'd just use the relevant program to enable stuff. As I use stow, I just create different configurations for different OS if they differ too much. At most, a handful of scripts to customize my user account.
Dotfiles are just a component, but not the whole story, of your personal compute environment. Your environment also includes things like:
* ~/bin scripts (etc)
* programming language stuff - e.g. go, rust, python, ruby etc have tooling for per-user package management, language version, etc.
* various forms of password/key/auth stuff like ssh allow lists, encrypted password stores, etc.
And the biggest one: Type of machine - work, daily driver, server, etc
The type of machine may require different dotfiles or different parts of dotfiles (e.g. what basrc includes from `. .config/bash/my_local_funcs`), and having some scripting around this makes life easier.
Similarly OS packages are great, and I use them heavily, but work and personal servers and personal desktop all use a different OS, so its useful to have provision scripts for the type of machine, and i keep all that together with my dotfiles (etc) in my "personal environment repo" (it's name is dots, and when i talk about dotfiles I really mean "personal environment". I suspect other share this view, which leads to this "pure dotfiles" vs "dotfiles+parts of provisioning" viewpoint difference even though they largely have the same set of problems and tooling.
The majority of my computing happens at my workstation (desktop). That is what I consider my personal environment, and I would script its setup, but I can't find the motivation to do so (and I like to do ad-hoc changes). Permanent configuration (related to my usage, not the computer. My core utilities, I can say) get added to my dotfiles. As for server and works, their intersection and my personal stuff are minimal (mostly bash, vim, emacs?) I'd rather have a different system/project to manage them.
This is why I use Nix + home-manager to manage my CLI, programming environment, and system configuration across Linux, macOS and WSL using one GitHub repo. It also handles differences across machine types well.
A dot file management system is only part of the picture.
To spin up a new machine is a 30 minute job, and then it feels like “home”.
I imagine that things like provisioning are essential to people that switch computers often. So it's not a dotfile-specific problem, but more of a dotfile-adjacent problem.
There's so many interesting edge-cases that affect UX even when distro-hopping between Debian-based distros... especially if you used it for several years and had plenty of custom scripts in your ~/.local/bin folder.
I may yet need to learn or (re)discover some best practices of how to get up to a working development environment faster. I'm thinking of using Guix for that... but I digress.
So far, my workflow goes like this (on a newly-installed distro):
1. Configure environment variables that affect package-specific file locations (/etc/security/pam_env.conf and a custom /etc/profile.d/xdg_std_home.sh script that creates and assigns correct permissions for required directories).
2. Provision packages
3. Deploy config files (using stow).
What I've yet to figure out (haven't really researched it yet), how do you handle app-specific configs (think Firefox add-ons, add-on configs, Thunderbird accounts, etc.)?
"Switch computers often" can also apply to "switch computers with little notice". Even if 95% of my time is spent on one computer, it's nice to know my config is safely squirreled away and, uh, trivially unsquirrelable if something terrible happens to this hardware and I have to get another computer. Seems like a relatively low probability event, but my child has already destroyed two ThinkPads (both were very old and very disposable--still an accomplishment).
As to your last question, nix+home manager gets you there, but that's a whole other Thing.
(n)vim for example: my dotfiles don't vendor the handful of plugins i use, they just include the directives to install those with plugin manager.
I generally use a makefile + stow to handle my dotfiles and home-dir setup. Each program has an entry in this Makefile - most of them are very simple, I keep a list of programs who's dots need to be in ~, and another for ~/.config/ and using make's variable expansion they just get a stow target.
This also allows me to not just copy preference, but provision a bunch of stuff that's invariant across machines (e.g. what i have installed via rustup, go install, etc).
For this broader problem, there are other more complete solutions that are more robust and flexible. Personally I like dotbot (https://github.com/anishathalye/dotbot) as a balance between power and simplicity, particularly when managing files across multiple OS homedirs (e.g. linux server, macos laptop).