Long story short, I did 180° on my opinion presented in my earlier post:
I’ve tried two times to leave Arch Linux for some systemd-less distro and failed both times. I’m always forgetting how convenient is a current state of our little free software ecosystem.
I have very limited free time and prefer spending it on my hobbies instead of maintaining my operating system.
So, to the topic!
I had my first round of distro hopping some time last year. Prompted by my curiosity and opinion on systemd I wanted to check some systemd-less distros. But at that time I only had one machine which I need for work and casual everyday tasks related to life and freelancing. Like: paying utility fees, doing taxes, online shopping, and so on. But then I received new laptop from my current employer and this machine was used only once during my travel, and was spending its time collecting dust. So I decided to check some interesting distros on this laptop. Based on some arbitrarily decided criteria, I choose: Alpine Linux, Void Linux and KISS Linux for my test.
This distro was the most problematic on my laptop due to it being first non-Arch disto I tried to install on this machine, so I discovered some hardware quirks this time. My laptop requires SOF firmware, have a NVME disk connected to Intel VMD and have a somewhat uncommon touchpad configuration. So there was a very little problem with Alpine Linux which was booting into emergency shell because it wasn’t able to detect the root drive, but I added the missing module to initramfs and it booted. I installed my typical suite of apps on it. I also noticed that it recommends using eudev and elogind on desktops, both of those software are forks of systemd’s parts, so this setup still somewhat depend on systemd’s codebase. But then I noticed that my touchpad didn’t work and it requires compiling custom kernel and passed on Alpine Linux.
Void Linux worked on my laptop without a custom kernel with very little tweaking. But I noticed some problems with it at that time. I was using some new C++ features in my day-job-projects and it required much newer C++ compilers than were available at that time. I also didn’t like xbps-src much. Idea of merging user’s custom packages into main port repository to build them didn’t work for me. Both Arch and Alpine allowed me to have my own repo with PKGBUILDs and APKBUILDs and build them like I wanted. So, I played with it a little and moved to next test.
That was the distro I totally loved, I loved total control over my machine, loved simplicity. But had some problems in patching and maintaining my packages. At that time I didn’t noticed that I’m using this computer only to maintain itself because it was pretty interesting hobby.
I also made a decision to maintain some packages from community repo, but my involvement was short and I’m still sorry for that…
My story with KISS Linux was brutally interrupted by complexly unexpected event: I needed my work laptop for work! I got this laptop from my employer for travels and I needed it quickly.
I was in something like panic and decided to quickly throw some fully featured and boring distro on my laptop and my first choice was Debian. And immediately was reminded how convenient computers can be when I used NetworkManager to connect to different WiFis during my travel and so on, everything was boring and everything worked well.
Weird story. I choose three distros to test, but was most impressed with forth one chosen by incident. I’m totally impressed by Debian. I can update it and trust none of my software will break, I can throw any shit I need at it and trust that it will work. I wasn’t spending all time on maintaining my OS but actually doing productive work on my machine instead. Soon I moved all data and software required for work to my laptop and it still have the same Debian albeit updated to new release to this day.
So roles was reverted. My main PC was spare machine and my work laptop become boring work laptop with Debian on it. The problem was that my PC had NVIDIA card and it pretty much blocked me from experimenting with more uncommon distros. But I decided to bite the bullet and bought used Radeon RX 580 for my PC. And decided to one last test before I’ll throw a towel and learn to like systemd.
I choose Alpine Linux for my final test, not KISS Linux. Why? I took a step back and noticed that KISS Linux was complete rabbit hole for me and made me maniacally debloating my system from everything I even remotely consider bloat. And everything is bloat! Until time comes and you need this feature… So since this time I learned to accept so called “bloat” if trade-offs are reasonable. Also I don’t need a machine used only to maintain itself, if so, not having it will actually give me more free time!
So, I wrote aforementioned post about systemd to reaffirm why I’m doing it and installed Alpine Linux on my PC.
Firstly everything seem working. I decided to refrain from using elogind but keep using eudev for a while. I solved some initial problems with lack of elogind, and for some time I was happy using Alpine Linux.
The very fist problem I was reminded of is that normal cron daemon skips jobs when computer is off during specified time and debugging problems with cron isn’t the simplest thing to do. And I have SSD and want to run fstrim once per week or month. So there is need for either anacron or dcron which handles such cases. And the BusyBox’ cron is fork of dcron, so problem solved? Nope. It seems that it is fork of some earlier version of dcron. How did I find about it? I studied BusyBox’ code after spending few hours trying to figure why I’m doing everything as specified in dcron documentation and my cron job isn’t working. The systemd timers are super intuitive straightforward to manage compared to traditional cron, anacron or dcron in comparison…
After first, initial problem I started noticing more, I noticed some weird crashes, probably related to muslc. And than I tried to use my encrypted USB drive and noticed that eudev is not 100% perfect replacement for systemd-udev.
I’m sometimes storing important documents on my USB drives, I’m also storing my GPG, SSH keys, Gemini certificates, pass’ git password store, recovery codes for 2FA and other important things on encrypted USB drive in my parent’s home. And small USB drives shares common weakness: It’s easy to lost them. So I’m firm proponent of encrypting USB drives by default! Seriously, encrypt all your USB drives and don’t encrypt only ones used for sharing data with other people and put mundane physical labels on all of them. Very often documents stored on USB contains important personal info which can be used to gain control over banking account or something even worse, losing unencrypted USB drive is serious privacy risk!
I also often taking photos with my smartphone when we are on walks with our son and dog, and noticed that mounting Android phone via MTP also can be quirky on eudev.
So After some pondering I decided if eudev is not 100% perfect, I’ll go full systemd-less route and use mdev. Simple and bare system should be better than much more complicated and subtly broken one. Right? And then problems started.
Many GUI apps indirectly requires udev to work: sway requires libinput which requires libudev, I installed libudev-zero¹ for sway to work. Thunar need GVfs to mount USB drives and even for trash support. I temporarily given up convenience of quickly mounting USB drives, but trash saved my ass way too many times… I cannot give up trash support, but GVfs without mount support doesn’t require udev. So I sacrificed some convenience to give up systemd completely. But than I needed to print something and noticed that while CUPS can be made to work it is… quirky? And plugging USB drives sometimes only show drive without partitions which I can mount and I need to re-plug to fix it. Not much of deal but I started to think if my dislike for systemd is strong enough to make such compromises. Second round of photos from smartphone required some tinkering, too. Generally I made many little sacrifices to keep systemd out of my machine. No problem was unsolvable but convenience and user friendliness went trough window. I was hit straight in my face with fact that Unix and POSIX was created in different times and typical computer usage evolved greatly since this time. I was starting to understand why computers need “system layer” today. Training users to type doas or sudo before every command they use in terminal isn’t great.
And then breaking point happened.
My girlfriend is a teacher in elementary school, and took some of school's laptops with Windows 10 to home to reinstall fresh OS on them and clean any residual data left after last year.
One laptop refused to reset to factory and I needed to reinstall OS. It failed once (stuck on preparing system screen) I installed it again and it broke itself doing update, and any “problem solving” tool just wasted my time showing spinning icon for some time to tell me that is was unable to fix problem. So I reinstalled it again, and again it tried to update but now it stuck installing it…
Worst part of it is when I have problem with Linux there are documentation sources manuals wikis etc. so I usually able to do something about it. But on Windows, I'm totally helpless. There are hundreds “tutorials” showing how to fix it on web, but they just show shitton of adds and then tells me to use some magic and automatic problem-fixing tools which only functionality is to show spinning icon or some progress bar because they simply don't work.
I once again noticed how user-hostile Windows is, I had so much problems and spend so much time on fixing it that I was completely mentally drained after this. And I started to think: What I’m fighting for? Systemd with all bloat on typical Linux desktop doesn’t even comes close to what I’ve seen on Windows. That was horrible experience for me, everything took ages, and computer was so busy doing some secret background tasks that it didn’t have time for me.
It’s hard to overstate how much I value having working Linux distro on my machines! Any distro will do, seriously! Forget about our Linux community infighting and go see what happened to most popular alternative!
This experience made me rethink my stance on systemd. Mundane tasks like plugging USB drive and mounting it must work, even if USB drive is encrypted. Computers are machines and should serve its users. People is what is important, not machines and software ideological purity. This is why open source and free software matters, to empower it’s users to audit and modify code, to serve its users not some shady corporate interest.
So, I’m no longer systemd opponent. Come as you are, as a friend, and help us build our software freedom enclave where all users are kings. I’ll believe in our community, in Arch developers, in Debian maintainers and many other distros that choose systemd as their default init system and system layer.
I want to have machine I’m proud of, on which I can show other people how much better computer can be. Mounting USB drive in terminal doesn’t help with this task at all… I’m not giving up my custom graphics environment build from sway, Thunar, fuzzel, foot and other parts, as it is super productive environment for me. But I’m giving up on sacrificing and reinventing things that works as expected on systemd.
So I reinstalled Arch Linux on my PC, keep Debian on my work laptop and cloned systemd repo and started to study it.
And, once again, everything I need to use my computer for, just works. Bloat is bloat but life is life and good user experience sometimes requires complexity. To stop reinventing the wheel over and over Linux needs an uniform system layer, which will provide various information and functionality to userspace. Written with modern computers’ users needs in mind. And this is what systemd is.
This is only a mirror of my Gemini capsule available here: