A 20 GB VPS is a tight fit. On a busy host, the system journal can grow fast, especially with frequent reboots, kernel messages, or misconfigured services that spam systemd-journald. When the journal consumes several gigabytes, you hit the “no space left on device” error on /var/log or /var/run, and the OS starts throttling services or, worse, refuses to start new ones.
systemd‑resolved is giving you 127.0.0.1 for every hostUbuntu 24.04 ships with systemd‑resolved as the DNS resolver.
If every hostname you look up ends up at 127.0.0.1, the most common culprit is the search‑domain order that resolved builds.
It pulls the list from three places:
/etc/resolv.conf (or the DNS=/Domains= directives in /etc/systemd/resolved.conf)$HOSTNAME)When localhost or the local hostname lands at the top, the resolver will try it before forwarding the query to the real DNS servers, and you’ll get a loopback address for any name that isn’t fully qualified.
When a server’s RAM jumps from 30 % to 90 % in a matter of minutes, the first instinct is to kill processes, restart services, or add more memory. Those are quick fixes, but they don’t tell you why the spike happened. On a system that runs systemd, the combination of journalctl and systemd‑coredump gives a precise, timestamp‑aligned view of what went wrong. This post walks through a real‑world scenario, shows how to extract the relevant logs, and explains how to interpret the data so you can fix the root cause and harden the system against future incidents.
When a home‑NAS sits on a single‑user machine, the first instinct is to give the NAS service full root access. That works, but it also opens a door for accidental or malicious privilege escalation. Rootless Podman lets you run containerized services—Nextcloud, Syncthing, or a simple Nginx reverse proxy—inside a user namespace that isolates the container from the host. The container can still read and write to your data directories, but it can’t touch the kernel or system services.
[Read More]When you power on a server and hit ssh user@host, the connection dies in a few seconds even though the key works on a fresh machine. The usual suspect is the SSH agent – it’s either not running or not forwarding the key after a reboot. The fix is straightforward: make the agent start automatically, add your key once, and let ssh_config do the rest.
ssh-add -l
If you get The agent has no identities, the agent isn’t started or $SSH_AUTH_SOCK isn’t set. On most modern distros the agent is started by the display manager or a systemd‑user unit. Check it with:
If a service starts spamming logs, a full /tmp can bring the whole system to a halt. On low‑RAM servers or in containers where /tmp lives on a regular disk partition, the problem is even worse. Mounting /tmp as a tmpfs keeps it in memory, guarantees instant cleanup, and stops disk‑space exhaustion. Below is a practical, step‑by‑step guide that covers the trade‑offs and security notes you’ll need to keep in mind.
When I run a fresh upgrade from Ubuntu 22.04 LTS to 24.04 LTS on a home server, the first thing that can bite me is the infamous “Broken Packages” dialog that stalls the upgrade process. It isn’t a bug in the installer; it’s the package manager trying to juggle a huge shift in dependencies between releases. The result is a long pause—sometimes hours—before the system finally moves on. For a server that should stay online, that pause is unacceptable.
[Read More]You tweak a unit file, hit systemctl, and nothing happens.
That’s the classic “I changed the file, but systemd still runs the old version” problem.
The daemon is still holding the old config in memory. systemd keeps a cache of unit files; it only rereads them after a daemon‑reload. If you skip that step, the changes stay invisible until you reboot or reload manually.
When you try to move a file into a directory that has the sticky bit set, the kernel first checks the directory’s owner and the user’s write permission. If the directory is owned by someone else, the sticky bit forces it to also check the file’s owner. ACLs can grant write access to a group or user, but the sticky‑bit logic still requires the file’s owner to match the directory’s owner unless the moving user has the write permission on the file itself. That subtle rule is the source of a lot of “Permission denied” errors that only surface when ACLs are involved.
[Read More]Putting a WAF in front of an application used to be a simple decision. Buy an appliance, load a signature set, place it at the perimeter, and call the problem solved. That model still exists, but it no longer matches how applications are built or how they are attacked. A traditional standalone WAF is now widely treated as obsolete as a primary security control — not because rule matching stopped working, but because the perimeter it was designed for has largely disappeared.
[Read More]