Quickly Pinpointing Unauthorised sudo Attempts with Journalctl Filters

When a user or a process tries to run something with sudo, the action is recorded in the system journal. Because sudo writes to the journal, you can filter on the SYSLOG_IDENTIFIER or MESSAGE fields to spot failed attempts, audit who tried what, and even trigger alerts.

Below is a step‑by‑step guide that shows how to:

  • Identify failed sudo attempts in real time
  • Pull historical data for a specific user or time window
  • Distinguish between legitimate and suspicious activity
  • Automate the process with a small script
  • Understand trade‑offs and common pitfalls

The examples assume a recent systemd‑based distro (Debian 12, Ubuntu 24.04, Fedora 40, etc.) and that sudo is compiled with the default --syslog flag (enabled since 2019). If your system uses a custom sudo build or a different logging backend, the commands will need adjustment.

[Read More]

Why my SSH jump host was silently dropping connections after an OpenSSH 9.5 update and how I fixed it

SSH jump host silently dropping connections after OpenSSH 9.5

A recent OpenSSH 9.5 release tweaked how the server handles TCP keepalive packets. My jump host – a modest Ubuntu 24.04 box – started dropping established SSH sessions after a few minutes of idle time. The problem popped up during a routine audit when several users complained of “connection closed” messages, yet nothing appeared in the logs.

What changed in 9.5?

OpenSSH 9.5 introduced new defaults for ClientAliveInterval and ClientAliveCountMax on the server side:

[Read More]

When systemd‑resolved silently drops a local /etc/hosts entry and the service you can’t reach

systemd‑resolved is the default DNS resolver on most modern distros. It pulls /etc/hosts unless you flip the --no-hosts flag. In 2026 a subtle tweak in its default behaviour started swallowing local hostname lookups, and no one noticed until a service went down.

Why it happens

systemd-resolved now gives the DNS stack a higher priority than the hosts file when you call getaddrinfo(3). If a name shows up in both /etc/hosts and a DNS zone, the DNS answer wins. When the zone is empty or the query times out, it silently falls back to the hosts file. But if the DNS server is unreachable and the hosts entry points to 127.0.0.1 or ::1, the resolver drops that entry and returns ENOTFOUND. The change was meant to stop a compromised DNS server from poisoning local lookups.

[Read More]

When resolvectl Caches Fail: Quick Steps to Clear Stale DNS Entries Without Restarting NetworkManager

When a host suddenly resolves a domain to an old IP, the first instinct is to restart NetworkManager or reboot the machine. That works, but it’s heavy‑handed and can disrupt services. In a production environment, a brief outage can trigger SLA violations or expose stale data to clients. Below is a concise, hands‑on guide to flushing the systemd‑resolved cache safely, with a focus on security‑aware operators who need minimal downtime.

[Read More]

When ss Shows No Open TCP Ports Even Though Services Are Running

Why ss Might Show No Open TCP Ports Even When Services Are Running

I’ve seen this happen on fresh installs, on servers that have just rebooted, and even on production machines that have been up for months. The symptom is simple: ss -tuln prints nothing, but systemctl status nginx (or whatever daemon you’re checking) says active (running). The mismatch usually boils down to one of three things:

  1. The service is listening on a non‑TCP socket
  2. The service is bound to a Unix domain socket or a local‑only address
  3. The socket is managed by a socket‑activated unit that isn’t yet in use

Below is a quick checklist that will help you zero in on the culprit, plus a few hardening tips that come with each case.

[Read More]

When nftables silently drops HTTP traffic: a step‑by‑step port‑filter investigation

HTTP is the backbone of most web services. When a client can’t reach a web server, the first instinct is to look at the application logs, DNS, or the network path. A silent drop by nftables is a common, yet often overlooked, culprit. This post walks through a systematic approach to uncovering why nftables is silently discarding HTTP packets, with a focus on the port‑filtering rules that most administrators set up for hardening.

[Read More]

Why My Nginx Service Stops After a Kernel Upgrade: A Systemd Timer Debugging Session in a Home‑Lab Server

I upgraded the kernel on my 2026‑era Debian‑based home‑lab server from 6.1.12‑generic to 6.5.0‑generic. The installer was a breeze – “All packages up to date” and a clean reboot. A few hours later, Nginx stopped answering HTTP. The systemctl status nginx looked like this:

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2026‑11‑06 14:32:12 UTC; 1h 12min ago
  Process: 1234 ExecStart=/usr/sbin/nginx -g daemon off; (code=exited, status=0/SUCCESS)
   Main PID: 1234 (code=exited, status=0/SUCCESS)

The master exited cleanly, yet the port was closed. The only clue? The kernel had changed. Below is the debugging walk‑through, the root cause, and how to harden the system so a kernel upgrade never silently kills a critical service again.

[Read More]

Hardening Serverless Function IAM: A Step‑by‑Step Fix for Over‑Permissive Execution Roles

When a serverless function is granted a blanket AdministratorAccess policy, the blast radius of a compromised function is the entire account. In 2025, a handful of high‑profile breaches were traced back to overly permissive IAM roles attached to Lambda, Cloud Functions, or Azure Functions. The fix is simple: reduce the scope of the role to the minimum set of actions required by the function. Below is a practical walkthrough that blends AWS, Azure, and GCP examples, with a focus on Linux sysadmins who are comfortable with CLI, JSON, and policy evaluation.

[Read More]

The Cost of Over‑Broad Retention: How a 30‑Day Backup Policy Exposed 2 TB of PHI

A 30‑day retention window is a default in many backup scripts and commercial tools. It feels safe: “I’ll keep the last month of data, that’s enough for recovery.” In 2025, a mid‑size clinic used exactly that policy on a set of encrypted LVM snapshots stored on a single NAS. When the NAS firmware update rolled out, the snapshots were corrupted, and the backup system fell back to a restore‑all mode. The result? 2 TB of unencrypted PHI leaked to a rogue process that had gained write access to the backup directory.

[Read More]

Turning a Splunk dashboard into a hands‑on threat‑hunting playbook: a step‑by‑step recipe for real‑world detection agility

Turning a Splunk Dashboard into a Hands‑On Threat‑Hunting Playbook

Splunk dashboards are great for visualizing data, but they rarely give you a ready‑to‑execute hunting workflow. In a real‑world environment you need a playbook that tells you what to look for, how to filter the data, and what to do when you find something. This post walks through turning an existing Splunk dashboard into a fully documented playbook that can be run by a sysadmin or a SOC analyst on a Linux host. The focus is on practical commands, trade‑offs, and troubleshooting tips that fit the Linux‑security‑first mindset of the Linux Security blog.

[Read More]