systemd

systemd is the init system and service manager on nearly every modern Linux distribution. It starts services, runs timers, collects logs (journald) and applies resource and security limits. Units are plain text files; systemctl controls them and journalctl reads their logs. Short versions of the commands are in Linux.

Documentation: systemd.service · systemd.exec · systemd.timer · systemd-analyze

Verification: options and commands below come from the systemd man pages and common practice. No command was executed. Check the hardening options against your systemd version (systemctl --version) and test with systemd-analyze security.

See also: Linux · logrotate · Linux Performance Troubleshooting · SELinux and AppArmor · Docker

systemctl #

Unit File Locations #

A Service Unit #

/etc/systemd/system/myapp.service:

[Unit]
Description=My app
Documentation=https://example.com/docs
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=-/etc/myapp/env
ExecStart=/opt/myapp/bin/myapp --config /etc/myapp/config.yml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp.service
systemctl status myapp.service

Drop-ins and Overrides #

Change a packaged unit without editing it. The change survives package updates.

sudo systemctl edit nginx.service

This opens an editor and saves /etc/systemd/system/nginx.service.d/override.conf. Only list what changes:

[Service]
Restart=always
RestartSec=3
Environment=TZ=Europe/London

To replace a list option such as ExecStart, clear it first:

[Service]
ExecStart=
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'

Then systemctl cat nginx to confirm and sudo systemctl restart nginx. systemctl edit runs daemon-reload for you.

Timers (cron replacement) #

A timer activates a service of the same name. Two files:

/etc/systemd/system/backup.service:

[Unit]
Description=Nightly backup

[Service]
Type=oneshot
User=backup
ExecStart=/usr/local/bin/backup.sh

/etc/systemd/system/backup.timer:

[Unit]
Description=Run backup nightly

[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=10m
Persistent=true

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer
systemctl list-timers --all
journalctl -u backup.service

Enable the timer, not the service.

Transient Units: systemd-run #

journalctl #

Persist the journal across reboots (if /var/log/journal does not exist):

sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

Or set Storage=persistent and SystemMaxUse=1G in /etc/systemd/journald.conf.

Sandboxing and Hardening #

Add to the [Service] section. Each option limits what the service can do. Test one at a time.

systemd-analyze #

Other Handy Tools #

Troubleshooting a Failed Service #

1. systemctl status <UNIT> for the exit code and last lines.
2. journalctl -xeu <UNIT> for full context.
3. systemd-analyze verify /etc/systemd/system/<UNIT> for syntax problems.
4. Run the ExecStart line by hand as the service user: sudo -u <USER> <CMD>.
5. Exit status 203/EXEC means the binary is missing or not executable; 200/CHDIR a missing WorkingDirectory; 217/USER a missing user.
6. Failing only on boot? Check After= and network-online.target.
7. Hit the restart limit? sudo systemctl reset-failed <UNIT> then start again.