Systemd Essential

Posted on | Edited on | In

Version

Different version may have different syntax and options in unit file, check it first:

1
systemctl --version

To see systemd service unit configuration:

1
man 5 systemd.service

Systemd

How To Use Systemctl to Manage Systemd Services and Units

History:

1
SysV Init -> Upstart -> Systemd

The systemd, system and service manager, is an init system used to bootstrap the user space and to manage system processes after boot. Use systemctl command to manage the service on a systemd enabled system.

The fundamental purpose of an init system is to initialize the components that must be started after the Linux kernel is booted (traditionally known as “userland” components). The init system is also used to manage services and daemons for the server at any point while the system is running.

Main commands, for example using nginx, some commands have to use sudo if you are non-root user, since it will affect te state of the operating system, you can leave off the .service suffix in command.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# start on boot
# This hooks it up to a certain boot “target”
# causing it to be triggered when that target is started.
sudo systemctl enable nginx.service
sudo systemctl disable nginx.service

# start and stop
sudo systemctl start nginx.service
sudo systemctl stop nginx.service

# when change the configuration of service
# To attempt to reload the service without 
# interrupting normal functionality
sudo systemctl reload nginx.service
# if reload is not available, restart instead
sudo systemctl reload-or-restart nginx.service

# status overview, you can see:
# unit file path
# drop-in
# enabled or disabled at vendor and custom
# up time
# Cgroup
systemctl status nginx.service

# find overridden config files for all units
# check unit drop in config snippets
sudo systemd-delta

enable will create a soft link into the location on disk where systemd looks for autostart files (usually /etc/systemd/system/some_target.target.wants).

The exit code can be used for shell script:

1
2
3
4
# may need sudo
systemctl is-active nginx.service
systemctl is-enabled nginx.service
systemctl is-failed nginx.service

Check system states managed by systemd:

1
2
3
4
5
6
7
8
9
10
11
# list enabled units only
systemctl [list-units]

# list all of the units that systemd has loaded or attempted to load into memory
# include not currently active
systemctl list-units --all [--state=active|inactive|failed] [--type=service|target]
# list failed daemons, this is useful when reboot VM but daemons not
systemctl list-units --state failed

# show every available units installed on the system
systemctl list-unit-files

list-unit-files state column: The state will usually be enabled, disabled, static, or masked. In this context, static means that the unit file does not contain an [Install] section, which is used to enable a unit. As such, these units cannot be enabled. Usually, this means that the unit performs a one-off action or is used only as a dependency of another unit and should not be run by itself.

1
2
3
# after mask, cannot enable or start service
sudo systemctl mask nginx.service
sudo systemctl unmask nginx.service

To see full content and path of a unit file, vanilla unit files(don’t touch them, override them if needed in other places, for example /etc/systemd/system) are in /usr/lib/systemd/system and customized are in /etc/systemd/system folder:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# show unit file content and path
# if has overriding snippet, will show them as well
systemctl cat nginx.service

# list dependencies 
systemctl list-dependencies nginx.service [--all] [--reverse] [--before] [--after]

# low-level detail of unit
# all key=values
# -p: display a single property
systemctl show nginx.service [-p ExecStart]

# edit unit file
sudo systemctl edit --full nginx.service
# append unit file snippet
sudo systemctl edit nginx.service
# then reload to pick up changes
sodu systemctl daemon-reload

You can also further override by creating override.conf file in /etc/systemd/system/xxx.service.d folder, there are some principles for overriding, see here.

In systemd, service and other unit files can be tied to a target.

Targets are special unit files that describe a system state or synchronization point. Like other units, the files that define targets can be identified by their suffix, which in this case is .target. Targets do not do much by themselves, but are instead used to group other units together.

This can be used in order to bring the system to certain states, much like other init systems use runlevels. (仍然可以显示系统的runlevel的)

For instance, there is a swap.target that is used to indicate that swap is ready for use. Units that are part of this process can sync with this target by indicating in their configuration that they are WantedBy= or RequiredBy= the swap.target. Units that require swap to be available can specify this condition using the Wants=, Requires=, and After= specifications to indicate the nature of their relationship.

1
2
3
4
5
6
7
8
9
10
11
12
# all available targets
systemctl list-unit-files --type=target

# current default target
systemctl get-default

# set default target
sudo systemctl set-default multi-user.target
sudo systemctl set-default runlevel3.target

# see what units are tied to a target
systemctl list-dependencies multi-user.target

Unlike runlevels, multiple targets can be active at one time. An active target indicates that systemd has attempted to start all of the units tied to the target and has not tried to tear them down again.

1
2
# show all active targets
systemctl list-units --type=target

This is similar to changing the runlevel in other init systems. For instance, if you are operating in a graphical environment with graphical.target active, you can shutdown the graphical system and put the system into a multi-user command line state by isolating the multi-user.target. Since graphical.target depends on multi-user.target but not the other way around, all of the graphical units will be stopped.

1
2
3
4
# check units that will be kept alive
systemctl list-dependencies multi-user.target
# transition
sudo systemctl isolate multi-user.target

Stopping and rebooting system, note shutdown, reboot, poweroff are actually softlink to systemctl!

1
2
3
4
sudo systemctl poweroff
sudo systemctl reboot
# boot into rescue mode (single-user)
sudo systemctl rescue

These all alert logged in users that the event is occurring.

有意思的是，这些都是同一个softlink，但是调用却有不同的效果呢? 利用了$0 作为判断, see here.

1
2
3
4
5
6
lrwxrwxrwx. 1 root root          16 Jun  6  2020 halt -> ../bin/systemctl
lrwxrwxrwx. 1 root root          16 Jun  6  2020 poweroff -> ../bin/systemctl
lrwxrwxrwx. 1 root root          16 Jun  6  2020 reboot -> ../bin/systemctl
lrwxrwxrwx. 1 root root          16 Jun  6  2020 runlevel -> ../bin/systemctl
lrwxrwxrwx. 1 root root          16 Jun  6  2020 shutdown -> ../bin/systemctl
lrwxrwxrwx. 1 root root          16 Jun  6  2020 telinit -> ../bin/systemctl

Systemd Journal

How To Use Journalctl to View and Manipulate Systemd Logs

The journal is implemented with the journald daemon, which handles all of the messages produced by the kernel, initrd, services, etc.

这里介绍了关于systemd service journal的使用，更详细的介绍Journal 可以参考我的blog <<Linux System Admin>>.

Set time of the prompt:

1
2
3
4
5
6
7
8
9
10
# available time zone
timedatectl list-timezones
# set time zone
sudo timedatectl set-timezone America/Los_Angeles
# check
timedatectl status

# or using UTC
# --utc: time zone
journalctl --utc

To see log of a specific service:

1
2
3
4
5
6
7
8
9
10
11
12
13
# full log
journalctl -e

# -u: unit, but some log may be missing in ExecStartPre
# -b: limit to current boot
# -e: show ending logs
# -r: show in reverse order
# -f: tail log
journalctl -u nginx.service [--since] [--until] [-e]

# combine related units
# good for debug
journalctl -u nginx.service -u php-fpm.service --since today

Service Unit File

Understanding Systemd Units and Unit Files

Here only focus on .service unit. Systemd manages a broad range of resources, such as .target, .socket, .device, .mount, .swap, etc. They may have different section blocks.

Section block names are case-sensitive. Non-standard section name [X-name] has a X- prefix.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
[Unit]
#This is generally used for defining metadata for the unit 
# and configuring the relationship of the unit to other units
Description=
Documentation=

Requires=
BindsTo=

Wants=
Before=
After=

Conflicts=
Conditionxxx=
Assertxxx=

[Service]
# provide configuration that is only applicable for services
Environment=
Type=simple(default)|forking|oneshot|dbus|notify|idle

PIDFile=

# deprecated syntax
# if true, User and Group only applied to ExecStart
PermissionsStartOnly=true

ExecStartPre=

ExecStart=
ExecStartPost=
ExecReload=
ExecStop=
ExecStopPost=

User=
Group=

RestartSec=
Restart=always|on-success|on-failure|on-abnormal|on-abort|on-watchdog
TimeoutSec=

[Install]
# only units that can be enabled will have this section
WantedBy=
RequiredBy=

Alias=
Also=
DefaultInstance=

key=value pairs 不止这些，遇到新的可以补充。

Systemd does not use a shell to execute commands and does not perform $PATH lookup, so for example in ExecStartPre, must specify shell context to run command:

1
ExecStartPre=/bin/sh -c 'pgrep process_name > /var/run/process_name.pid'

And because of no $PATH lookup, must use absolute path to binary or executable.

Setting PermissionsStartOnly=true means that User & Group are only applied to ExecStart. So switching to the new syntax will be :

1
2
3
4
5
6
ExecStartPre=+/bin/bash -c '/bin/journalctl -b -u ntpdate | /bin/grep -q -e "adjust time server" -e "step time server"'
ExecStartPre=+/bin/mkdir -p /path/to/somedir
ExecStartPre=-/usr/bin/<command that may fail>
ExecStart=/path/to/myservice
ExecStop=+/bin/kill -INT ${MAINPID}
ExecReload=+/bin/kill -INT ${MAINPID} && /path/to/myservice

Prefix with + will be executed with higher privilege as root. Prefix - means if any of those commands fail, the remaining exec will not be interrupted, otherwise the unit is considered failed.

Also note that per ExecStartPre is running in isolation.

If needs to check the log of latest daemon start process, this will show you all log instead of only unit, I found if use -u to specify unit, some log will miss:

1
2
3
4
# use `b` and `f` to move page
# -e: tail
# -x: more details
journalctl -ex

Systemd with multiple execStart: serivce type 和 ExecStart 个数有关.

0%