systemd management using systemctl and journalctl to check systemd logs
systemd is like the service manager for your Centos and other modern Linux distributions (including Debian/Mint/Ubuntu) allows you to enable services, stop them, restart them, check their status and even reboot your system.
The key commands or arguments you will use with systemctl are the following:
Unit Commands:
list-units [PATTERN...] List loaded units
list-sockets [PATTERN...] List loaded sockets ordered by address
list-timers [PATTERN...] List loaded timers ordered by next elapse
start NAME... Start (activate) one or more units
stop NAME... Stop (deactivate) one or more units
reload NAME... Reload one or more units
restart NAME... Start or restart one or more units
try-restart NAME... Restart one or more units if active
reload-or-restart NAME... Reload one or more units if possible,
otherwise start or restart
reload-or-try-restart NAME... Reload one or more units if possible,
otherwise restart if active
isolate NAME Start one unit and stop all others
kill NAME... Send signal to processes of a unit
is-active PATTERN... Check whether units are active
is-failed PATTERN... Check whether units are failed
status [PATTERN...|PID...] Show runtime status of one or more units
show [PATTERN...|JOB...] Show properties of one or more
units/jobs or the manager
cat PATTERN... Show files and drop-ins of one or more units
set-property NAME ASSIGNMENT... Sets one or more properties of a unit
help PATTERN...|PID... Show manual for one or more units
reset-failed [PATTERN...] Reset failed state for all, one, or more
units
list-dependencies [NAME] Recursively show units which are required
or wanted by this unit or by which this
unit is required or wanted
check the state of all running services
systemctl list-units --type service --all --state failed
systemctl list-units --type service --all --state running
systemctl list-units --type service --all --state active
How Do You Enable or Disable Services
systemctl enable sshd
systemctl disable crond
Note that disabling does NOT stop them if they are currently running.
systemctl disable crond
Removed symlink /etc/systemd/system/multi-user.target.wants/crond.service.
[root@bcitcourse ~]# systemctl status crond
● crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2018-11-03 19:12:14 UTC; 2 days ago
Main PID: 139 (crond)
CGroup: /system.slice/crond.service
└─139 /usr/sbin/crond -n
Sep 18 07:19:54 bcitcourse systemd[1]: Starting Command Scheduler...
Sep 18 07:19:54 bcitcourse crond[31554]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 34% if used.)
Sep 18 07:19:54 bcitcourse crond[31554]: (CRON) INFO (running with inotify support)
Sep 18 07:19:54 bcitcourse crond[31554]: (CRON) INFO (@reboot jobs will be run at computer's startup.)
Sep 18 07:23:54 bcitcourse systemd[1]: Stopping Command Scheduler...
Sep 18 07:23:54 bcitcourse systemd[1]: Started Command Scheduler.
Sep 18 07:23:54 bcitcourse systemd[1]: Starting Command Scheduler...
Sep 18 07:23:54 bcitcourse crond[32128]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 44% if used.)
Sep 18 07:23:54 bcitcourse crond[32128]: (CRON) INFO (running with inotify support)
Sep 18 07:23:54 bcitcourse crond[32128]: (CRON) INFO (@reboot jobs will be run at computer's startup.)
Nov 01 18:20:58 bcitcourse systemd[1]: Stopping Command Scheduler...
Nov 01 18:20:58 bcitcourse systemd[1]: Stopped Command Scheduler.
Nov 01 18:21:26 bcitcourse systemd[1]: Started Command Scheduler.
Nov 01 18:21:26 bcitcourse systemd[1]: Starting Command Scheduler...
Nov 01 18:21:26 bcitcourse crond[156]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 72% if used.)
Nov 01 18:21:26 bcitcourse crond[156]: (CRON) INFO (running with inotify support)
Nov 02 17:24:11 bcitcourse systemd[1]: Stopping Command Scheduler...
Nov 02 17:24:11 bcitcourse systemd[1]: Stopped Command Scheduler.
Nov 02 17:25:08 bcitcourse systemd[1]: Started Command Scheduler.
Nov 02 17:25:08 bcitcourse systemd[1]: Starting Command Scheduler...
Nov 02 17:25:08 bcitcourse crond[135]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 37% if used.)
Nov 02 17:25:08 bcitcourse crond[135]: (CRON) INFO (running with inotify support)
Nov 02 19:26:01 bcitcourse crond[135]: (wwwmaster) BAD FILE MODE (/var/spool/cron/wwwmaster)
Nov 02 19:56:01 bcitcourse crond[135]: (wwwmaster) BAD FILE MODE (/var/spool/cron/wwwmaster)
Nov 03 19:11:56 bcitcourse systemd[1]: Stopping Command Scheduler...
Nov 03 19:11:56 bcitcourse systemd[1]: Stopped Command Scheduler.
Nov 03 19:12:14 bcitcourse systemd[1]: Started Command Scheduler.
Nov 03 19:12:14 bcitcourse systemd[1]: Starting Command Scheduler...
Nov 03 19:12:14 bcitcourse crond[139]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 66% if used.)
Nov 03 19:12:14 bcitcourse crond[139]: (wwwmaster) BAD FILE MODE (/var/spool/cron/wwwmaster)
Nov 03 19:12:14 bcitcourse crond[139]: (CRON) INFO (running with inotify support)
[root@bcitcourse ~]# ps aux|grep crond
root 139 0.0 0.0 22700 1496 ? Ss Nov03 0:00 /usr/sbin/crond -n
root 5833 0.0 0.0 112660 952 pts/3 S+ 19:11 0:00 grep --color=auto crond
How do targets work and what are they?
It is similar to the traditional runtimes where generally normal non GUI operation would be considered "runlevel 3" and the GUI would be "runlevel 5".
Instead, with systemctl you have targets for these that are based on names.
A good example is when installing your GNOME GUI:
You can do that, type startx and get your GUI. But upon reboot you'll still be at the CLI. Why?
You need to set your default target to the graphical target like below.
systemctl set-default graphical.target
What targets are available on my system?
ls /usr/lib/systemd/system
arp-ethers.service finger@.service nfs-blkmap.service rhel-configure.service sockets.target.wants systemd-poweroff.service
auth-rpcgss-module.service finger.socket nfs-client.target rhel-dmesg.service sound.target systemd-quotacheck.service
autovt@.service firewalld.service nfs-config.service rhel-domainname.service sshd-keygen.service systemd-random-seed.service
basic.target fstrim.service nfs-idmapd.service rhel-import-state.service sshd.service systemd-readahead-collect.service
basic.target.wants fstrim.timer nfs-idmap.service rhel-loadmodules.service sshd@.service systemd-readahead-done.service
blk-availability.service garbd.service nfs-lock.service rhel-readonly.service sshd.socket systemd-readahead-done.timer
bluetooth.target geoclue.service nfslock.service rpcbind.service suspend.target systemd-readahead-drop.service
brandbot.path getty-pre.target nfs-mountd.service rpcbind.socket swap.target systemd-readahead-replay.service
brandbot.service getty@.service nfs-rquotad.service rpcbind.target sys-fs-fuse-connections.mount systemd-reboot.service
canberra-system-bootup.service getty.target nfs-secure.service rpc-gssd.service sysinit.target systemd-remount-fs.service
canberra-system-shutdown-reboot.service graphical.target nfs-server.service rpcgssd.service sysinit.target.wants systemd-rfkill@.service
canberra-system-shutdown.service graphical.target.wants nfs.service rpcidmapd.service sys-kernel-config.mount systemd-shutdownd.service
console-getty.service gssproxy.service nfs-utils.service rpc_pipefs.target sys-kernel-debug.mount systemd-shutdownd.socket
console-shell.service halt-local.service nmb.service rpc-rquotad.service syslog.socket systemd-suspend.service
container-getty@.service halt.target nscd.service rpc-statd-notify.service syslog.target.wants systemd-sysctl.service
crond.service hibernate.target nscd.socket rpc-statd.service systemd-ask-password-console.path systemd-timedated.service
cryptsetup-pre.target htcacheclean.service nss-lookup.target rsyncd.service systemd-ask-password-console.service systemd-tmpfiles-clean.service
cryptsetup.target httpd.service nss-user-lookup.target rsyncd@.service systemd-ask-password-wall.path systemd-tmpfiles-clean.timer
ctrl-alt-del.target hybrid-sleep.target ntalk.service rsyncd.socket systemd-ask-password-wall.service systemd-tmpfiles-setup-dev.service
dbus-org.freedesktop.hostname1.service initrd-cleanup.service ntalk.socket rsyslog.service systemd-backlight@.service systemd-tmpfiles-setup.service
dbus-org.freedesktop.import1.service initrd-fs.target paths.target runlevel0.target systemd-binfmt.service systemd-udevd-control.socket
dbus-org.freedesktop.locale1.service initrd-parse-etc.service polkit.service runlevel1.target systemd-bootchart.service systemd-udevd-kernel.socket
dbus-org.freedesktop.login1.service initrd-root-fs.target poweroff.target runlevel1.target.wants systemd-firstboot.service systemd-udevd.service
dbus-org.freedesktop.machine1.service initrd-switch-root.service poweroff.target.wants runlevel2.target systemd-fsck-root.service systemd-udev-settle.service
dbus-org.freedesktop.timedate1.service initrd-switch-root.target printer.target runlevel2.target.wants systemd-fsck@.service systemd-udev-trigger.service
dbus.service initrd.target proc-fs-nfsd.mount runlevel3.target systemd-halt.service systemd-update-done.service
dbus.socket initrd.target.wants proc-sys-fs-binfmt_misc.automount runlevel3.target.wants systemd-hibernate-resume@.service systemd-update-utmp-runlevel.service
dbus.target.wants initrd-udevadm-cleanup-db.service proc-sys-fs-binfmt_misc.mount runlevel4.target systemd-hibernate.service systemd-update-utmp.service
debug-shell.service ip6tables.service quotaon.service runlevel4.target.wants systemd-hostnamed.service systemd-user-sessions.service
default.target iptables.service rc-local.service runlevel5.target systemd-hwdb-update.service systemd-vconsole-setup.service
default.target.wants kexec.target rdisc.service runlevel5.target.wants systemd-hybrid-sleep.service system.slice
dev-hugepages.mount kmod-static-nodes.service rdma-hw.target runlevel6.target systemd-importd.service system-update.target
dev-mqueue.mount local-fs-pre.target rdma-load-modules@.service saslauthd.service systemd-initctl.service tcsd.service
dracut-cmdline.service local-fs.target rdma-ndd.service sendmail.service systemd-initctl.socket timers.target
dracut-initqueue.service local-fs.target.wants rdma.service serial-getty@.service systemd-journal-catalog-update.service timers.target.wants
dracut-mount.service machine.slice reboot.target shutdown.target systemd-journald.service time-sync.target
dracut-pre-mount.service machines.target reboot.target.wants shutdown.target.wants systemd-journald.socket tmp.mount
dracut-pre-pivot.service messagebus.service remote-cryptsetup.target sigpwr.target systemd-journal-flush.service umount.target
dracut-pre-trigger.service multi-user.target remote-fs-pre.target sleep.target systemd-kexec.service user.slice
dracut-pre-udev.service multi-user.target.wants remote-fs.target -.slice systemd-localed.service var-lib-nfs-rpc_pipefs.mount
dracut-shutdown.service named.service rescue.service slices.target systemd-logind.service vzfifo.service
ebtables.service named-setup-rndc.service rescue.target smartcard.target systemd-machined.service vzreboot.service
emergency.service network-online.target rescue.target.wants smb.service systemd-machine-id-commit.service xinetd.service
emergency.target network-pre.target rhel-autorelabel-mark.service sm-client.service systemd-modules-load.service
final.target network.target rhel-autorelabel.service sockets.target systemd-nspawn@.service
We can check by symlink to see what the default target is:
lrwxrwxrwx 1 root root 16 Sep 18 07:18 /usr/lib/systemd/system/default.target -> graphical.target
Checking to see what services start with a target
systemctl list-dependencies default.target
default.target
● ├─display-manager.service
● ├─mysqld.service
● ├─network.service
● ├─quotaon.service
● ├─systemd-readahead-collect.service
● ├─systemd-readahead-replay.service
● ├─systemd-update-utmp-runlevel.service
● ├─vzfifo.service
● └─multi-user.target
● ├─brandbot.path
● ├─dbus.service
● ├─httpd.service
● ├─mysqld.service
● ├─network.service
● ├─rhel-configure.service
● ├─rsyslog.service
● ├─saslauthd.service
● ├─sendmail.service
● ├─sm-client.service
● ├─sshd.service
● ├─systemd-ask-password-wall.path
● ├─systemd-logind.service
● ├─systemd-update-utmp-runlevel.service
● ├─systemd-user-sessions.service
● ├─xinetd.service
● ├─basic.target
● │ ├─iptables.service
● │ ├─rhel-dmesg.service
● │ ├─paths.target
● │ ├─slices.target
● │ │ ├─-.slice
● │ │ └─system.slice
● │ ├─sockets.target
● │ │ ├─dbus.socket
● │ │ ├─rpcbind.socket
● │ │ ├─systemd-initctl.socket
● │ │ ├─systemd-journald.socket
● │ │ ├─systemd-shutdownd.socket
● │ │ ├─systemd-udevd-control.socket
● │ │ └─systemd-udevd-kernel.socket
● │ ├─sysinit.target
● │ │ ├─dev-hugepages.mount
● │ │ ├─dev-mqueue.mount
● │ │ ├─kmod-static-nodes.service
● │ │ ├─proc-sys-fs-binfmt_misc.automount
● │ │ ├─rhel-autorelabel-mark.service
● │ │ ├─rhel-autorelabel.service
● │ │ ├─rhel-domainname.service
● │ │ ├─rhel-import-state.service
● │ │ ├─rhel-loadmodules.service
● │ │ ├─sys-fs-fuse-connections.mount
● │ │ ├─sys-kernel-config.mount
● │ │ ├─sys-kernel-debug.mount
● │ │ ├─systemd-ask-password-console.path
● │ │ ├─systemd-binfmt.service
● │ │ ├─systemd-firstboot.service
● │ │ ├─systemd-hwdb-update.service
● │ │ ├─systemd-journal-catalog-update.service
● │ │ ├─systemd-journal-flush.service
● │ │ ├─systemd-journald.service
● │ │ ├─systemd-machine-id-commit.service
● │ │ ├─systemd-modules-load.service
How can we see what switchable "isolate"able targets are available?
ls -al /usr/lib/systemd/system/|grep ^d
drwxr-xr-x 22 root root 16384 Nov 3 22:59 .
drwxr-xr-x 13 root root 4096 Sep 18 07:18 ..
drwxr-xr-x 2 root root 4096 Sep 18 07:23 basic.target.wants
drwxr-xr-x 2 root root 4096 Aug 16 18:34 dbus.target.wants
drwxr-xr-x 2 root root 4096 Aug 16 18:34 default.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 graphical.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 initrd.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:23 local-fs.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:23 multi-user.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 poweroff.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 reboot.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 rescue.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 runlevel1.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 runlevel2.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 runlevel3.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 runlevel4.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 runlevel5.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 shutdown.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 sockets.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 sysinit.target.wants
drwxr-xr-x 2 root root 4096 Aug 16 18:34 syslog.target.wants
drwxr-xr-x 2 root root 4096 Sep 18 07:18 timers.target.wants
How do we switch targets without rebooting?
systemctl isolate targetname
systemctl isolate poweroff
systemctl sockets
systemctl list-sockets
LISTEN UNIT ACTIVATES
/dev/log systemd-journald.socket systemd-journald.service
/run/dbus/system_bus_socket dbus.socket dbus.service
/run/systemd/initctl/fifo systemd-initctl.socket systemd-initctl.service
/run/systemd/journal/socket systemd-journald.socket systemd-journald.service
/run/systemd/journal/stdout systemd-journald.socket systemd-journald.service
/run/systemd/shutdownd systemd-shutdownd.socket systemd-shutdownd.service
/run/udev/control systemd-udevd-control.socket systemd-udevd.service
/var/run/rpcbind.sock rpcbind.socket rpcbind.service
kobject-uevent 1 systemd-udevd-kernel.socket systemd-udevd.service
9 sockets listed.
Pass --all to see loaded but inactive sockets, too.
journalctl (systemd logging)
journalctl -u sshd
Aug 07 06:22:45 bcitcourse sshd[595]: Received disconnect from 59.63.188.32: 11: [preauth]
Aug 07 06:23:00 bcitcourse sshd[597]: Received disconnect from 221.194.47.221: 11: [preauth]
Aug 07 06:23:15 bcitcourse sshd[599]: Received disconnect from 59.63.188.32: 11: [preauth]
Aug 07 06:23:16 bcitcourse sshd[601]: Received disconnect from 221.194.44.232: 11: [preauth]
Aug 07 06:23:42 bcitcourse sshd[603]: Received disconnect from 59.63.188.32: 11: [preauth]
Aug 07 06:24:14 bcitcourse sshd[607]: Received disconnect from 59.63.188.32: 11: [preauth]
Aug 07 06:24:46 bcitcourse sshd[609]: Received disconnect from 59.63.188.32: 11: [preauth]
Jul 22 10:21:09 bcitcourse sshd[867]: reverse mapping checking getaddrinfo for 88.150-182-91.adsl-dyn.isp.belgacom.be [91.182.150.88] failed - POSSIBLE BREAK-IN ATTEMPT!
Jul 22 10:21:09 bcitcourse sshd[867]: Invalid user pi from 91.182.150.88
Jul 22 10:21:09 bcitcourse sshd[867]: input_userauth_request: invalid user pi [preauth]
Jul 22 10:21:09 bcitcourse sshd[865]: reverse mapping checking getaddrinfo for 88.150-182-91.adsl-dyn.isp.belgacom.be [91.182.150.88] failed - POSSIBLE BREAK-IN ATTEMPT!
Jul 22 10:21:09 bcitcourse sshd[865]: Invalid user pi from 91.182.150.88
Jul 22 10:21:09 bcitcourse sshd[865]: input_userauth_request: invalid user pi [preauth]
Jul 22 10:21:09 bcitcourse sshd[867]: pam_unix(sshd:auth): check pass; user unknown
Jul 22 10:21:09 bcitcourse sshd[867]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=91.182.150.88
Jul 22 10:21:09 bcitcourse sshd[865]: pam_unix(sshd:auth): check pass; user unknown
Jul 22 10:21:09 bcitcourse sshd[865]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=91.182.150.88
Jul 22 10:21:11 bcitcourse sshd[867]: Failed password for invalid user pi from 91.182.150.88 port 46846 ssh2
Jul 22 10:21:11 bcitcourse sshd[865]: Failed password for invalid user pi from 91.182.150.88 port 46842 ssh2
What's the difference?
journalctl -b 2 and journalctl -b -2
It is the fact the first "2" is a positive and the second "-2" is a negative.
Thus, 1 means the first boot found in the
journal in chronological order, 2 the second and so on; while -0 is
the last boot, -1 the boot before last, and so on.
Tags:
systemd, systemctl, journalctl, logssystemd, centos, allows, enable, restart, reboot, commands, arguments, sockets, timers, elapse, activate, deactivate, reload, active, isolate, processes, pid, runtime, properties, ins, manual, reset, dependencies, recursively, targets, runtimes, generally, gui, quot, runlevel, installing, gnome, startx, ll, cli, default, graphical, disable, sshd, crond, disabling, currently, symlink, etc, multi, user, bcitcourse, scheduler, usr, lib, disabled, vendor, preset, enabled, utc, cgroup, slice, sbin, sep, cron, info, random_delay, scaled, factor, inotify, startup, nov, wwwmaster, mode, var, spool, ps, aux, grep, ss, pts, auto, ls, arp, ethers, nfs, blkmap, rhel, configure, poweroff, auth, rpcgss, module, socket, dmesg, quotacheck, autovt, firewalld, config, domainname, keygen, fstrim, idmapd, import, readahead, timer, idmap, loadmodules, blk, availability, garbd, readonly, bluetooth, geoclue, nfslock, rpcbind, suspend, brandbot, getty, mountd, swap, replay, rquotad, sys, fs, fuse, connections, mount, canberra, bootup, rpc, gssd, sysinit, remount, shutdown, server, rpcgssd, rfkill, rpcidmapd, kernel, shutdownd, console, gssproxy, utils, rpc_pipefs, debug, shell, halt, nmb, syslog, container, nscd, statd, notify, sysctl, hibernate, password, timedated, cryptsetup, htcacheclean, nss, lookup, rsyncd, tmpfiles, httpd, ctrl, alt, hybrid, ntalk, dev, dbus, org, freedesktop, hostname, initrd, cleanup, rsyslog, backlight, paths, binfmt, udevd, locale, parse, polkit, bootchart, login, firstboot, fsck, udev, timedate, printer, proc, nfsd, update, binfmt_misc, automount, resume, utmp, udevadm, db, ip, quotaon, hostnamed, sessions, iptables, rc, hwdb, vconsole, kexec, rdisc, hugepages, kmod, static, nodes, rdma, hw, importd, mqueue, modules, saslauthd, initctl, tcsd, dracut, cmdline, ndd, sendmail, initqueue, catalog, journald, sync, tmp, pivot, messagebus, sigpwr, flush, umount, localed, slices, logind, vzfifo, ebtables, rndc, smartcard, machined, vzreboot, online, smb, xinetd, autorelabel, sm, nspawn, thus, chronological,