7

After deleting my /var/log/journal by accident, I am unable to recover from the consequences. systemd-journald keeps failing over and over again every time I try to start it. when I run:

$ journalctl --verify
PASS: /run/log/journal/system.journal

which doesn't bring me one step closer to solve it because /run/log/journal/system.journal is created when I run /usr/lib/systemd/systemd-journald directly, which on it self outputs 

$ /usr/lib/systemd/systemd-journald
Failed to open runtime journal: Invalid argument

Things I have already tried:

  1. cleaning /var/log/journal and/run/log/journal`.
  2. copying /var/log/journal from another machine to both paths, running journalctl --verify to check them, and once again with --file flag.
  3. changing Storage setting in /etc/systemd/journald.conf to Storage=volatile just to disable /var/log/journald.
  4. Rebooting.
techraf
  • 4,243
  • 8
  • 29
  • 44
lyoko the
  • 241
  • 1
  • 2
  • 6

3 Answers3

7

After renaming the hostname, deleting & recreating /etc/machine-id by running systemd-machine-id-setup and rebooting to apply these, systemd-journald came back to life!

lyoko the
  • 241
  • 1
  • 2
  • 6
  • I deleted and recreated `/etc/machine-id` and then ran `systemd-tmpfiles --create --prefix /var/log/journal` without rebooting and it worked. – Federico Mar 31 '17 at 11:00
  • My journal service was failing in the same way due to a stray newline in the middle of the `/etc/machine-id` file. Dunno how that got there! Removing the file, running `systemd-machine-id-setup` then `systemctl start systemd-journald.service` fixed it. No reboot necessary. – markshep Oct 01 '17 at 08:28
2

Try deleting /var/log/journal in case that there are any remainders from previous restoring attempts. Then set Storage=persistent in /etc/systemd/journald.conf and reboot.

Normally systemd should recreate /var/log/journal during reboot.

If this doesn't help try

mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal
aventurin
  • 221
  • 1
  • 2
  • 7
-1

Try: systemctl stop systemd-journald && strace systemctl start systemd-journald

That should stream a lot of detail about what's going while the process is starting up. The final lines may provide a clue why it continues to fail.

Mark Stosberg
  • 3,901
  • 24
  • 28
  • 3
    Then strace will trace only systemctl, not the actual systemd. It uses IPC to communicate with main daemon. – Top Sekret Jan 04 '18 at 20:40