1

I have a program that continually reads the output of:

journalctl -o json --follow

And sends it to another process. And while it outputs journal entries initially from the tail, new ones don't appear on stdout while it's running. It's as though follow mode isn't following but stalls.

So to test this, I ran up two terminal windows:

  1. journalctl -o json --follow
  2. echo "Hello world" | systemd-cat x 10 times

The first time I ran 2. it appeared to follow. Subsequent runs of 2 and nothing appears.

This is a huge problem. Follow isn't really following?

OS is Ubuntu 16.04 on x86_64

hookenz
  • 14,472
  • 23
  • 88
  • 143

3 Answers3

0

It may also be the case you/someone/ntp changed the time backwards. It turns out that journalctl -f shows the latests logs based on date, not based on latest incoming logs...

Check out: https://github.com/systemd/systemd/issues/5345

Rocq
  • 1
0

You may be out of inotify resources. Oddly My remote VSCode was taking up too many of them. You can increase these by doing something like this: https://stackoverflow.com/a/56292289/1394731

Apparently newer versions of systemd give you warnings when this happens: https://github.com/systemd/systemd/issues/2321#issuecomment-1032425855

theicfire
  • 101
  • 3
0

Two things might be happening:

  1. If you log the same line 10 times, duplicated lines might be filtered, try changing the hello world msg

  2. There might be some buffering between the program you use to parse the output of journalctl -f -o json and journalctl itself, see View unbuffered log output from journalctl

Luca Gibelli
  • 2,731
  • 1
  • 22
  • 30
  • I tried both of those things. Interestingly, that doesn't occur on another machine with a systemd journal. The journalctl binary is newer. It might just be a bug. – hookenz May 27 '18 at 21:47
  • upgraded to 18.04, same issue. – hookenz Jun 07 '18 at 21:32
  • 1
    Turned out it was a corrupt /var/log/journal. Did vaccuum several times to preserve least 10 mins and it was showing events from 6 months ago so. Blew it all away then voila! – hookenz Jul 14 '18 at 01:54