5

I search a way for a reliable export of journalctl logs.

I could use the --since=... option, but this is a bit fuzzy.

In my case a script would call journalctl --output=json every ten minutes.

I don't want to miss a single line and (if possible) I would like to avoid duplicate lines.

Some days after asking this question I came across RELP: https://en.wikipedia.org/wiki/Reliable_Event_Logging_Protocol

2
  • Why did you decide to use --output=json instead of --output=export if you want a "reliable" export? – Victor Yarema May 23 '20 at 20:07
  • @VictorYarema I like the json output. It is repliable, but I want to avoid duplicates. Why is --output=export better? – guettli May 24 '20 at 14:25
9
+25

You can install a syslog daemon such as rsyslog (the default on Red Hat derived systems). This will log all journal entries in a more backward compatible manner, and of course you can specify a custom log for whatever you wish.

If you don't need logs exported in real time, you can use journalctl --since as some people have mentioned. You can run it daily at midnight with the time specifier yesterday to get exactly 24 hours of logs.


If you really need to get logs at short intervals and don't want to miss a single entry, then you need to learn about the cursor. For each log entry journalctl will provide a cursor which can be used to skip to exactly that log entry with --cursor, or the immediately following log entry with --after-cursor. Consider the following sample JSON:

{
    "__CURSOR" : "s=6ad7dcf190f3409c8bf8086fec22888c;i=286c44;b=6b134acc25e94d69b4713422b7c773be;m=46f7a97d25;t=55f5e93131a32;x=aecce3d8b96df5dc",
    "__REALTIME_TIMESTAMP" : "1512234682620466",
    "__MONOTONIC_TIMESTAMP" : "304802790693",
    "_BOOT_ID" : "6b134acc25e94d69b4713422b7c773be",
    "PRIORITY" : "6",
    "_MACHINE_ID" : "770056613d554df2abcb7757ba2e6270",
    "_HOSTNAME" : "dalaran.example.us",
    "_PID" : "1",
    "_UID" : "0",
    "_SELINUX_CONTEXT" : "system_u:system_r:init_t:s0",
    "_GID" : "0",
    "_CAP_EFFECTIVE" : "3fffffffff",
    "SYSLOG_FACILITY" : "3",
    "SYSLOG_IDENTIFIER" : "systemd",
    "_TRANSPORT" : "journal",
    "_COMM" : "systemd",
    "_EXE" : "/usr/lib/systemd/systemd",
    "_CMDLINE" : "/usr/lib/systemd/systemd --switched-root --system --deserialize 25",
    "_SYSTEMD_CGROUP" : "/init.scope",
    "_SYSTEMD_UNIT" : "init.scope",
    "_SYSTEMD_SLICE" : "-.slice",
    "CODE_FILE" : "../src/core/unit.c",
    "CODE_LINE" : "1505",
    "CODE_FUNC" : "unit_status_log_starting_stopping_reloading",
    "MESSAGE_ID" : "de5b426a63be47a7b6ac3eaac82e2f6f",
    "MESSAGE" : "Stopping OpenSSH server daemon...",
    "UNIT" : "sshd.service",
    "_SOURCE_REALTIME_TIMESTAMP" : "1512234682615526"
}

For your purposes, the __CURSOR is an opaque blob. Just capture the value from the last log entry you receive on one call to journalctl and feed it to the next call:

journalctl --output=json --after-cursor="s=6ad7dcf190f3409c8bf8086fec22888c;i=286c44;b=6b134acc25e94d69b4713422b7c773be;m=46f7a97d25;t=55f5e93131a32;x=aecce3d8b96df5dc" ...
1
  • "This will log all journal entries" it does not seem like just having rsyslog installed is sufficient for this. Seems like it requires further configuration. – Chris Stryczynski Jan 27 at 13:57
1

Use the --since option. To get logs from the last 10 minutes, just use:

--since -10m

That will give you logs 10 minutes prior to the current time. See the man page https://www.freedesktop.org/software/systemd/man/journalctl.html and this page on time specifications for systemd https://www.freedesktop.org/software/systemd/man/systemd.time.html#

1
  • OK, if use --since -10m very 10minutes, then I guess some log entries get lost. I could use it every 8 minutes. This would give me some log entries twice .... Could work, but I need to do deduplication :-( – guettli Dec 5 '17 at 14:08
1

one way of doing it (not very reliable, but can work):

$ crontab -l | tail -1
*/10 * * * * journalctl --output=json --since -10m >> journalctl.json
$ 

alternative way to do it; (assuming you need to import json output into elasticsearch):

0

syslog-ng can read from the journal and export to plain old text files. You can also setup syslog-ng to send data to other systems (including elasticsearch).

0

you can create python script for polling journalctl with using query_unique function. Probably running as a service could help also, with restart option -- to not to miss anything.

https://www.freedesktop.org/software/systemd/python-systemd/journal.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.