Seprate json part of incoming log with syslog-ng by config

Question

I'm receiving my log without any parse:

source s_network {
     network( transport(tcp) port(601) flags(no-parse));
};

My saved log have two part:

1. reciver information
2. json part of log ( like below ):

Feb 18 00:01:58 82.172.112.1 <40> 2020-02-17T20:29:49Z logserver-2 : {"method":"GET","scheme":"https","domain":"test.com","uri":"/App_Themes/font.woff","referer":"https://test.com/","ip":"31.135.115.92","ua":"Mozilla/5.0 (iPhone; CPU iPhone OS 13_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.5 Mobile/15E148 Safari/604.1","content_type":"text/html","status":404,"server_port":443,"bytes_sent":1222,"bytes_received":1245,"upstream_time":0.016}

My questions are:

How can I separate json part(remove first part of text)?

How can I generate file name base on json data? For example I want to save get method in the separate file.

Answer 1

To separate the JSON from the other information it could be as easy as:

https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.18/administration-guide/33

destination d_file {
    file("/var/log/${YEAR}.${MONTH}.${DAY}/json"
         template("${MESSAGE}\n")
         template-escape(no));
};

To access values in the JSON use the according parser:

https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.16/release-notes/the-json-parser-the-json-parser

parser parser_name {
    json-parser(
        marker()
        prefix()
    );
};

And combine the parts to get your desired result:

source s_network {
     network( transport(tcp) port(601) );
};

destination d_date_method {
    file("/var/log/${YEAR}.${MONTH}.${DAY}/${method}.json"
         template("${MESSAGE}\n")
         template-escape(no));
};

parser p_json {
    json-parser();
};

log {
    source(s_network);
    parser(p_json);
    destination(d_date_method);
};

Make sure to disable the flag no-parse in the source:

https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.16/administration-guide/para-flags-no-parse