I recently set up Logstash with Kibana and Elasticsearch, but am having a problem whereby it's not reading logs unless the directory permissions are world read/execute (and the files at least world read).

I've installed version 1.4.2 from the RPM, which runs as the logstash user. I've made sure the logstash user is a member of the groups relating to logs it needs to read, e.g. "apache".

[root@rhel-00081 conf]# id logstash
uid=497(logstash) gid=497(logstash) groups=48(apache),10081(nexus),27666(spark),497(logstash)

The default permission on /var/log/http is drwxr-x--- root apache, yet Logstash will not process the logs until I change this to drwxr-xr-x

Any idea why this is?

  • Have you restarted logstash after it was given that group membership? Please show the output from pgrep logstash. – Jenny D says Reinstate Monica Sep 30 '14 at 10:28

Ok, I have discovered the problem. The init script supplied in the RPM starts logstash as follows:

nice -n ${LS_NICE} chroot --userspec $LS_USER:$LS_GROUP / sh -c "
ulimit -n ${LS_OPEN_FILES}
exec \"$program\" $args
" > "${LS_LOG_DIR}/$name.stdout" 2> "${LS_LOG_DIR}/$name.err" &

But when running under the chroot command, all secondary group membership is removed unless you supply the --groups option. You can see this if you run:

[root@rhel-00081 ~]# chroot --userspec logstash:logstash / bash -c "id"
uid=497(logstash) gid=497(logstash) groups=497(logstash),0(root)

Therefore the apache logs are unreadable...

[root@rhel00081 httpd]# chroot --userspec logstash:logstash / bash -c "cat /var/log/httpd/access_log"
cat: /var/log/httpd/access_log: Permission denied

The chroot line in the init script needs to be preceded with something like:

SUPP_GROUPS=$(groups $LS_USER | cut -d " " -f 4- | tr " " ",")
if [ ! -z ${SUPP_GROUPS} ]

and then modify the beginning of the chroot line:

nice -n ${LS_NICE} chroot ${SUPP_GROUP_STR} --userspec=$LS_USER:$LS_GROUP / sh -c "
... etc ...

This is a known issue in the logstash 1.4.2 init script. https://github.com/elastic/logstash/pull/1398

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.