I'm trying to figure out the best way to setup a central place to store and interrogate server logs. syslog, Apache, MySQL etc.
I've found a few different options but I'm not sure what would be best. I'm looking for something that is easy to install and keep updated on many virtual machines. I can add it to a VM template going forward but I'd also like it to be easy to install to keep the VM complexity down.
The options I've found so far are:
- syslogd/syslog-ng/rsyslog to logstash/ElasticSearch
- logstash agent in each log "client" to send to Redis/logstash/ElasticSearch
And all sorts of permutations of the above.
What's the most resilient and light from the log "client" perspective?
I'd like to avoid the situation where log "clients" hang because they are unable to send their logs to the logging server. Also I would still like to keep local logging and the rotation/retention provided by logrotate in place.
Any ideas/suggestions or reasons for or against any of the above?
Or suggestions of a different structure entirely?