I've needed to do some more sophisticated graphing and plotting as of late. While I know that gnuplot is up to the task, and can do it all from the command line, are there other, simpler or more powerful tools that I'm missing?
I'm a bit oldschool, so I tend to think of sed, awk, bash and gnuplot when I think of statistics and charting.
Ideally, I'd like to parse stuff into a local mysql db, create batch jobs, set alerts on incoming data, etc. A real-time graph would be fantastic too. Gnuplot can do all this, but I don't want to reinvent the wheel if I don't have to. There have been some great advances in visualization tools, and to be honest, although the output is great, parsing, grouping, and sorting logs and data into forms that are agreeable to gnuplot slows down my ability to quickly assemble intelligent queries.
Mixing SQL with gnuplot seems like the way to go... but they don't connect too nicely. It will be a bit of a pain.
As an example, my most recent project would be taking months of Apache log data and looking for attack patterns to build defensive signatures. Date/time/geography/site/url visited, there are endless possible things to plot and sift through. Grouping by date-ranges, etc.
The followup project to that would be to apply the signature (statistical or otherwise), and generate alerts/responses to the attack patterns. While one tool might dig through static logs nicely, it won't necessarily do dynamic stats.