3

say I run a command that outputs space separated values over lines. Some of which are numbers. Is there a utility I can use to calculate the mean, median, standard deviation of these numbers? something like 'cut' but that outputs these statistics.

1

Check out st: https://github.com/nferraz/st or sta (recommended): https://github.com/simonccarter/sta

From GitHub:

sta

Simple statistics from the command line interface (CLI), fast. Description

This is a lightweight, fast tool for calculating basic descriptive statistics from the command line. Inspired by https://github.com/nferraz/st, this project differs in that it is written in C++, allowing for faster computation of statistics given larger non-trivial data sets.

Additions include the choice of biased vs unbiased estimators and the option to use the compensated variant algorithm.

Given a file of 1,000,000 ascending numbers, a simple test on a 2.5GHz dual-core MacBook using Bash time showed sta takes less than a second to complete, compared to 14 seconds using st.

1

You might also consider using clistats. It is a highly configurable command line interface tool to compute statistics for a stream of delimited input numbers.

I/O options

  • Input data can be from a file, standard input, or a pipe
  • Output can be written to a file, standard output, or a pipe
  • Output uses headers that start with "#" to enable piping to gnuplot

Parsing options

  • Signal, end-of-file, or blank line based detection to stop processing
  • Comment and delimiter character can be set
  • Columns can be filtered out from processing
  • Rows can be filtered out from processing based on numeric constraint
  • Rows can be filtered out from processing based on string constraint
  • Initial header rows can be skipped
  • Fixed number of rows can be processed
  • Duplicate delimiters can be ignored
  • Rows can be reshaped into columns
  • Strictly enforce that only rows of the same size are processed
  • A row containing column titles can be used to title output statistics

Statistics options

  • Summary statistics (Count, Minimum, Mean, Maximum, Standard deviation)
  • Covariance
  • Correlation
  • Least squares offset
  • Least squares slope
  • Histogram
  • Raw data after filtering

NOTE: I'm the author.

  • 2
    If you recommended a product, you must disclose your affiliation with the product in your answers. meta.stackexchange.com/questions/57497/… – masegaloeh Jul 16 '14 at 2:48
  • I've updated the post. However, it's not that hard to figure out I'm the author. I used the same account names so folks could figure it out. Clearly, you did. – dpmcmlxxvi Jul 18 '14 at 0:01
1

The R language would be perfect for this. It runs in the terminal, its free and probably the most popular statistics languages out there. So, it has tons of documentation on how to do neat things, including the ones you are asking for.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.