So here is the setup: we've got temporary access to a very large TCP WAN connection and we want to use this pipe to do WAN filesystem testing.
We would like to generate massive amounts of data on the fly, writing it to the filesystem on the other side. We've got a large number of servers to work with, so generating enough data with the right simulation isn't a problem, but we would like to simulate actual HPC application data instead of something like just piping /dev/zero.
Like I mentioned, we're looking to actually write the data, so looking for something more than iperf or netperf. My question then is do any of you know of any HPC application data simulators? How would you test writing data to the other side of the link?
I am getting close to finding a tool that fits the bill.
The most promising is MADbench2 which is actual scientific simulation code adapted for parallel I/O testing purposes.
I'm going to investigate some more tools on this page listing Parallel I/O Benchamrks
It is unclear which of these actually writes data though, which is really what we're aiming for.