I'm writing a script to consume an XML feed provided by a third party. I'm having problems downloading the files listed inside it but the provider swears there's no problem.
The files are usually around 700kb, but between 20% and 80% of the downloads just stall mid-way, and then after 10 minutes of no packets being received, I receive a RESET packet (according to wireshark).
In order to test this I have a very simple shell script:
FILE="http://www.domain.com/uploads/g4nsloshv0ppubwxxxulr4bx/an-example-file.pdf"
OUTPUT="output.pdf"
curl $FILE > $OUTPUT
I've run this many times on two machines in my office, as well as from an Amazon EC2 instance, and in all cases I see a mixture of very quick successes and also progress just halting at some mid-way point (different each time) and then never completing.
The success rate varies hugely - I might get 10 successes and then 10 stalls in a row.
I'm hoping someone can tell me if this is a common symptom for an overloaded .NET server (IIS I presume?) or whether you think there's anything I can do to prove the problem more strongly. Or indeed if there's something different I can do at my end.