0

What should I be looking for, when wget returns 500 but the same URL works fine in my web browser? I don't see any access_log entries that seem to be related to the error.

DEBUG output created by Wget 1.14 on linux-gnu.

<SSL negotiation info stripped out>

---request begin---
GET /survey/de/tools/clear-caches/password/<some-token> HTTP/1.1
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Host: testing.thesurveylab.net
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.0 500 Internal Server Error
Date: Wed, 12 Dec 2012 14:53:07 GMT
Server: Apache/2.2.3 (CentOS)
Set-Cookie: blueprint2-staging=8jnbmkqapl30hjkgo0u6956pd1; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Strict-Transport-Security: max-age=8640000;includeSubdomains
X-UA-Compatible: IE=Edge,chrome=1
Content-Length: 5
Connection: close
Content-Type: text/html; charset=UTF-8

---response end---
500 Internal Server Error

Stored cookie testing.thesurveylab.net -1 (ANY) / <session> <insecure> 
[expiry none] blueprint2-staging 8jnbmkqapl30hjkgo0u6956pd1
Closed 3/SSL 0x0000000001f33430
2012-12-12 15:53:07 ERROR 500: Internal Server Error.
8
  • 1
    Please share the full output of your wget command. It could be that the server is checking for a matching user agent string or performing a redirect wget interprets differently.
    – gertvdijk
    Dec 12 '12 at 14:51
  • You should add --debug to wget command and provide more information to us Dec 12 '12 at 14:52
  • My suggestion would be to try and use curl. It probably will throw a 500 identical arror but might fetch the page. Let me know of the progress Dec 12 '12 at 15:02
  • I will try that but I want to understand and fix the problem, not just switch to curl and let it be.
    – markus
    Dec 12 '12 at 15:03
  • 2
    I am almost certain that this problem is a result of a buggy webpage. You should contact the maintainer of the webpage and report this. wget gives a faulty 500 report. Dec 12 '12 at 15:06
1

There are probably a gazillion possible reasons for this but whenever I've struck the same issue, whether it be using wget or some other downloader, it's usually because there is code on the site which is designed to respond in certain ways to "normal" web browsers. Along comes wget, or similar, and the code breaks because it doesn't have code for it, nor a proper fallback. Such site usually break for very old browsers as well. In such cases you have to get the downloader to mimic a common browser to get it to work correctly.

1
  • Yes, after all I realize, everyone here was quite right. Once I tried the call with curl I finally got the feedback I needed to realize there was an early error from which the site was able to recover when viewed in a 'normal' browser. wget wasn't able to handle the error but curl output the stack trace.
    – markus
    Dec 15 '12 at 12:26
1

This question had been already asked and answered into this thread.

It's a bug in the webpage. 
The HTTP status is indeed seemingly incorrectly set to HTTP 500. Firefox/Firebug also confirms this. 
Basically, you're facing a HTTP 500 error page with "normal" content.

    Report it to the site admin.

https://stackoverflow.com/questions/6083102/http-500-error-in-wget

2
  • 1
    "It's a buggy webpage" is NOT an answer as much as "doesn't work" is not a valid error message.
    – markus
    Dec 12 '12 at 15:37
  • 1
    We really do prefer that answers contain content not pointers to content. Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – user9517
    Dec 14 '12 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.