Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a problem with downloading a website using wget.

First I need to login to the website with a username and password:

wget --save-cookies cookies.txt --post-data --cookies=on --keep-session-cookies 
     --post-data="strUser=username&strPW=password" 
     http://hostname/wwwawelle/member/nixlog.asp

Then I try to download the complete website http://hostname/wwwawelle/member/ :

wget -k -r -m -np --load-cookies cookies.txt 
     http://hostname/wwwawelle/member/80_01_member.asp

I get a response saying that I am not logged in then it redirects me to:

http://hostname/wwwawelle/member/nixlog.asp?grund=notloggedin

Any ideas?

HttpFox

00:00:07.712    0.193   597 301 POST    302 Redirect to: 80_01_member.asp   http://hostname/wwwawelle/member/login.asp
00:00:07.943    0.255   481 11957   GET 200 text/html   http://hostname/wwwawelle/member/80_01_member.asp
00:00:08.232    0.060   428 (6054)  GET (Cache) text/css    http://hostname/wwwawelle/style.css
00:00:08.260    0.051   420 (589)   GET (Cache) application/x-javascript    http://hostname/wwwawelle/news/jscr_inc.js

Cookies

ASPSESSIONIDCSATRCQQ    APPIOGADBMLHOMFHCALPBFNL    /   hostname    End Of Session

POST Data

strUser username
strPW   password
Abschicken  Anmelden

wget cookies.txt

# HTTP cookie file.
# Generated by Wget on 2011-03-28 14:23:17.
# Edit at your own risk.

hostname    FALSE   /   FALSE   0   ASPSESSIONIDCSATRCQQ    EBAJOGADDHGHCANLKJCMDCPP
share|improve this question
    
Is there any javascript on the site that might be getting in the way? Also could the site be checking the REFERER? –  Kev Mar 28 '11 at 9:29
    
Is your cookies.txt populated? –  PriceChild Mar 28 '11 at 9:37
    
After logging-in with wget, do the contents of cookies.txt look sensible? –  nickgrim Mar 28 '11 at 9:43
    
so here some inputs... –  Rama Mar 28 '11 at 12:47
    
were you able to success in this? –  Mowgli Mar 28 '13 at 12:43

3 Answers 3

Could be that the site is checking referrer, user-agent etc.

Check out the HttpFox and Tamper Data firefox plugins, they let you see/manupulate data going to/from any server. Make sure you note any redirects and cookies coming from the server.

share|improve this answer

If the site is saving the session with cookies you can log on using a browser, then have wget use the appropriate cookie to continue accessing the site. However, if the site uses other means, such as PHP $_SESSION variables you're out of luck.

If this is a site to which you have legitimate access surely you can configure other access means. On the other hand, if you don't have legitimate access and are merely trying to leach someone else's site, which is the impression I get from your question, I hope you fail.

share|improve this answer
    
Re PHP session variables, I think that any data in $_SESSION will rely on a session-id stored in a cookie (or optionally in the URL) so that might not be a problem –  grojo Mar 28 '11 at 11:51
    
@grojo, that cookie is in memory, not in a file, so it's kind of hard to pass on to wget. –  John Gardeniers Mar 28 '11 at 11:59
    
the session data is on the server, however the session ID is stored in the cookie, and AFAIK wget will save it to disk if told to. One possible tip is to GET the page with the form first to aquire/initialize any session data, before attempting to POST login data to the form URL. –  grojo Mar 28 '11 at 12:10

Try this before the URL:

wget username:password@http://me.com/me.html

share|improve this answer
2  
Looks like the site is password protected using a form, not basic auth. –  Kev Mar 28 '11 at 9:27
1  
FYI: The form for basic auth is username:password@example.com/me.html –  Sean Reifschneider Mar 28 '11 at 9:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.