Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am currently dealing with an old website where there are political issues preventing the migration of the database or re-designing the site.

Since I can't get access to the database I basically I want to write some Javascript to download all the data from this website by just parsing the returned HTML. This would be easy if each page of data was an actual URL but it's not. Each page is created dynamically via ASP so I need to somehow parse the returned page format and save the data.

I also need to deal with inserting session cookies into the script so that the website thinks I am a valid logged in user.

Any ideas on how one would go about making this task easier ?

share|improve this question

You can use a program for download sites, there are many of them, still, if is using a database it will lose all funcionality and you wont have all the feautures in it. All the javascript could be downloading due if is client-side you can download with a program, but if has something specific server-side with asp I doubt u can.

share|improve this answer

Have a look at phantomjs : It may do most of the webscraping job for you.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.