Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am currently dealing with an old website where there are political issues preventing the migration of the database or re-designing the site.

Since I can't get access to the database I basically I want to write some Javascript to download all the data from this website by just parsing the returned HTML. This would be easy if each page of data was an actual URL but it's not. Each page is created dynamically via ASP so I need to somehow parse the returned page format and save the data.

I also need to deal with inserting session cookies into the script so that the website thinks I am a valid logged in user.

Any ideas on how one would go about making this task easier ?

share|improve this question
add comment

2 Answers

You can use a program for download sites, there are many of them, still, if is using a database it will lose all funcionality and you wont have all the feautures in it. All the javascript could be downloading due if is client-side you can download with a program, but if has something specific server-side with asp I doubt u can.

share|improve this answer
add comment

Have a look at phantomjs : It may do most of the webscraping job for you.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.