3

I have a link to a file that I would like to download from the shell. Unfortunately, the URL redirects to a software license agreement page which requires clicking an accept button. This is fine when I'm in a standard browser but when I'm in the shell wget will just download the html page and not the actual file. What options are available to download the file to the shell when the terms must first be accepted?

Specifically, I'm looking at the following LSI download page:

http://www.lsi.com/Pages/user/eula.aspx?file=http%3a%2f%2fwww.lsi.com%2fdownloads%2fPublic%2fSATA%2fSATA%2520Common%2520Files%2f3DM2_CLI-Linux_10.2.1_9.5.4.zip&Source=http%3a%2f%2fwww.lsi.com%2fdownloads

I did view the source code on the page and found a link to the following location but it also pulls up the eula page when using wget.

http://www.lsi.com/downloads/Public/SATA/SATA%20Common%20Files/3DM2_CLI-Linux_10.2.1_9.5.4.zip

So far the only options I have come up with is to:

  1. Download it to a different machine that has a browser and then upload it to the server that only has a shell. Slow and painful.
  2. Install something like Lynx. I'm not sure if Lynx handles downloads well and don't love the idea of installing extra software on a production server.
  3. Continue analyzing the EULA code to determine how it is preventing my download (assuming it is javascript and not server side).

There has got to be an easier way?

Improve this question
1
  • 1
    Updated my answer for this particular file - basically, LSI needs some new web designers, because their current ones are broken.
    – Grant
    Mar 7, 2013 at 22:18

4 Answers 4

Reset to default
3

I typically just use lynx for this. Take a moment and familiarize yourself with the navigation keys. The download process is straightforward and allows you to select the output location...

However, in this case, I tried to run the download through lynx, but had no luck.

So just take the resulting filename, 3DM2_CLI-Linux_10.2.1_9.5.4.zip, and search for an alternate source that will allow you to wget. Easiest approach.

Improve this answer
3
  • I like the simple idea of finding another source. However, doesn't this also bring some risk as we don't know if it is a reputable source or not? By the way thanks for the lynx video.
    – Brett
    Mar 7, 2013 at 22:37
  • 1
    Yes, there's some risk, but for the software you're seeking, you can compare the sizes/checksums of a few different sources... Or download from LSI to your computer or another trusted source and copy to the target system. It wasn't a large file.
    – ewwhite
    Mar 7, 2013 at 22:56
  • I ended up using my hack but feel that your solution will work best for most websites that have this issue. Thanks for the help!
    – Brett
    Mar 11, 2013 at 17:43
3

For most websites, you can:

  • Start the download in firefox on your PC.
  • While its downloading right click and copy the download url. Cancel the download.
  • Use that url with wget.

That works on 95% of websites that use redirects like that.

However, this site has some strange behaviour:

  • Tried it on my PC and it instantly downloaded from that URL - having just accepted their EULA from my phone on the same wireless network.
  • Downloading in wget from another PC on the same network worked fine.
  • Downloading in wget from a remote server got me the EULA page.

Since I did that all from different devices, it's not cookies or post parameters. The only thing they share is they are behind the same NAT.

So I can only conclude that they are keeping track of IP addresses that have accepted the EULA and only showing the EULA page to ones that haven't. To download it from wget - either accept the EULA on another computer from the same public IP address, or call LSI and get their web designers fired!

Improve this answer
2
  • This does not work in this case.
    – ewwhite
    Mar 7, 2013 at 22:05
  • Then they are either using cookies or post parameters...will check it out when I am at a pc.
    – Grant
    Mar 7, 2013 at 22:06
2

Hack: This is a specific solution for LSI. Just add the source=1 to the url. So the link

http://www.lsi.com/downloads/Public/SATA/SATA%20Common%20Files/3DM2_CLI-Linux_10.2.1_9.5.4.zip

would change to:

http://www.lsi.com/downloads/Public/SATA/SATA%20Common%20Files/3DM2_CLI-Linux_10.2.1_9.5.4.zip?source=1

The wget command is then:

wget http://www.lsi.com/downloads/Public/SATA/SATA%20Common%20Files/3DM2_CLI-Linux_10.2.1_9.5.4.zip?source=1 -O 3DM2_CLI-Linux_10.2.1_9.5.4.zip

LSI WHY!?

Improve this answer
1
  • Nice one, works with curl too. For example curl -O http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/MR_LINUX_DRIVER-06.506.00.00.tgz?source=1
    – George Tasioulis
    Apr 6, 2013 at 10:58
0

http://www.lsi.com/downloads/Public/SATA/SATA%20Common%20Files/3DM2_CLI-Linux_10.2.1_9.5.4.zip

That link goes to it from a browser without EULA, but it didnt work for wget.. weird. Can you just download it to a desktop and then sftp it to your linux box?

Improve this answer
1
  • Does not work from a text-based browser (lynx) or wget.
    – ewwhite
    Mar 7, 2013 at 22:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .