3

I'm looking for a server software product that:

  • Allows my users to share large files with:

    • The general public
    • securely to 1 or more people (notification via email, optionally with a token that gives them x period of time to download)
  • Allows anyone in the general public to share files with my users. Perhaps by invitation.

  • Has to be user friendly enough to allow my users to use this with out having to bug me as the admin.

  • It needs to be a system that we can install on our own server (we don't want shared data sitting on anyone else's server)

  • A web based solution.

  • Using some kind or secure comms channel would be good too, eg, ssh

  • Files to share could be over 1 GB.

I found the question below. WebDav does not sound user friendly enough: Recommendations for a secure (and simple) dropbox system

I've done a lot of searching, but I can't get the search terms right. There are too many services that provide this, but I want something we can install on our own server.

A last resort would be to roll my own.

Any ideas appreciated.

  • Glen

EDIT

Sorry Tom and Jeff but Glen specifically says that he's looking for a 'product' so given that I specialise in this field thought that my expertise in this area may have been of use to him.

I don't see how him writing services is going to be easy for him to maintain going forward (large IT admin overhead) or simple for his users and the general public to work with.

1
  • Product and service recommendations, including product recon, are off topic per the FAQ.
    – sysadmin1138
    Mar 19, 2012 at 2:27

6 Answers 6

4

If you are okay with using Amazon S3 as your base setup, you could try writing your own service for it, which allows creating temporary keys to download a file. Or, you could use Dropbox (built on top of Amazon S3) which lets you share files with friends, but they need to have accounts in Dropbox.

I would suggest you this. Keep it on dropbox/S3 after encrypting the file with a 1024 encryption key and then share the decrypt key with your users you want to share with.

2
  • +1 for Amazon S3. It's such good value for money, easy API with plenty of supporting libraries. Feb 8, 2010 at 10:26
  • Ideally, we want to keep all our data on our own servers. So I don't think Amazon S3 is an option for us. Feb 12, 2010 at 5:59
2

A few years back I implemented a system that functioned in a very similar way to your description. It was a file sharing portal (centered around some user-generated video game content), and served up thousands of files ranging from 5 to 500Mb.

I couldn't find an off-the-shelf solution, and had to roll my own. All the other folks that I knew in the same situation as me (large user generated uploads submitted then re-distributed on the web) had rolled their own solutions as well. I know this isn't what you want to hear, and I hope you find something, but the hive mind doesn't seem to have located a solution for you yet.


If you do go with the home-rolled solution, you may come across some of the same issues I did:

  • Upload of files tops out at 50Mb without tweaking the web server settings. Anything over 100Mb is very unreliable to upload over HTTP, mainly because most user upload bandwidths are limited, and uploading often cripples all other web activity.

  • Integrity of files once uploaded is very flaky. Over just HTTP, upload of big .zip files ended up with mismatched hashes in about 1 in 10 uploads.

  • Management oversight of the files being traded became very important, to the extent that we had to review all files before posting back to the download pool. This would be less of an issue in a corporate environment, but you still need to be very wary of allowing your customers to send you any large, potentially executable content. Especially hundreds of Mb of it, with the destination being a web server.

1

Very large files are not dependably transferred via the "web" (ie. browsers). End of story. Why do you think video sharing sites build special apps that install on your computer that assist in the upload? What happens when you are uploading a 1Gb file and it fails at 50%? You try again and again until it succeeds. Lame.

FTP is a good solution. Anyone wanting to transfer large files should learn how to use it. You can setup accounts for all your users and an anonymous account for downloads. Webdav may work well too. When all you have is a hammer, everything looks like a nail. When all you know is a browser, all you can use is the web.

Now that my lecture is over(pardon me), take a look at: http://jupload.sourceforge.net/index.html

Perhaps you could integrate that with some kind of PHP web based file manager. That should keep the interwebers happy.

0

I guess the question is: "How Big?"

I recently set up an installation of PHPfileNavigator for this purpose. Since it uses PHP for uploading, you'll need to modify your php.ini (or .htaccess) to specify the memory_limit size.

I have had good success uploading files up to 300MB in size.

0

Edit: A great solution is: http://justattach.com/

I also looked at: http://www.ipswitchft.com/Business/Products/AdHocTransferModule/

WS_FTP is an FTP server for Windows. It includes a web transfer module for sharing files via a web browser. But better than that, they have just released an "Ad Hoc Transfer" module. Which seems to almost fulfil my requirements (my users can easily share files with 3rd parties - on an ad hoc basis). The only thing it does not seem to do, that I want, is to make it easy for third parties (customers) to share large files with my users. What I'm thinking here is that my user initiates the transfer (sending an email with a link to allow the 3rd party to upload to).

0

http://www.filecatalyst.com/

You can use the above - you can setup a dedicated windows box for this and create your folder structure for different clients, share those folders on your local lan, any file that goes in those folders - starts to get transferred automatically.

You can also provide your clients with a web interface url or a file catalyst client which connects to your server and downloads these large files.

Riverbed is another solution (google it).

Hope this helps

Not the answer you're looking for? Browse other questions tagged .