Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We've got a server setup at work which we'd like to use to exchange large files with known clients easily. We're looking into software to facilitate this, but somewhow typing "large file hosting" into Google gives questionable results.. ;)

We've come up with the following requirements, and I hope any of you can points us in the direction of a solution that offers this functionality, or is malleable to our needs.

  • Synchronization / revision management is of no concern, it's mostly single large (up to 1+ GB) file uploads & downloads we'll need.

  • We'd like to make the downloads expire & be removed after a certain number of days / downloads (say 5 each), to limit the amount of cleanup we'd have to do.

  • The data files exchanged sometimes hold confidential information, so the URLs generated should be random and not publicly visible.

  • Our users are of the less technically savvy variety, so a simple webform would be best over a desktop client (because we also have to support a mix of operating systems).

  • As for use of the system we'd either like to send out generated random URLs for them to upload their files, or have an easy way manage & expire users.

  • Works on a linux (Ubuntu) server (so nothing .Net-related please)

Does anyone know of software that fits the above criteria? We've already seen a few instances of this within the scientific community, but nothing we could use directly..

Best regards, Tim


Copied from the comments below:

My problem with ftp is the large amount of overhead in creating users, settings rights, deploying clients, move files and remembering to afterwards delete users, folders, rights etc..

What we're envisioning is more single-use tokens for short-running projects than long-running returning clients with appropriate rights.

share|improve this question

migrated from superuser.com May 6 '10 at 8:42

This question came from our site for computer enthusiasts and power users.

add comment

7 Answers

up vote 2 down vote accepted

This sounds like a simple PHP script might be able to handle it. Though I don't know about timeouts etc. if it comes to that large files.

The script could work like the following:

  • If no login is provided, show login form
  • Verify this login against a database
  • Show a simple file chooser
  • Upload the file to a random generated folder/filename and tell the user about that name

The following additional things would be needed:

  • Administration of the Login-Database
  • A cronjob which deletes old files
share|improve this answer
    
We chose this solution some time ago after we came across a (unfortunately not open source) existing solution.. –  Tim Oct 29 '10 at 7:34
add comment

The data files exchanged sometimes hold confidential information, so the URLs generated should be random and not publicly visible.

This is very bad. If the data is really confidential, you can't trust just the URL for enforcing privacy.

What about ftp?

It supports authentication, you can run scripts to delete old data, it's available on every platform, and your only problem would be to choose a suitable client to tell your users to use.

share|improve this answer
    
My problem with ftp is the large amount of overhead in creating users, settings rights, deploying clients, move files and remembering to afterwards delete users, folders, rights etc.. What we're envisioning is more single-use tokens for short-running projects than long-running returning clients with appropriate rights. –  Tim May 6 '10 at 9:02
    
I understand. Makes sense, and I don't know the answer, sorry. Still, don't use URLs as a substitute for a real security method. –  Lohoris May 6 '10 at 9:15
    
You're right on this, but combined with a limited amount of downloads & days (5 and 5), the risk is already somewhat reduced. –  Tim May 6 '10 at 9:20
    
How many users do you anticipate using this? Unless the number is really very large, then the overhead shouldn't be that high - you create a folder, you create a group, you give the group appropriate access rights to the folder. Then for each user, you create an account, send them the login details, and assign that user to the group. Use the web browser as client - zero distribution challenge. It is a tried and tested model and can be automated to an extent. You think the overhead is high, but if you try and roll your own system, I am sure you will find you spend more time and energy. –  dunxd May 6 '10 at 9:28
add comment

FTP would be a good choice as on the server side you could set up scripts to manage your files as required. Your clients would be able to access the files using most web browsers or even Windows Explorer. You can password protect your ftp folders, which gives some security, however, the file transfers would be in the clear. Also, many corporate environments block FTP. But this would be the simplest and most flexible route.

If you require secure transfer of your files, then WebDav gives a similar end user experience, but you can securely transfer files over HTTPS, and http and https are less likely to be blocked by your clients. WebDav runs on top of most web servers (Apache and IIS both have plugins available), and since it really just gives access to your file system, then you can use any scripts or operating system functions to manage your files automatically.

share|improve this answer
add comment

Here a very basic PHP solution that should be expanded upon to do what you actually want:

<form enctype="multipart/form-data" method="POST">
File: <input name="file" type="file">
<input type="submit" value="Upload">
</form> 
<?php
    $directory = "/var/www/secure/files/";
    $url_prefix = "http://www.mywebsite.com/secure/files/";

    if(array_key_exists("file", $_FILES)) {
        if($_FILES["file"]["error"] > 0) {
            echo "Error occurred during upload: " . $_FILES["file"]["error"];
        }
        else {
            # Randomly generate a filename with the correct extension
            $filename = $_FILES["file"]["name"];
            $ext = split("[/\\.]", strtolower($filename)); 
            $ext = $exts[count($ext)-1]; 
            $target = $directory . rand() . "." . $ext;

            # Move the file to the desired directory and rename using the generated filename
            if(move_uploaded_file($_FILES["file"]["tmp_name"], $target)) {
                echo "File upload successful: " . $target;
            } 
            else {
                echo "Error occurred while storing file.";
            }
        }

        $dirHandler = opendir($directory);

        // Read the directory
        while($entry = readdir($dirHandler)) {
            $dir[] = $entry;
        }

        closedir($dirHandler);
        sort($dir);

        echo "<table>\n";
        echo "<tr><th>Filename</th><th>Filetype</th><th>Filesize</th></tr>\n";
        $count = count($dir);
        for($i = 0; $i < $count; $i++) {
                if (substr("$dirArray[$i]", 0, 1) != "."){ // don't list hidden files
                echo "<tr><td><a href=\"" . $url_prefix . $dir[$i] . "\">" . $dir[$i] . "</a></td>";
                echo "<td>" . filetype($dir[$i]) . "</td>";
                echo "<td>" . filesize($dir[$i]) . "</td></tr>\n";
            }
        }
        echo "</table>";
    }
?>
share|improve this answer
    
"What is your server running?" > All of the above. I know there are similar scripts, but it's hard to find one that fits among thousands of scripts each with their own quirks and limitations. –  Tim May 6 '10 at 8:58
    
@Tim: is the above solution similar to what you are looking for? An uploaded file is renamed and moved to the secure directory and a list of all the secure directory files is shown. File names are just random numbers so they are not representative of the file purpose. Database access or a different file naming scheme would fix this. This script should not be used without added security but adding appropriate security to this should not be too difficult. –  Trey Hunner May 6 '10 at 20:03
    
Thanks for the effort thusfar, a colleague of mine is working with something similar in PHP, so I'll pss this along. I am worried however (as pointed out by others here) about complications when uploading very large (gigabytes) files... –  Tim May 6 '10 at 21:32
    
Thanks for this code snippet. I've modified my copy a little to use random directories rather than random filenames. +1 for a nice easy (partial) solution. –  John Gardeniers May 6 '10 at 22:26
add comment

This is an old question but the class of application that the OP is looking for is called "Managed File Transfer". There are lots of vendors in this space. All of these run on Linux: CrushFTP, JSCAPE, RhinoSoft (CentOS/RHEL 6.x only), GoAnywhere. There are many more to choose from...

share|improve this answer
add comment

Have you looked at services such as drop.io or dropbox.com? drop.io even provides an API with help libraries for Javascript, PHP, Ruby and .Net

share|improve this answer
    
The data is sometimes confidential, so we anticipate our clients are not going to like it if we redirect them to a site like dropbox or drop.io.. –  Tim May 6 '10 at 9:18
add comment

Any large file upload to the web via a broswer is problematic. You have much more chance to have an error while uploading a 1Go file than during the upload of a 100kb image. And browsers are not know for their ability to handle upload retries well.

So if you want to put in place a robust, easy to use solution, I definitely think you will need a thick client on the customer PC. That could be a Java applet, an FTP client, a general purpose downloader ...

A general purpose FTP client could be quite an easy solution. User management could probably be easily automated around your FTP server with a few scripts ...

Sorry, I dont have a proper solution, just a few ideas ...

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.