Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What's the technical limitation preventing us, in the glorious year 2011, from emailing each other 1GB files?

Or is it just the main email platforms dragging their feet?

If I can set my inbox to grab headers only, and then full attachments if I want them, what is the problem?

I feel like email attachment sizes are stuck in 1992...

share|improve this question

migrated from stackoverflow.com Aug 24 '11 at 9:04

This question came from our site for professional and enthusiast programmers.

closed as not constructive by Zoredache, MadHatter, Iain, Ben Pilbrow, Shane Madden Aug 24 '11 at 20:58

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

21  
Attachment sizes stuck in 1992? Puh-leeze. I'd like to see you transfer a 50 MB file in 1992, by any method available, let alone attach it to an e-mail (yes, I do have several such e-mails from this current month in 2011, no, I'm not very happy about it). Hint: the preferred method, in 1992, could have included copying to tape and driving to the destination, or perhaps an all-night dial-up and uucp session. –  Piskvor Aug 24 '11 at 7:56
2  
@Piskvor: Alternatively, a grocery bag full of disks containing multi-volume-spanning arj archives. Kinda unrelated: cs-exhibitions.uni-klu.ac.at/index.php?id=187 –  sum1stolemyname Aug 24 '11 at 12:00
15  
Bandwidth has little or nothing to do with it; if what you have to communicate with someone else is bigger than 20 megabytes, email is not the way to send it. –  Shadur Aug 24 '11 at 12:13
2  
@Shadur: It does,in case of unwanted mail - a link in an e-mail (which the recipient chooses to click or not) takes thousands of bytes at the extreme end; an attached file in an e-mail is, in most cases, downloaded without prompting (I'm aware of IMAP's capabilities in this regard,but most users have this set to "download everything",which would somewhat affect the bandwidth - also used for other purposes besides mail:the usual complaint of the non-IT users before broadband:"Why is my web so slow? Yes,I did send a 10MB e-mail with dancing pigs with 100 people in BCC,how is that relevant?") –  Piskvor Aug 24 '11 at 12:22
4  
@Piskvo "Never underestimate the bandwidth of a truck full of tapes"; as true today as ever: you can get >1TB on one tape.... –  Richard Aug 24 '11 at 18:51

6 Answers 6

The problem is this: e-mail (SMTP/POP3/IMAP/what-have-you) is an ancient, simple protocol originally intended for sending plaintext messages in a trusted network. Using it for sending or receiving large amounts of binary data across today's Internet is a bolted-on hack, completely different from the original use case, and it performs rather miserably in this role.

When you attach a file to the e-mail, it gets base64-encoded, which increases its size by 1/3. Thus, your 1 GB file becomes another 300 MB larger; also, there is no built-in compression to the download protocol, thus no way to speed up the transfer (and in some cases (SMTP for sending,POP3 for receiving), even no way to resume a broken transfer - connection broke at 1.2 GB? Sorry, you need to re-transmit it all again). Moreover, SMTP is a store-and-forward protocol. Guess what? Yup, that 1.3 GB file needs to be copied across multiple servers; cue unbounded happiness from the mail server admins.

This was a problem in the 1990s, when there was no useful alternative (FTP? HTTP/1.0? Puh-leeze); but in the glorious year 2011, with various ways of seamlessly up/downloading data to/from the cloud (e.g. Dropbox, Ubuntu One, Amazon S3, to name the most known), the excuse of "there's no other useful way to do this" is not true any more.

Note also that not everyone is on a 100 Mbit link to the Internet - e.g. mobile and smartphone; not every mail client is capable of downloading only the headers (e.g. POP3 is still in much use), and not every user is willing to download the 20 inevitable "look at this funneh 1 GB video" e-mails per week that will appear (people will send as large files as the system will let them; and yes, there is something like FUP with most ISPs).

TL;DR: while it would be technically possible to do such things as e-mailing a 1GB file, it would also be technically possible to pound in a nail using a screwdriver - it's just not a good way to do it, as there are tools that are more suitable for such tasks.

share|improve this answer
10  
+1 That is one very very good answer :) –  Antoine Benkemoun Aug 24 '11 at 9:26
1  
100Mb link? Unless you're a corporation, noone has a 100Mb link here in Australia. –  Matthew Scharley Aug 24 '11 at 12:30
14  
+1 for the TLDR version :-) –  Doctor Jones Aug 24 '11 at 12:41
1  
@Matthew Scharley: Some cable (optical fiber, actually) companies are starting to offer that around here (with 10MBit upload, I concede), but that's beside the point: the point is that 1) people assume that everyone has at least the same bandwidth as they do (improbable), and 2) resource usage grows to saturate the resources available, including bandwidth. Thus, 1 GB e-mail could look reasonable to send from the POV of a person on a 10-Mbit link, but everyone else would be gnashing their teeth. –  Piskvor Aug 24 '11 at 12:56
2  
My email client would love a 1GB file encoded in base64. –  Prisoner Aug 24 '11 at 15:24

The same but from a slightly different view:

Email is electronic mail. You know mail as this ancient paper thingy in another little paper envelope. You could write a lot of text on it but not very more than 5 or 6 pages. And email is the same but electronic. It is designed for text (plain text like on a typewriter). Then there was a MIME extension where you could send these fancy red-blinking HTML mails.

No one in the world would complain and say "Oh mail stuck in the way it was at 1322 AD. Why can't I send an elephant in a paper envelope?" It is how it is. For this kind of stuff people invented something like a packet or a transport container. This is how to send goods and elephants. And the Internet guys invented FTP (file transfer protocol), got the name?

In the real world there are many alternatives to FTP because FTP is also an ancient protocol with big disadvantages (mostly in security and not in transferring files). But HTTP is not an alternative as it was developed for centralized document storage with meta data. That you can download and upload files is a brutal extension to it.

So use a letter to send text and a packet to send goods; use email to send information and file transport protocols to send files.


Edit:

To stay in the picture I have to add: Even if you convince your local post office to accept elephants in paper envelops (and pay the additional fee) there are more parties involved delivering the elephant. There is the mailman who has to carry it to the next post office and probably he has not the right bag for the elephant to fit in. But maybe he has and wants to deliver it to the next post office which in turn says: "No we don't accept elephants".

What to do then? The good mailman in the real world would carry the elephant back to the first post office -- back to sender afterwards. (In the electronic world this would be a bad mailman because he should have shot the elephant and only deliver the certificate of death back to the sender).

So even if you could convince all links in the chain of post delivery to accept elephants you are faced with the recipient. He could say that he wants the elephant but the letterbox is too small for an elephant to fit in. Leading to a return-to-sender elephant delivery. (Not to mention what happens if the elephant doesn't fit in the sender's letterbox...)

share|improve this answer
16  
Come on! As long as there's the Content-Type: application/x-pachyderm header, HTTP is perfectly capable of transmitting elephants ;) Good points, although my protocol of choice would be rsync - reasonably well available, allows for compression, delta syncs, continued transfer, plus works well with SSH (for auth+encryption). –  Piskvor Aug 24 '11 at 13:01
    
Even a P2P approach is reasonable. It depends on the audience. Multicasting a file through email should ring everybody's alarm bell. And as you said in other words one shouldn't follow this approach: "If you only have a hammer then every problem looks like a nail". –  mailq Aug 24 '11 at 14:46
    
Hmm, yes - for multiple intended recipients, e.g. torrents make a lot of sense; but in my experience, you still need a (FTP?HTTP?) fallback for those users who aren't savvy of all these newfangled transfer protocols. Depends on the audience, as you said. –  Piskvor Aug 24 '11 at 14:51

Having been in a situation with Exchange 2007 where management subscribed to the "no limit on email size" philosophy:

An internal user sent a message to their hotmail address with an .iso of a music CD. Jammed the queue on one transport server while processing the message, lit up back pressure, stopping message submission. The user's outlook then dutifully re-submitted the message to the other transport server that was functioning; back pressure, no message submission.

With both transport servers choking on the message, all outbound email halted for about 90 seconds. Hotmail, of course, rejected the message. There was a size limit in place very soon after.

share|improve this answer

Here is another view:

Since an email is stored in multiple instances along the way, sending a 1 GB file would use up several times that all the way.

It will usually be a copy on your client in "Sent items", and if using IMAP, a copy on the server might show up as well (on your account).

Then the receiving end will keep a copy (the server), as well at the local client on the receiving end. If using IMAP, then it will not be deleted on the server (once again).

That is a total of 4 GB for a single 1GB file. Of course, you can consider it a backup, but there are better options for that as well. Not to mention the slowness that might occur on the server because the users mailboxes grows indefinitely.

And I just realized that if the file is base64 encoded it will be even larger (approx 33% larger I guess).

share|improve this answer

To supplement Piskvor's answer.

Yes, the "main email platforms" are dragging their feet. They do this by using a protocol (SMTP) that is not up to today's standards (in many ways).

In today's world, we could easily design a protocol to replace SMTP that would solve the current attachment issue.

The problem would be getting the world to switch over to it.

share|improve this answer

The problem mentioned are mostly logistic issues with storage and transmission of data - in the modern cloud abstraction, a file no-longer need to be physical - a file-handle abstraction may be used to wrap around various storage methods (e.g. local disk, ftp, http, torrent, youtube, cloud storage, darknet, attachment, mule, distributed fs, excerpts, revisions) - this isn't a new idea, it's just not fully here or in one piece yet. when or if it arrives, your mail attachment would simply be a file pointer that can be used directly (e.g. not a .torrent file nor a link) by video players or whatever software. the actual handling of content download or storage would happen transparently, content may be partially located from multiple sources defined in collaboratively-revisable manifest (e.g. like a .torrent file but universally accepted, and with revisable availability and locality constraints); actual download and storage/caching may often be partial, depending on which part was viewed and if you've even bothered to access the content - so your mother-in-law's huge attachment wouldn't eat up any of your to-house bandwidth if you are not a fan of her emails. For permanence or availability, maybe you'll have a mail client that can filter and revise the storage manifest that then result in relocation of certain unopened attachment from its sources to your cloud storage when its availability dwindles, for example

share|improve this answer
1  
As much as I hate "cloud" terminology, you're description is half true; but there are still transfer requirements (bandwidth), storage even if it's only intermediate, and significant delay could be induced by the lack of presence on a "local" server. Even if the file is never accessed by the recipient, the original sender still has to transmit the whole file "to the cloud", the "cloud" has to hold the whole file (perhaps indefinitely), and the structures to communicate all this would have to be put in place. If we're reinventing the wheel, it could be done better than this. –  Chris S Aug 24 '11 at 20:38
1  
"a file no-longer need to be physical" - hold on while I get rid of my disks, as they're no longer needed for those spiritual files ;) You do have a good point that the storage and transfer can be abstracted away, but they're merely hidden somewhere by the abstraction, not gone. You'd need a really fat pipe to alleviate the file access latency - e.g. start playing a HD video, seek to the middle, twiddle your thumbs for minutes while the requested data is streamed to you (as opposed to mere milliseconds for local storage). And there's also the one-foot-per-nanosecond pesky speed of light. –  Piskvor Aug 25 '11 at 8:56
    
true - this could all get pretty slow if locality or availability isn't defined or implemented well. but user gets to define them and take some responsibility for various trade-offs of speed, bandwidth, availability, so on, using prepackaged policies, filter rules, attributes, tags, inferencing rules. when I send an email with attachment, I don't need to 'upload' them, as the data is simply made available to the receiver, then the data move or replicate itself to disks and/or cloud storage based on behavior of two parties' storage managers' user-configured inference rules. –  Alife Toy Aug 25 '11 at 9:37
    
"user gets to define them and take some responsibility" - You must be a manager. –  Chris S Aug 25 '11 at 15:44
    
not manager but something much worse - a broken futurist –  Alife Toy Aug 25 '11 at 17:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.