Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We provide Email Marketing service through our online App. We have about 30 customers. And each one has it's own mailling list (5k to 20k emails each).

What we really want is to distribute email's delivery between 2 or more servers. I was wondering What kind of aproach/solutions MailChimp , Constant Contact uses to provide a great service ? use many servers ? many IPs ?

Our spam policy suspends ANY user/customer that gets 10% bounced .

share|improve this question

migrated from stackoverflow.com Mar 31 '10 at 3:13

This question came from our site for professional and enthusiast programmers.

3 Answers 3

We have 1 server at The Planet

Dual Xeon Quad Core, com 12 GB of RAM

Processor #1 Vendor: GenuineIntel
Processor #1 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #1 speed: 1995.080 MHz
Processor #1 cache size: 6144 KB

Processor #2 Vendor: GenuineIntel
Processor #2 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #2 speed: 1995.080 MHz
Processor #2 cache size: 6144 KB

Processor #3 Vendor: GenuineIntel
Processor #3 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #3 speed: 1995.080 MHz
Processor #3 cache size: 6144 KB

Processor #4 Vendor: GenuineIntel
Processor #4 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #4 speed: 1995.080 MHz
Processor #4 cache size: 6144 KB

Processor #5 Vendor: GenuineIntel
Processor #5 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #5 speed: 1995.080 MHz
Processor #5 cache size: 6144 KB

Processor #6 Vendor: GenuineIntel
Processor #6 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #6 speed: 1995.080 MHz
Processor #6 cache size: 6144 KB

Processor #7 Vendor: GenuineIntel
Processor #7 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #7 speed: 1995.080 MHz
Processor #7 cache size: 6144 KB

Processor #8 Vendor: GenuineIntel
Processor #8 Name: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz
Processor #8 speed: 1995.080 MHz
Processor #8 cache size: 6144 KB

When all our customers are delivering campaigns at the same time (it happens before commemorative dates) we are getting too many rejected emails due the higher delivery flow. We rotate our mail outgoing IPs using 45 IPs. And control/limit the flow to a host ( eg: @hotmail , @gmail) from the each IP.

Our approach is

  1. Our Application delivers 300 emails to @hotmail.com at 2pm
  2. App blocks deliveries to @hotmail.com and start delivering to diferent hosts until it raise the limit by host
  3. App rotates the Mail Server outgoing IP ( using 45 diferents IPs ) every 10 minutes

I 'm affraid , as the business grows and we get more customers , the DELIVERY FLOW gets higher and this approach FAILS by getting rejected by destination hosts .

share|improve this answer

Typically a message queue is used for something like this. Commands to send e-mail to particular recipients or groups of recipients are queued up on a producer and then any number of consumers can pull off the next command in the queue and process.

There are many options for how to implement but it will depend on your specific environment.

That said, the mail volume you're sending is not that high for a single server.

share|improve this answer

What exactly IS the problem here?

Your volume is not high enough to justify splitting - I mean, even if all your 30 customer s send out 20.000 emails at once, that is not THAT much - enough for ONE powerfull server to handle (and I don't mean super powerfull - just get some discs and form a raid for the mail spool).

You could possibly go with multiple sending servers for queueing. A block of IP addresses for all mail servers. Either using somehting like MS Exchange for sending (exchange is great in automatically maintaining server farms), or using some mechanism to cluster emails between servers (so that those to the same domain end up on the same server - simply because the server can then send multiple emails in one TCP connect).

Multiple ISP's sounds like a non-issue. Do not forget - we talk about way too much traffic for a cheap home ADSL line. Once you get business grade, your servers move into a proper data center. Uptime is very high then - for a proper one (carefull, some really seem cheap). Even if thigns go down a day in 2-3 years, you have to be REALLY big or clustering to make sense. And even then things can go really bad (as in: Wikipedia also was down for some time recently).

The main issue willb e you need dedicated servers built on specs to handle the mail flow. Network traffic is not that much. But the mail spool will be reall hard on your discs. If you really get spikes of 600.000 emails, a RAID 10 of 4-8 high speed discs (or: A raid 5 of SSD's) sounds like the best bet. Like all database transactional systems your limit will be the IO. I would go with a custom formatted file system with non-standard node size (64k), allowing every email to be read/written in on node pretty much per guarantee. Note that this is based on SPIKE usage - if the emails dont get all send at once, you can get away with a lot less.

Next step would be a cluster of servers.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.