Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

High latency on domain with CDN

I have problems with latency on my server, these are the tests I made with blitz.io

Latency album

http://imgur.com/a/bCmwo

I would uploaded images here but it requires 10+ rep, and can't post multiple links so I uploaded pictures in album on imgur.

My domain is hosted with Cloudflare CDN. I am hosting on Cloud SSD based hosting, DigitalOcean.

Latency on server was ~40ms when there was no traffic.

Latency on server with CDN is now ~270ms

Latency on server with no CDN is ~300ms

I was previously hosting on Windows Azure,

Latency with no traffic was ~70ms

Latency with traffic was ~210ms

Still even though latency is higher on DigitalOcean my content is loading much faster then on Windows Azure servers.

Old Azure server, album

imgur.com/a/fpHK9

I can't post more then 2 links, c/p in browser

No firewall configured on DigitalOcean Server, was default Azure FW on Microsoft Azure servers, configuration with endpoints.


Content

Payload size

images ~156 KB

CSS ~12 KB

logo ~20 KB

random content loaded on each pageviews 4KB

total random content size 116 KB thats gets loaded

total ~300KB, all hosted over CDN. Not all get loaded at once.

Cloudflare stats, album

imgur.com/a/qxnbo

I can't post more then 2 links, c/p in browser

Network analyze with Firefox, album

http://imgur.com/a/XLZNc

Content is loaded after page has finished loading.


Memory

Server is running on lighttpd.

Memory

XXX@XXX:~# free -h
             total       used       free     shared    buffers     cached
Mem:          995M       835M       159M         0B       150M       471M
-/+ buffers/cache:       214M       780M
Swap:         1.0G        26M       997M

Top

top - 14:29:13 up 21 days,  1:00,  1 user,  load average: 0.00, 0.04, 0.05
Tasks:  95 total,   1 running,  94 sleeping,   0 stopped,   0 zombie
%Cpu(s):  3.0 us,  1.4 sy,  0.0 ni, 94.3 id,  0.7 wa,  0.0 hi,  0.7 si,  0.0 st
KiB Mem:   1019056 total,   856144 used,   162912 free,   153748 buffers
KiB Swap:  1048572 total,    27084 used,  1021488 free,   482660 cached

Droplet is on 1Gbit shared connection.


Connections

When issuing netstat -n I count ~620 TIME_WAIT/ESTABLISHED connections

When issuing netstat -n | grep 'ESTABLISHED' I count ~120 connections


100MB test

100MB download test

XXX@XXX:~# wget speedtest.london.linode.com/100MB-london.bin
--2013-09-05 14:35:08--  speedtest.london.linode.com/100MB-london.bin
Resolving speedtest.london.linode.com (speedtest.london.linode.com)... 176.58.107.39, 2a01:7e00::4b
Connecting to speedtest.london.linode.com (speedtest.london.linode.com)|176.58.107.39|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: ‘100MB-london.bin’

100%[===================================================================================================>] 104,857,600 8.46MB/s   in 11s

2013-09-05 14:35:19 (9.47 MB/s) - ‘100MB-london.bin’ saved [104857600/104857600]

100MB upload test

I created another server and just downloaded from first one same 100MB file

XXX@XXX:~# wget domain_with_no_cdn.com/test/100MB-london.bin
--2013-09-05 12:38:00--  domain_with_no_cdn.com/test/100MB-london.bin
Resolving domain_with_no_cdn.com (domain_with_no_cdn.com)... Server_IP
Connecting to domain_with_no_cdn.com (domain_with_no_cdn.com)|Server_IP|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100MB-london.bin'

100%[=======================================================================================>] 104,857,600 13.4M/s   in 6.1s

2013-09-05 12:38:07 (16.5 MB/s) - `100MB-london.bin' saved [104857600/104857600]

100MB upload test over CDN

XXX@XXX:~# wget domain_over_CDN.com/test/100MB-london.bin
--2013-09-05 12:44:26--  domain_over_CDN.com/test/100MB-london.bin
Resolving domain_over_CDN.com (domain_over_CDN.com)... 108.162.200.169, 141.101.127.168
Connecting to domain_over_CDN.com (domain_over_CDN.com)|108.162.200.169|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100MB-london.bin'

100%[=======================================================================================>] 104,857,600 12.5M/s   in 11s

2013-09-05 12:44:37 (8.96 MB/s) - `100MB-london.bin' saved [104857600/104857600]

Can you tell me what are the possible issues with my setup?

Can you give me some tutorials or links where I can learn how to balance load between servers or what exactly do I need to do to get latency under 100ms.


Day 2, testing

I contacted Cloudflare support and they resolved issue in my country, but not all servers are reporting latency below 100ms.

I posted new pictures in album

imgur.com/a/xajge

I can't post more then 2 links, c/p in browser

As you can see from picture 2, there are still issues with servers in US on domains with CDN. I suppose they are experience problems cause of updates on hardware last few weeks but I would like to know how to lower the latency on my own server bellow 100ms.

I am dividing pixel ping server and data server so hopefully that will lower some of latency down.

If someone has any guidance which can resolve my issues, please any help is appreciated :)

share|improve this question

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.