Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm building a web application and I've decided to host it on Windows Azure.

My application has three components:

  • A front-end created with Play Framework (machine A)
  • A back-end core written in Java that executes my business logic (machine B)
  • A database (machine C)

Since I have a free Bizspark account I created 3 free Azure users: for each one I created a medium Ubuntu 12.04 LTS machine and I installed one component on each machine.

Each of the component is running on its own machine to boost performance. The connections I'm interested in is the connection from machines A and B to machine C.

I successfully connect to the database machines, I created all of the needed endpoints and the corrects access rights have been given

BUT

the problems I'm getting are the following:

SEVERE: FATAL: DataSourcePool [mysql] is down!!!
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 1,197,486 milliseconds ago.
Caused by: java.net.SocketException: Connection timed out

Or:

WARNING: DataSourcePool closing leaked connection?
Nov 27, 2013 8:55:46 PM com.avaje.ebeaninternal.server.lib.sql.DataSourcePool validateConnection
WARNING: heartbeatsql test failed on connection[mysql.31]
Nov 27, 2013 8:55:46 PM com.avaje.ebeaninternal.server.lib.sql.PooledConnection  closeConnectionFully
INFO: Closing Connection[mysql.31] psReuse[19] psCreate[19] psSize[19]

Or:

javax.persistence.PersistenceException: java.sql.SQLException: Unsuccessfully waited
[1000] millis for a connection to be returned. No connections are free. You need to
Increase the max connections of [100] or look for a connection pool leak using
datasource.xxx.capturestacktrace=true

Since my application on another web server was running fine I think there's something wrong in the configuration of my network. I think that the 1000ms run out before eBean can get a connection from the pool. I tried to execute some basic connectivity tests like Ping or Traceroute from my core machine to the db machine but Azure blocks ICMP messages. I then tried TCPPING and the output is the following:

tcpping xxx.cloudapp.net 3306
TCP Ping 1:0,808ms
TCP Ping 2:0,934ms
TCP Ping 3:1,036ms
TCP Ping 4:0,860ms
TCP Ping 5:0,927ms
TCP Ping 6:0,905ms
TCP Ping 7:0,949ms
TCP Ping 8:1,079ms
TCP Ping 9:1,274ms
TCP Ping 10:0,983ms

TCPTraceroute is not working, it only shows asterisks...

I don't know if these results are believable or not but I don't know how Azure works. I think that 1 second on average is an enormity.

I then tried to connect remotely to the database machine: I installed a MySQL 5.5 client on the core machine and connected like this:

mysql -u [username] -h [db machine hostname].cloudapp.net --port=3306 -p[password]

All's fine and it seems also pretty fast, I can't see any sort of lag from the console view.

Some details about the machines:

  • all are in the same region (West Europe)

  • they are created from different accounts (like email1@hotmail.com owns db machine, email2@hotmail.com owns core machine, etc.)

  • they all run the same Ubuntu version

Can you please tell me what can I do to decrease this delay?

share|improve this question
    
Why do you have different users owning each virtual machine? –  Michael Hampton Nov 28 '13 at 17:35
    
Because I have a credit limit on each account. 115€ per month per account, and I can create up to 8 free accounts, so I put a medium sized machine in three accounts –  Andrea Nov 28 '13 at 17:47
add comment

1 Answer

Well, apart from the fact that you are exploiting the platform benefits, here are my thoughts.

I would say the best way to interconnect VMs is to utilize the Azure Virtual Network. However, as of today (Dec. 5, 2013) you cannot span a Virtual Network across multiple accounts. Your only option is to access the machines via the VIP (public IP Address), which you do now. I would say that what you observe is about 1 ms (millisecond) and not 1 s (second) as you second! And I would say that 1 ms is somehow neglectable. Also, you have to rewind your Match classes. The error message you quote:

last packet successfully received from the server was 1,197,486 milliseconds ago

Refers to 1,197,486 milliseconds. There are just 1000 milliseconds in 1 second. This means that 1,197,486 milliseconds are approx. 1,198 seconds, or roughly 20 minutes. Well, 20 minutes of timeout is definitely not related to network latency.

If you want the things right, or correct - just put all within same account, single Virtual Network and use internal IP Addresses for communication - this way you will skip the Azure Load Balancer, which you cannot avoid when doing cross-deployment communication over public IP Addresses.

More or less, again as of today, every Azure Deployment (including your VMs) runs under the picture described in this blog post. Just consider that you have 3 such diagrams, every single one of them with just one VM, instead of 5 as shown on the diagram.

share|improve this answer
    
I don't know about MySQL standards, but in my math classes I've learned to differentiate thousands separator from decimal separator. I see two commas and I interpret 1 million 197 thousands 4 hundreds 86 milliseconds. I would have read 1 thousand 1 hundred 97 milliseconds if it was printed like this: 1.197,486 ms... Since the two separators are the same it's obvious to assume they are the same kind of separator, so in this case thousands separator. –  Andrea Dec 9 '13 at 14:27
    
Where is the confusion? Indeed you have ~ 1M milliseconds, which is ~1k seconds, which is roughly 20 minutes. I would time-out if there was 20 minutes of nothing... Not to mention that if you go through the public IP Address (and not direct via VPN) I don't expect a connection to be kept at Windows Azure Load Balancer for even 5 minutes without any activity. Now the globalization comes into place. For the tcpping, I would have accepted "," for thausand separator if there wasn't 0,926ms. I have never seen result like this, it it wasn't a decimal. –  astaykov Dec 9 '13 at 15:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.