2

I'm using Amazon EC2 with Elastic Load Balancing. On my web-based application, end-users connect to app.myapp.com get a 504 gateway timeout error when performing certain operations like converting a document to PDF. Otherwise, the entire application is working normally.

However, there is NO ERROR when connecting to one of the servers directly via IP address and performing the same action. Never.

On my registrar's DNS entries I had already made a CNAME record which points app.myapp.com to my Elastic Load Balancer DNS name, and then my server instances sit behind that. Note: the exact problem still occurs even when I stop using the load balancer and point the DNS settings so that app.myapp.com goes directly to the IP address of one of my servers.

The bottom line is that this operation, though no URLs are hard-coded, times out when you've connected to the application via hostname >> but never happens when connecting via IP address. Over the course of months, I've tried just about every troubleshooting step I can think of. Willing to try anything.

Summary: what could cause a 504 gateway timeout when making requests with a hostname but not an IP?

MY RELEVANT VHOSTS ENTRY:

<VirtualHost *:80>
    ServerAdmin webmaster@localhost

    DocumentRoot /var/www/application/public
    <Directory /var/www/application/public>
            Options Indexes FollowSymLinks MultiViews
            AllowOverride None
            Order allow,deny
            Allow from all
            RewriteEngine On
            RewriteCond %{REQUEST_FILENAME} -s [OR]
            RewriteCond %{REQUEST_FILENAME} -l [OR]
            RewriteCond %{REQUEST_FILENAME} -d
            RewriteRule ^.*$ - [NC,L]
            RewriteRule ^.*$ index.php [NC,L]
    </Directory>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
            AllowOverride None
            Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
            Order allow,deny
            Allow from all
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, cri$
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined

Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order deny,allow
    Deny from all
    Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>

  • Does the vhost entry for the DNS name which this affects, match exactly the configuration which would be served via the default server (which is what is served when accessing via IP)? – BE77Y Dec 19 '14 at 11:14
  • Would it be safe to assume that the 504 error occurs pretty much exactly the same amount of time into a long-running request, probably 60 seconds? – Michael - sqlbot Dec 19 '14 at 12:51
  • @BE77Y, I think so, because there's only one active VirtualHost entry and it's for port 80. Connections via IP or Hostname should both use this. Note: I'm somewhat of a newbie with Apache. – user3035649 Dec 22 '14 at 3:48
  • @Michael-sqlbot yes that's correct -- usually around 1 minute. I've toyed with the MaxKeepAliveRequests and MaxAliveTimeout to no avail. When the PDF generation works (connecting via IP), the operation always takes 5-10 seconds. Willing to try any suggestions you have. Thanks a lot. – user3035649 Dec 22 '14 at 3:52
  • Lacking the lastest info, about the 5-10 second typical response time, I assumed you were hitting the idle timeout on the ELB, which is now user-definable... aws.amazon.com/blogs/aws/elb-idle-timeout-control ...but defaults to 60 seconds. – Michael - sqlbot Dec 22 '14 at 4:21
2

Add a ServerName attribute in your VirtualHost config with the hostname you want to use.

  • I just tried this to no avail. Also for giggles I tried the LB hostname, and also tried the elastic ip address associated with my instance. Still scratching my head. – user3035649 Dec 23 '14 at 11:30
0

Fixed it. There were 2 core problems working in tandem which caused this....

Core problem 1: When looking at Ubuntu's /etc/hosts file, though I had no memory of ever editing it, the problem became apparent. The third line was routing to a non-existent server and I really wish I knew why. This third line was removed:

127.0.0.1 localhost
127.0.1.1 ubuntu-server
42.139.126.191 app.myapp.com

Core problem 2: The problem with the hosts file above shouldn't even matter since all links are relative. But for some reason, even though the PDF generating tool uses only relative links, there's a line of code where you have to specify the domain, and that tipped me off that somewhere within the code it was doing rewrites for the full URL which the hosts file said existed at 42.139.126.191 even though in reality nothing existed at that IP, and hence the '504 gateway time-out'.

One more thing I decided to do: Instead of just removing the problematic third line, a new third line (see below) was added because we decided there was no reason to route traffic out to the DNS server only for that DNS server to point back to the same server that the request was sent from (an international round-trip, by the way). The registrar having the DNS authority -- and the host being in different country really makes no sense at all -- so avoid this if you can.

So I added a different third line to circumvent this (but I probably should dump my old registrar and use Amazon's Route53 anyway):

127.0.0.1 localhost
127.0.1.1 ubuntu-server
127.0.0.1 app.myapp.com    

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.