3

I'm running an Ubuntu 14.04 server with Apache2 v2.4.7.

Recently our site was getting errors from Chrome saying we needed to update our certs. I just completed that earlier this week, updating the SSL Certs and then restarting Apache. A couple of days layer, my client came back and mentioned that certain images and files weren't accessible anymore through their reporting server. The reporting server was using HTTPS to get to the images and files.

My conf file has this, which hasn't changed since I inherited this server:

<VirtualHost *:80>
    ServerName abc.example.com
    DocumentRoot /home/deployer/html/_abc/public
    RewriteEngine On

    RewriteCond %{HTTPS} off
    RewriteCond %{REQUEST_URI} !^/files/
    RewriteCond %{REQUEST_URI} !^/images/checksheets/
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</VirtualHost>

<VirtualHost *:443>
    ServerName abc.example.com
    DocumentRoot /home/deployer/html/_abc/public
    SSLEngine on
    SSLProtocol all -SSLv2 -SSLv3
    SSLCipherSuite          ECD....C3-SHA:!DSS
    SSLHonorCipherOrder     on
    SSLCompression          off
    SSLCertificateFile /etc/ssl/ssl_certificate.crt
    SSLCertificateKeyFile /etc/ssl/abc_staging.key
    SSLCertificateChainFile /etc/ssl/IntermediateCA.crt
    # HSTS (mod_headers is required) (15768000 seconds = 6 months)
    Header always set Strict-Transport-Security "max-age=15768000"
    <Directory /home/deployer/html/_abc/public>
        AllowOverride all
        Options FollowSymLinks
        # -MultiViews
        #   Order allow,deny
        Require all granted
    </Directory>

The only thing I've done on the server has literally been to update the ssl_certificate and IntermediateCAs and restart Apache, so I'm not sure what happened. The last time the SSL Cert was updated was probably about a year ago, and the reporting server's way of grabbing images/files hasn't changed. I also restart Apache fairly frequently and this had never been an issue previously.

On a whim, my client created a new report using http instead of https, and that seemed to work perfectly OK.

My question really is, what could have changed to cause the server to suddenly no longer accept https connections for images/files?

EDIT: These are only GET requests (there are no POST/DELETE actions).

Also, navigating to the image (or file) as https://abc.example.com/images/path/to/file.jpg with HTTPS completely works fine. But when the reporting server uses that same link, a blank page gets displayed.

Apache access logs will show the above link as a hit when navigating directly, but when the reporting server uses it, and I navigate to the report, Apache access logs show nothing at all.

6
  • Q: Can you reproduce the problem yourself, or is only the client having the problem? Does it affect everything (all GET and POST requests), or only specific files or URLs on your server?
    – paulsm4
    Oct 11, 2018 at 5:13
  • @paulsm4 anyone/everyone can see the issue. the reporting server (separate server) accesses the /images and /files directories for the reports that need to be generated as GETs only. So when the reporting server tries to display the results as a PDF or doc, the images they were supposed to pull don't show up. Apache access logs show nothing as well. For the new report that the client created w/ http, apache access logs show a successful 200 status and displays the images/files.
    – risa_risa
    Oct 12, 2018 at 13:58
  • Q: Do I understand that there are THREE hosts involved? 1) The client (browser), 2) Ubuntu/Apache (Web server) ... and 3) this "reporting server"? Q: How does Apache "communicate" with the reporting server? A file share (e.g. NFS or Samba)? "Something else" (e.g. http/https)?
    – paulsm4
    Oct 12, 2018 at 15:49
  • Apache simply serves up the static files, so the reporting server (Clear Reports) creates a PDF that has links to the static images/files, and then displays the PDF in the browser for a user to view. User then sees no images because apache never saw the requests for them. But if you take the same URL that uses https that the reporting server uses and plop them into the browser, it totally works. And before I updated the SSL certs from the old to the new, the reporting server was working and the PDFs were rendering the images totally fine.
    – risa_risa
    Oct 12, 2018 at 20:44
  • Q: Did you get this working?
    – paulsm4
    Oct 16, 2018 at 15:55

3 Answers 3

1

On your <VirtualHost *:443> you enable HSTS. That tells the browser to internally rewrite every request for the domain to HTTPS. After this, there's no use for the RewriteCond %{REQUEST_URI} !^/files/ as everything should be rewritten unconditionally already, giving you the possibility to simplify your configuration:

<VirtualHost *:80>
    ServerName abc.example.com
    Redirect permanent / https://abc.example.com/
</VirtualHost>

The blank page problem doesn't seem a server side issue caused by your Apache at all. The fact that you can access the URL from another computer pretty much proves it.

First clear all caches on the "reporting server" and see whether it helps or not. There's no details how this "reporting server" works, but the blank page might for example be a result of not displaying a cached HTTP URL image on a HTTPS page due to a changed mixed content policy. If so, the consistency between the redirection and the HSTS policy might prevent such caching in the future.

Also, fix all http:// URLs on your content to https://.

1
  • "Clearing caches on the reporting server" (assuming there ARE any caches there to clear) is a good suggestion. But we really need to isolate the problem. I agree with you that it's probably NOT "Apache" per se. Hopefully some of my troubleshooting suggestions might help. I encourage you to add any additional troubleshooting suggestions you might think of.
    – paulsm4
    Oct 13, 2018 at 4:06
1

Troubleshooting suggestions:

  1. Please confirm your environment:

    • Ubuntu 14.04 server with Apache2 v2.4.7.
    • Your httpd.conf is configured to serve both HTTP (:80) and HTTPS (:443).
    • Your certificate configuration (posted above) is correct, and the files and (SSLCertificateFile, etc.) all exist and haven't been "corrupted".
  2. Please confirm the problem description:

    • "Everything used to work" before you updated your SSL certificate, "the problem" has something to do with the new certificate.
    • All users who try to "get reports" via https (EXAMPLE https://apache-server.com/myreport) fail - they just see "white". Is this correct?
    • The same users are successfully able to get the same reports if they use http (EXAMPLE: http://apache-server.com/myreport) instead. Is this also correct?
    • Is there anything "special" about reports? For example, is it a web application?
  3. I'm curious if ALL https is "broken", or just "some stuff" is inaccessible.

    Please create a test file (EXAMPLE: /var/www/html/myreport/hello.html) and verify whether or not you can read it with both http and https.

  4. When you test, please use one (or both) of these tools on your PC to analyze the browser's attempt to connect (and thus help determine the specific point of failure):

  5. Please post back what you find.

PS:

Here are some additional tips. In particular, "enable verbose logging":

https://www.futurehosting.com/blog/troubleshooting-the-most-common-apache-problems-youll-encounter/

PPS:

I would NOT disable http until you've resolved the problem:

  • If nothing else, it's a temporary workaround for those folks who might need reports.
  • Being able to compare http vs. https behavior will help troubleshooting.
0

We figured it out. While the Apache server was fine as is, apparently we previously had a specific DH param snippet saved in the certificate file. This snippet was added because the reporting server that asks for the images and files runs on Java 6. And Java 6 can only handle DH params of 1024bit.

So in my new ssl_certificate.crt, I added

-----BEGIN DH PARAMETERS-----
MIGHAoGBAP//////////yQ/aoiFowjTExmKLgNwc0SkCTgiKZ8x0Agu+pjsTmyJR
Sgh5jjQE3e+VGbPNOkMbMCsKbfJfFDdP4TVtbVHCReSFtXZiXn7G9ExC6aY37WsL
/1y29Aa37e44a/taiZ+lrp8kEXxLH+ZJKGZR7OZTgf//////////AgEC
-----END DH PARAMETERS----- 

from http://httpd.apache.org/docs/current/ssl/ssl_faq.html#javadh

Unfortunately that downgrades our certificate from A+ rating to a B, but I'm not allowed to upgrade the Java server from 6 to whatever newer one.

Once I added that snippet to my ssl cert, the reporting server was able to access the images over https!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.