0

I'm hosting a clients wordpress site on a CentOS 7 server, with apache 2.4.6, hosted behind a cloudflare proxy. I set everything up about 8 months ago, and it has been running fine ever since.

At arround 5:30 this morning the website went down, showing timeouts. I checked the logs, in /var/log/httpd/error_log I found the following entries, around the time the site stopped working:

[Thu Sep 19 04:35:57.343495 2019] [mpm_prefork:notice] [pid 13447] AH00170: caught SIGWINCH, shutting down gracefully
[Thu Sep 19 04:35:58.598479 2019] [core:notice] [pid 22161] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Thu Sep 19 04:35:58.602722 2019] [suexec:notice] [pid 22161] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
[Thu Sep 19 04:35:58.638160 2019] [lbmethod_heartbeat:notice] [pid 22161] AH02282: No slotmem from mod_heartmonitor
[Thu Sep 19 04:35:58.701474 2019] [mpm_prefork:notice] [pid 22161] AH00163: Apache/2.4.6 (CentOS) PHP/5.4.16 configured -- resuming normal operations
[Thu Sep 19 04:35:58.701545 2019] [core:notice] [pid 22161] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

Since then, the only entries in the access_log have been from making local requests.

In order to debug, I've currently disabled the cloudflare proxy (so the DNS points directly to my server now) and I only have an index.html file in the web root. if I run wget mydomain.com locally on the machine, I get the contents of my test index.html just fine, and an entry appears in the access_log. If I try the same thing on an external server, I get the following error and no entry in the access_log:

(mydomain.com)|<the server's ip>|:80... failed: Resource temporarily unavailable

I've double checked the iptables config, but it hasn't been modified since May, so I'm pretty sure it's not a firewall issue.

I've also restarted httpd, mariadb and rebooted the server, to no avail.

Is there anything else that can be causing these issues with apache?

If it helps, the server is hosted in rackspace cloud, and the configs are as close to default as possible; I've not set up VirtualHosts or anything like that.

Here's the output of netstat -l -n -t -p

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1459/master
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      1383/mysqld
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1500/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      1459/master
tcp6       0      0 :::443                  :::*                    LISTEN      1110/httpd
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd
tcp6       0      0 :::80                   :::*                    LISTEN      1110/httpd
tcp6       0      0 :::22                   :::*                    LISTEN      1500/sshd

nmap output

Not shown: 999 filtered ports

PORT   STATE SERVICE VERSION

22/tcp open  ssh     OpenSSH 7.4 (protocol 2.0)
8
  • Can you edit your question and add the output of nmap -Pn <serverip> (run from another server, and you may need to install it first) and netstat -l -n -t -p (run on the server in question).
    – Halfgaar
    Sep 19, 2019 at 10:23
  • Certainly! I've just added the netstat output. I had to install nmap, and it gave a lot of socket_bindtodevice: Protocol not available Problem binding to interface , errno: 92 errors when I ran it. Is this an issue with my local machine, or to be expected?
    – Sam
    Sep 19, 2019 at 10:31
  • That's a local issue. Can you run it from somewhere else? Or as root?
    – Halfgaar
    Sep 19, 2019 at 10:33
  • Ah, I was running it in the windows ubuntu, which was probably causing the issue. Just tried zenmap instead, and added the output to the question
    – Sam
    Sep 19, 2019 at 10:41
  • I think it might have highlighted an issue too, as only port 22 appeared. I'm assuming ports 80 and 443 should be on the list too for http access to work?
    – Sam
    Sep 19, 2019 at 10:42

1 Answer 1

2

After looking at the output of nmap, it becomes apparent that the http(s) ports aren't being exposed. This is further confirmed by running firewall-cmd --list-all on the server:

public (active)
  target: default
  icmp-block-inversion: no
  interfaces: eth0 eth1
  sources:
  services: dhcpv6-client ssh
  ports:
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

to remedy this, run: firewall-cmd --add-service=https --permanent. You can confirm it works, by running firewall-cmd --list-all again, and seeing if http appears in the services: section.

After this, apache chan see all external requests again.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .