I'm running the active Zabbix agents on all the servers in my production environment, however two of them aren't able to connect to the Zabbix server. All I've got to go on in the Zabbix logs is
active check configuration update from [<zabbix server>:10051] started to fail (cannot connect to [[<zabbix server>]:10051]:  Interrupted system call)
which seems to be a very common error message! The difficulty I'm having is working out why node 2 in my DB cluster is connecting fine, but node 1, which is configured identically, isn't. The setup is fairly simple, the only complexity is that my Zabbix server lives on a different network (192.168.10.0/24) to my agents (10.10.0.0/24) so I've had to configure the necessary routes accordingly. There are no proxies between the networks.
Things I've checked:
- The Zabbix agent is running everywhere it needs to be
- The hostnames in both agent and server configs match
- The IP addresses in the server config are correct
- The correct routes show in
/etc/sysconfig/network-scripts/route-eth0on my agents and server
- There are no rules in
iptableson servers or agents to block traffic on 10050 or 10051
- The server can reach the agents via
- None of the agents (working and broken) can reach the server via
telnetisn't installed on any of my agents. I'd prefer to hold off installing things on my production servers until the next maintenance window!
What have I missed?