I am using puppet agent v3.0.1 (On FreeBSD), trying to connect to a puppet master v3.1.1 (On Ubuntu) and the puppet agent hangs with the following debug output:
puppet agent --debug --no-daemonize
...
Debug: /File[/var/puppet/ssl/private_keys]: Autorequiring File[/var/puppet/ssl]
Debug: /File[/var/puppet/ssl/private]: Autorequiring File[/var/puppet/ssl]
Debug: /File[/var/puppet/ssl/private_keys/zg-3.XXXX.ch.pem]: Autorequiring File[/var/puppet/ssl/private_keys]
Debug: /File[/var/puppet/ssl/public_keys/zg-3.XXXX.ch.pem]: Autorequiring File[/var/puppet/ssl/public_keys]
Debug: Finishing transaction 17214983260
[hangs.. about two minutes..]
Error: Could not request certificate: execution expired
Error: Could not request certificate: execution expired
The puppet master is hosted in a virtual machine and accessed via port forwarding over the VM host machine. I have tested access to the master from another node behind the firewall, which works correctly.
When the agent runs there is no indication on the puppet master that a connection has been made (although it's not clear that there should be).
I have checked with netcat & telnet and confirmed that I able to initiate a connection from the agent to the server and exchange data.
The same puppet master has worked correctly with a different FreeBSD agent running Puppet v3.0.2
Running tcpdump during the agent run shows that some packets are going back and forth between the agent and server
I have run truss on the (FreeBSD) puppet agent command and saw that the agent is getting the following error (twice):
read(6,0x80711c3c6,1854) ERR#35 'Resource temporarily unavailable'
File handle 6 is from a 'connect' system call to the correct IP address and port number (8140) of the master system.
I currently suspect the problem could be caused by one of the following:
An agent/master puppet software compatibility issue
SSL library compatibility problem
Some strange firewall or NAT issue, which allows netcat/telnet communication, but not Puppet communication. A connection is being made as I get a different error when I run the agent but don't start the master
A timing error or race condition
I don't think the problem is caused by:
SSL certificate issue, as there ano certificate related errors. I've also tried deleting the contents of the ssl directory on both the agent and master systems
Time synchronization problem, since both systems run NTP
Any hints on how to debug this?
Thanks in advance,
Alan Hodgkinson
P.S. Please ask for any addition (config) information necessary to help diagnose the problem.