We are scaling our puppet infrastructure and would like to split off the CA component from the puppet master server to another server. Part of the change involves a servername change for the puppetmaster too.

I'm running into a problem whereby I can't get the ca_server directive to work properly in either the [main] or [agent] section. It's just not taking effect. So when I change server= to the new servername, it breaks the agents ability to check in because the servername has changed and doesn't match the certificate anymore.

I'm no puppet expert but what i'm thinking I need to do is create a SAN cert with both the old and new names in it (to be safe), and then re-sign all the agent nodes all over again which is going to be a royal PITA.

Is there a quicker/smarter way to do this? We already have hundreds of agent nodes out there and individually re-signing them will be an arduous task.

  • Hm, are you sure that the problem is that ca_server isn't working? It sounds like your Puppet master doesn't have a certificate that includes its new name as an alternative name to me, but maybe I'm misunderstanding the problem description. You will definitely need to issue a new certificate for the Puppet master that includes both names. – rra Mar 18 '13 at 1:01

We approached this in a different way, which seems to be more flexible and reliable in the long run.

We created a frontend apache server, running mod_proxy and mod_balancer. This then identifies the incoming URL request, and routes CA-related requests to a local CA server, and puppetmaster requests to a pool of puppetmasters. This has the added benefit that we can have a separate server(s) handling different environments.

The puppetmasters have to be configured so that they accept the authentication information from the frontend server.

Define the balancer (note, 600 timeout is important):

<Proxy balancer://puppetmaster>
  BalancerMember http://pupappprd01.its.auckland.ac.nz:18140 timeout=600
  BalancerMember http://pupappprd02.its.auckland.ac.nz:18140 timeout=600
  BalancerMember http://pupappprd03.its.auckland.ac.nz:18140 timeout=600
# CA, facts and filebucket server
<Proxy balancer://puppetmasterca>
  BalancerMember http://puprepprd01.its.auckland.ac.nz:18140

Now define the frontend:

Listen 8140
<VirtualHost *:8140>
  SSLEngine on
  SSLProtocol -ALL +SSLv3 +TLSv1
  SSLCertificateKeyFile /var/lib/puppet/ssl/private_keys/puppet.auckland.ac.nz.pem
  SSLCertificateFile /var/lib/puppet/ssl/certs/puppet.auckland.ac.nz.pem
  SSLCACertificateFile /var/lib/puppet/ssl/ca/ca_crt.pem
  SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
  SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
  SSLVerifyClient optional
  SSLVerifyDepth  1
  SSLOptions +StdEnvVars

  # Send info to downstream workers
  RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
  RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
  RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

  <Location / >
    SetHandler balancer-manager
    Order allow,deny
    Allow from all

  # The manifest can take up to 10min to build (default timeout is 2min)
  Timeout 600
  ProxyTimeout 600
  # This is required to prevent a race condition that can cause
  # the puppet agent to lock up
  SetEnv proxy-nokeepalive 1
  SetEnv proxy-initial-not-pooled 1

  ProxyPreserveHost On

  # CA - centralise the authentication
  # members of the puppetmasterca cluster will rsync the cert stores
  ProxyPassMatch   ^(/.*?)/(certificate.*?)/(.*)$ balancer://puppetmasterca
  ProxyPassReverse ^(/.*?)/(certificate.*?)/(.*)$ balancer://puppetmasterca

  # Filebucket - this be on the central server to minimise duplication
  # members of the puppetmasterca cluster will rsync the file bucket
  ProxyPassMatch   ^(/.*?)/file_bucket_file/(.*)$ balancer://puppetmasterca
  ProxyPassReverse ^(/.*?)/file_bucket_file/(.*)$ balancer://puppetmasterca

  # ALL Report uploads handled by central servers
  # These will in turn upload reports to dashboard, depending on settings
  # in the puppet.conf for that environment
  ProxyPassMatch   ^(/.*?)/report/(.*)$ balancer://puppetmasterca
  ProxyPassReverse ^(/.*?)/report/(.*)$ balancer://puppetmasterca

  # Production servers - catalogue, cache, facts, file metadata and fetch
  # These servers all synchronise with subversion every 15 min
  # Need the extended timeout because some manifest generation can
  # be slow. 5min should be sufficient.
  ProxyPassMatch   ^/production/ balancer://puppetmaster timeout=600
  ProxyPassReverse ^/production/ balancer://puppetmaster timeout=600


Now, we can define the puppetmaster handling the requests on the CA server and on the Puppetmaster. Note how we pass the authentication info in the additional header fields:

Listen 18140
<VirtualHost *:18140>
  SSLEngine off

  # Obtain Authentication Information from Client Request headers
  SetEnvIf X-Client-Verify "(.*)" SSL_CLIENT_VERIFY=$1
  SetEnvIf X-Client-DN "(.*)" SSL_CLIENT_S_DN=$1

  RackAutoDetect On
  DocumentRoot /usr/share/puppet/rack/puppetmasterd/public
  <Directory /usr/share/puppet/rack/puppetmasterd/>
      Options None
      AllowOverride None
      Order allow,deny
      allow from
      allow from puprepprd01.its.auckland.ac.nz
      deny from all

    LogLevel warn
    ErrorLog /var/log/httpd/puppetmaster_error.log
    CustomLog /var/log/httpd/puppetmaster_access.log combined

In the puppet.conf, you need a couple more lines to pull the authentication info from the environment:

    ssl_client_header = HTTP_X_CLIENT_DN
    ssl_client_verify_header = HTTP_X_CLIENT_VERIFY

This is more complex, but allows us to expand horizontally and to split environments to their own puppetmaster servers as much as we want. One separate server holds the report frontend and CA (though this can be split to multiple CA backends if you set up some sort of cert replication).

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.