I am new to chef (using hosted chef server) and am muddling along ok and basically understand how to provision individual servers. What I am having trouble with is figuring out how to integrate the various provisioned servers into a functional cluster.
In my current use case I am using Amazon EC2. I am using a load balancer with a few varnish servers which pass requests to several front end application servers connected to an RDS server. I also have a backend utility server which has to occasionally sync some files to the FE app servers.
How would you glue all of this together? The FE servers need to know about the rds instance and a redis server but the backend utlity server and the varnish nodes have to know about the FE app servers. Ideally the app servers will implement some kind of autoscaling where more nodes are provisioned as necessary.
Finally, couple this with also needing to have dev and stage environments as well where often the varnish servers are on the same VM as the app server.
Do you use tagging to sort of register the FE nodes and then query those values when you run recipes on the varnish and BE servers last?
I am just looking for some best practices on what I would assume is a fairly common n-tier web cluster use case.