Answers to your questions.
In this answer, "Chef" or "chef" refers to Chef the product, usually in a client form. "CHEF" refers to Chef Software, Inc. Opscode renamed to Chef Software, Inc in 2013.
Not clear if it's better to setup roles in ruby DSL, JSON, or from the management console? Why are there multiple ways to do the same thing?
There are multiple ways to do the same thing because people have different workflows. You pick the workflow that is best for your environment. Let me explain what the differences are so you can make an informed decision.
The Ruby DSL for Roles exists to make it easier to write roles without knowing the syntax of JSON. It is a simple way to get started with Roles. Once you have made changes, you upload them to the Chef Server with knife.
knife role from file myrole.rb
This converts the role to JSON and stores it on the server. If you have an environment that enforces the Chef Repository where your roles live as the source of truth, this works quite well.
JSON is what the Chef Server stores, so you also edit JSON directly in the management console. It does require more fields than the Ruby DSL in order for Knife to recognize it properly to upload. Those details are hidden to some degree via the web UI.
The disadvantage of using the webui/management console for editing roles is they aren't in your local version control system unless you download them from the server. You can do this with knife:
knife role show myrole -Fj
-Fj tells knife to "display in JSON format." You can redirect the output to a .json file if you like.
Update: There are additional knife commands for working with the files in the local chef repository. Currently these commands support only JSON format files. A community RFC is open which will address adding support for the Ruby DSL for these plugins. Here's a brief summary of the workflow.
Check content differences between the server and the local file.
knife diff roles/myrole.json
Upload a JSON formatted role file. The
roles/ path is required. This gets mapped to the same API endpoint on the server.
knife upload roles/myrole.json
Download the content from the server overwriting the content of the file in the repository.
knife download roles/myrole.json
These commands come from
knife-essentials, which is built into the chef client package.
Can you organize cookbooks into subdirectories? eg- we have custom software that I'd like to write a cookbook for and stick that into: chef-repo/cookbooks/ourcompanystuff/customsoftwarecookbook would this be a good practice?
No. Knife has an expectation of where cookbooks should live because it uses an API to upload cookbooks to the Server. This is set in the knife.rb with
cookbook_path. In older versions of Chef, you could specify an array of paths for cookbooks, but this is being deprecated because it required more maintenance and was confusing to users.
By convention we name customer specific or site specific cookbooks with the name prefixed in the cookbook diretory. For your example, it would be:
There might be multiple different cookbooks for "ourcompany" depending on what you're doing.
Do I create a cookbook for each type of role that specifies what it does? Do I have these cookbooks include other cookbooks (i.e.- the cookbook for my webserver role includes the apache cookbook). I'm not sure how cookbook inter-dependencies and inheritance are handled.
There is no direct relationship or dependency between roles and cookbooks.
Roles have a run list, which specifies the recipes and other roles that should be applied to any node that has that role. Nodes have a run list that can contain roles or recipes. When Chef runs on the node, it will expand the run list for all the roles and recipes it includes, and then download the cookbooks required. In a node run list:
Chef will download the apache2 cookbook for the node so it can apply this recipe.
You might have a cookbook specific for a role in your infrastructure. More commonly you'll have cookbooks that are for setting up certain types of services like apache2, mysql, redis, haproxy, etc. Then you would put those into appropriate roles. If you have custom application specific things that need to happen to fulfill a role, then you could write this into a custom cookbook (like I referenced above).
Is there anything like puppets external node classifier so nodes automatically determine their roles?
"Yes." The Chef Server does node data storage (in JSON) automatically, and the server also automatically indexes all the node data for search.
It seems like you can configure things with knife or within the management console, or editing JSON files? This is super confusing to me why there are so many ways to do things, it's paralyzing! Is there a reason to use one or the other?
The Chef Server has a RESTful API that sends and receives JSON responses. Knife and the management console are user interfaces for interacting with the API from an administration point of view.
You can use the tool you like better, though the management console doesn't have as many features as Knife. Most people that use Chef prefer the command-line interface for the power and flexibility it provides, even folks who are using Chef on Windows. Further, knife is a plugin based tool that you can create new plugins to interact with the Chef Server, or with other parts of your infrastruture.
Chef is a set of libraries and primitives, and an API. It gives you the flexibility to build the configuration management system that works best for your infrastructure.
How can I automatically provision nodes with chef in my dev cluster? With puppet I fire up a VM that connects to the puppermatser and kicks off a puppet run and sets itself up (role is determined by external node classifier). How do I do this with chef? - Install chef with pem/rb files that tie it to a chef server, manually tell the node its roles with knife or editing this in the management interface, and then kicking off a chef-client run to set itself up?
You'll want to use Knife Bootstrap. This is a built in plugin that comes with knife. You invoke it like this:
knife bootstrap 10.1.1.112 -x root -i ~/.ssh/root_id_rsa -r 'role[webserver]'
- SSH to the target system (10.1.1.112) as the
root user using an SSH key (you could ssh as another user and then use
- Install Ruby
- Install Chef
- Create the Chef configuration file for your Chef Server, reading knife's configuration (.chef/knife.rb).
- Copy the "validation" RSA private key, which the node will use to automatically register with the Chef Server.
chef-client using the comma separated run listed specified. In this example only the
webserver role is applied.
This assumes that the target system has been provisioned, has an IP address and you can SSH as root. Depending on your local policies and provisioning process, you may need to adjust how this works. The knife bootstrap page on the wiki describes more about how this works.
The knife bootstrap command uses bootstrap "templates" which are platform specific shell scripts that will install Chef in the best known working configuration. Different distributions have different Ruby versions available, unfortunately, so there's different scripts. You can also create your own bootstrap templates. You can specify a different one with the
knife bootstrap [ ... ] -d centos5-gems
Opscode is working on a full stack Chef client installer and bootstrap template that will remove a lot of these issues. It is in beta testing and it will be the way Opscode will recommend installing Chef.
Knife also has plugins for a number of public cloud computing providers such as Amazon EC2 and Rackspace Cloud. There are plugins available for private cloud environments like Eucalyptus and OpenStack. There are also plugins for VMware, Vsphere and others. You can see further information on the wiki.
Are there any other good chef resources I might be missing in my searches?
The Chef Documentation the primary source of documentation. CHEF Opscode is working on updating the Wiki documentation across the board to be consistent with the current release of Chef, and to appropriately separate the beginner information from advanced information. This should be completed fairly soon.
CHEF runs Chef Fundamentals training, which is a 2 day classroom based technical training course with lectures and exercises. Public training courses are offered periodically, and the materials for the training are available free of charge by registering with the Open Training program. Registration is required so CHEF can find out what people are interested in, and to contact people that sign up about new releases of the materials, or new public classes.
CHEF's Open Training program:
I occasionally post tips, tricks and guides about Chef to my blog: http://jtimberman.housepub.org/. I also have a series called "quick tips"
CHEF customers can get help and support on the support site:
The Chef user community is an excellent source of additional help:
Additional resources are available on CHEF's web site.
I hope this helps.