Chef Infra Overview

Chef Infra is a powerful automation platform that transforms infrastructure into code. Whether you’re operating in the cloud, on-premises, or in a hybrid environment, Chef Infra automates how infrastructure is configured, deployed, and managed across your network, no matter its size.

This diagram shows how you develop, test, and deploy your Chef Infra code.

Diagram showing Chef Workstation with Chef Infra Server and Chef Infra Client

Chef Workstation is the location where users interact with Chef Infra. With Chef Workstation, users can author and test cookbooks using tools such as Test Kitchen and interact with the Chef Infra Server using the knife and chef command line tools.
Chef Infra Client Chef Infra Client runs on systems that are managed by Chef Infra. The Chef Infra Client executes on a schedule to configure a system to the desired state.
Chef Infra Server acts as a hub for configuration data. Chef Infra Server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that’s being managed by Chef. Nodes use the Chef Infra Client to ask the Chef Infra Server for configuration details, such as recipes, templates, and file distributions.

Chef Infra Components

The following diagram shows the relationships between the various elements of Chef Infra, including the nodes, the server, and the workstation. These elements work together to provide the Chef Infra Client the information and instruction that it needs so that it can do its job. As you are reviewing the rest of this topic, use the icons in the tables to refer back to this image.

Diagram of Chef Infra Client, Server, and Workstation

Chef Infra has the following major components:

Component	Description
	One (or more) workstations are configured to allow users to author, test, and maintain cookbooks. Workstation systems run the Chef Workstation package which includes tools such as Chef Infra Client, Chef InSpec, Test Kitchen, ChefSpec, Cookstyle, and other tools necessary for developing and testing your infrastructure with Chef products.
	Cookbooks are uploaded to the Chef Infra Server from these workstations. Some cookbooks are custom to the organization and others are based on community cookbooks available from the Chef Supermarket.
	Ruby is the programming language that's the authoring syntax for cookbooks. Most recipes are simple patterns (blocks that define properties and values that map to specific configuration items like packages, files, services, templates, and users. The full power of Ruby is available for when you need a programming language.
	A node is any device—physical, virtual, cloud, network device, etc.—that’s under management by Chef Infra.
	Chef Infra Client is installed on each node that's managed with Chef Infra. Chef Infra Client configures the node locally by performing the tasks specified in the run-list. Chef Infra Client will also pull down any required configuration data from the Chef Infra Server during a Chef Infra Client run.
	The Chef Infra Server acts as a hub of information. Cookbooks and policy settings are uploaded to the Chef Infra Server by users from workstations. The Chef Infra Client accesses the Chef Infra Server from the node on which it's installed to get configuration data, performs searches of historical Chef Infra Client run data, and then pulls down the necessary configuration data. After a Chef Infra Client run is finished, the Chef Infra Client uploads updated run data to the Chef Infra Server.
	Chef Supermarket is the location in which community cookbooks are shared and managed. Cookbooks that are part of the Chef Supermarket may be used by any Chef user. How community cookbooks are used varies from organization to organization.

Chef Infra Client run reporting, compliance reporting, high availability configurations, and Chef Infra Server replication are available as part of Chef Automate.

The following sections discuss these elements (and their various components) in more detail.

Workstations

A workstation is your local computer running Chef Workstation that you use to author cookbooks, interact with the Chef Infra Server, and interact with nodes.

The workstation is where users do most of their work, including:

Developing and testing cookbooks
Keeping the Chef Infra repository synchronized with version source control
Configuring organizational policy by including defining roles and applying Policyfiles or policy groups
Interacting with nodes, as (or when) required, such as performing a bootstrap operation

Chef Workstation Components and Tools

Some important tools and components of Chef Workstation include:

Component	Description
	Start your infrastructure automation with Chef Workstation. Chef Workstation gives you everything you need to get started with Chef - ad hoc remote execution, remote scanning, configuration tasks, cookbook creation tools as well as robust dependency and testing software - all in one easy-to-install package. Chef Workstation includes: Chef Infra Client Chef InSpec Chef Habitat chef and knife command line tools Testing tools such as Test Kitchen and Cookstyle Everything else needed to author cookbooks and upload them to the Chef Infra Server
	Chef Workstation includes important command-line tools: Chef Infra: Use the chef command-line tool to work with items in a chef-repo, which is the primary location in which cookbooks are authored, tested, and maintained, and from which policy is uploaded to the Chef Infra Server Knife: Use the knife command-line tool to interact with nodes or work with objects on the Chef Infra Server Chef Infra Client: an agent that configures your nodes Test Kitchen: a testing harness for rapid validation of Chef code Chef InSpec: Chef's open source security & compliance automation framework chef-run: a tool for running ad-hoc tasks Chef Workstation App: for updating and managing your chef tools
	The chef-repo is the repository structure in which cookbooks are authored, tested, and maintained: Cookbooks contain recipes, attributes, custom resources, libraries, files, templates, tests, and metadata The chef-repo should be synchronized with a version control system (such as git), and then managed as if it were source code The directory structure within the chef-repo varies. Some organizations prefer to keep all of their cookbooks in a single chef-repo, while other organizations prefer to use a chef-repo for every cookbook.
	Use Test Kitchen to automatically test cookbooks across any combination of platforms and test suites: Test suites are defined in a kitchen.yml file. See the configuration documentation for options and syntax information. Supports cookbook testing across many cloud providers and virtualization technologies. Uses a comprehensive set of operating system base images from Chef’s Bento project.
	Use ChefSpec to simulate the convergence of resources on a node: Is an extension of RSpec, a behavior-driven development (BDD) framework for Ruby Is the fastest way to test resources and recipes

Cookbooks

A cookbook is the fundamental unit of configuration and policy distribution in Chef Infra.

A cookbook defines a scenario and contains everything that’s required to support that scenario:

Recipes that specify which Chef Infra built-in resources to use, as well as the order in which they’re to be applied
Attribute values, which allow environment-based configurations such as dev or production.
Custom Resources for extending Chef Infra beyond the built-in resources.
Files and Templates for distributing information to systems.
Custom Ohai Plugins for extending system configuration collection beyond the Ohai defaults.
The metadata.rb file, which describes the cookbook itself and any dependencies it may have.

The Chef Infra Client uses Ruby as its reference language for creating cookbooks and defining recipes, with an extended DSL for specific resources. A reasonable set of resources are available to the Chef Infra Client, enough to support many of the most common infrastructure automation scenarios; however, this DSL can also be extended when additional resources and capabilities are required.

Components

Cookbooks are comprised of the following components:

Component	Description
	An attribute can be defined in a cookbook (or a recipe) and then used to override the default settings on a node. When a cookbook is loaded during a Chef Infra Client run, these attributes are compared to the attributes that are already present on the node. Attributes that are defined in attribute files are first loaded according to cookbook order. For each cookbook, attributes in the `default.rb` file are loaded first, and then additional attribute files (if present) are loaded in lexical sort order. When the cookbook attributes take precedence over the default attributes, Chef Infra Client applies those new settings and values during a Chef Infra Client run on the node.
	Use the cookbook_file resource to transfer files from a sub-directory of `COOKBOOK_NAME/files/` to a specified path located on a host that’s running Chef Infra Client. The file is selected according to file specificity, which allows different source files to be used based on the hostname, host platform (operating system, distro, or as appropriate), or platform version. Files that are located in the `COOKBOOK_NAME/files/default` sub-directory may be used on any platform.
	A library allows arbitrary Ruby code to be included in a cookbook. The most common use for libraries is to write helpers that are used throughout recipes and custom resources. A library file is a Ruby file that’s located within a cookbook’s `/libraries` directory. Because a library is built using Ruby, anything that can be done with Ruby can be done in a library file, including advanced functionality such as extending built-in Chef classes.
	Every cookbook requires a small amount of metadata. The contents of the `metadata.rb` file provides information that helps Chef Infra Client and Server correctly deploy cookbooks to each node. A `metadata.rb` file is: Located at the top level of a cookbook’s directory structure. Compiled whenever a cookbook is uploaded to the Chef Infra Server or when the `knife cookbook metadata` subcommand is run, and then stored as JSON data. Created automatically by knife whenever the `knife cookbook create` subcommand is run. Edited using a text editor, and then re-uploaded to the Chef Infra Server as part of a cookbook upload.
	A recipe is the most fundamental configuration element within the organization. A recipe: Is authored using Ruby, which is a programming language designed to read and behave in a predictable manner Is mostly a collection of resources, defined using patterns (resource names, attribute-value pairs, and actions); helper code is added around this using Ruby, when needed Must define everything that’s required to configure part of a system Must be stored in a cookbook May be included in another recipe May use the results of a search query and read the contents of a data bag (including an encrypted data bag) May have a dependency on one (or more) recipes Must be added to a run-list before it can be used by Chef Infra Client Is always executed in the same order as listed in a run-list The Chef Infra Client will run a recipe only when asked. When the Chef Infra Client runs the same recipe more than once, the results will be the same system state each time. When a recipe is run against a system, but nothing has changed on either the system or in the recipe, the Chef Infra Client won't change anything. The Chef Infra Language is a comprehensive systems configuration language with resources and helpers for configuring operating systems. The language is primarily used in Chef Infra recipes and custom resources to tell the Chef Infra Client what actions to take to configure a system. The Chef Infra Language provides resources for system-level components such as packages, users, or firewalls, and it also includes helpers to allow you to make configuration decisions based on operating systems, clouds, virtualization hypervisors, and more.
	A resource is a statement of configuration policy that: Describes the desired state for a configuration item Declares the steps needed to bring that item to the desired state Specifies a resource type—such as `package`, `template`, or `service` Lists additional details (also known as resource properties), as necessary Are grouped into recipes, which describe working configurations Chef has many built-in resources that cover all of the most common actions across all of the most common platforms. You can build your own resources to handle any situation that's not covered by a built-in resource.
	A cookbook template is an Embedded Ruby (ERB) template that’s used to dynamically generate static text files. Templates may contain Ruby expressions and statements, and are a great way to manage configuration files. Use the template resource to add cookbook templates to recipes; place the corresponding Embedded Ruby (ERB) template file in a cookbook’s `/templates` directory.
	Testing cookbooks improves the quality of those cookbooks by ensuring they're doing what they're supposed to do and that they're authored in a consistent manner. Unit and integration testing validates the recipes in cookbooks. Syntax testing---often called linting---validates the quality of the code itself. The following tools are popular tools used for testing Chef recipes: Test Kitchen, ChefSpec, and Cookstyle.

Nodes

A node is any device—physical, virtual, cloud, network device, etc.—that’s under management by Chef Infra.

Node Types

The types of nodes that can be managed by Chef include, but aren’t limited to, the following:

Node Type	Description
	A physical node is typically a server or a virtual machine, but it can be any active device attached to a network that's capable of sending, receiving, and forwarding information over a communications channel. In other words, a physical node is any active device attached to a network that can run a Chef Infra Client and also allow that Chef Infra Client to communicate with a Chef Infra Server.
	A cloud-based node is hosted in an external cloud-based service, such as Amazon Web Services (AWS), OpenStack, Rackspace, Google Compute Engine, or Microsoft Azure. Plugins are available for knife that provide support for external cloud-based services. knife can use these plugins to create instances on cloud-based services. Once created, Chef Infra Client can be used to deploy, configure, and maintain those instances.
	A virtual node is a machine that runs only as a software implementation, but otherwise behaves much like a physical machine.
	A network node is any networking device---a switch, a router---that's being managed by a Chef Infra Client, such as networking devices by Juniper Networks, Arista, Cisco, and F5. Use Chef to automate common network configurations, such as physical and logical Ethernet link properties and VLANs, on these devices.
	Containers are an approach to virtualization that allows a single operating system to host many working configurations, where each working configuration---a container---is assigned a single responsibility that's isolated from all other responsibilities. Containers are popular as a way to manage distributed and scalable applications and services.

Chef on Nodes

The key components of nodes that are under management by Chef include:

Component Description

Component	Description
	Chef Infra Client is an agent that runs locally on every node that’s under management by Chef Infra Server. When Chef Infra Client runs, it performs all of the steps required for bringing a node into the expected state, including: Registering and authenticating the node with Chef Infra Server Synchronizing cookbooks from the Chef Infra Server to the node Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies Taking the appropriate and required actions to configure the node based on recipes and attributes Reporting summary information on the run to Chef Automate Chef Infra Client authenticates with the Chef Infra Server using RSA public key-pairs each time a Chef Infra Client needs access to data that is stored on the Chef Infra Server. This prevents any node from accessing data that it shouldn’t and it ensures that only nodes that are properly registered with the Chef Infra Server can be managed.
	Ohai is a tool for collecting system configuration data, which it then provides to Chef Infra Client to use in cookbooks. Chef Infra Client runs Ohai at the start of every Chef Infra run to determine system state. The attributes that Ohai collects are called `automatic attributes`. Chef Infra Client uses these attributes to ensure that nodes are in the desired state after each configuration run. The types of attributes Ohai collects include but aren’t limited to: Operating System Network Memory Disk CPU Kernel Host names Fully qualified domain names Virtualization Cloud provider metadata Ohai includes required and optional plugins to detect common configuration information. Ohai has a plugin model and language to write custom plugins to collect additional system state information.

Chef Infra Client is an agent that runs locally on every node that’s under management by Chef Infra Server. When Chef Infra Client runs, it performs all of the steps required for bringing a node into the expected state, including:

Registering and authenticating the node with Chef Infra Server
Synchronizing cookbooks from the Chef Infra Server to the node
Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies
Taking the appropriate and required actions to configure the node based on recipes and attributes
Reporting summary information on the run to Chef Automate

Chef Infra Client authenticates with the Chef Infra Server using RSA public key-pairs each time a Chef Infra Client needs access to data that is stored on the Chef Infra Server. This prevents any node from accessing data that it shouldn’t and it ensures that only nodes that are properly registered with the Chef Infra Server can be managed.

Ohai is a tool for collecting system configuration data, which it then provides to Chef Infra Client to use in cookbooks. Chef Infra Client runs Ohai at the start of every Chef Infra run to determine system state. The attributes that Ohai collects are called automatic attributes. Chef Infra Client uses these attributes to ensure that nodes are in the desired state after each configuration run.

The types of attributes Ohai collects include but aren’t limited to:

Operating System
Network
Memory
Disk
CPU
Kernel
Host names
Fully qualified domain names
Virtualization
Cloud provider metadata

Ohai includes required and optional plugins to detect common configuration information. Ohai has a plugin model and language to write custom plugins to collect additional system state information.

The Chef Infra Server

The Chef Infra Server acts as a hub for configuration data. The Chef Infra Server stores cookbooks, the policies that are applied to nodes, and the metadata that describes each registered node that is under management by Chef Infra Client. Nodes use Chef Infra Client to ask the Chef Infra Server for configuration details, such as recipes, templates, and file distributions. Chef Infra Client then does as much of the configuration work as possible on the nodes themselves (and not on the Chef Infra Server). This scalable approach distributes the configuration effort throughout the organization.

Feature	Description
	Search indexes allow queries to be made for any type of data that’s indexed by the Chef Infra Server, including data bags (and data bag items), environments, nodes, and roles. A defined query syntax is used to support search patterns like exact, wildcard, range, and fuzzy. A search is a full-text query that can be done from several locations, including from within a recipe, by using the `search` subcommand in knife, the `search` method in the Chef Infra Language, the search box in the Chef management console, and by using the `/search` or `/search/INDEX` endpoints in the Chef Infra Server API. The search engine is based on Elasticsearch and is run from the Chef Infra Server.
	Chef management console is a web-based interface for the Chef Infra Server that provides users a way to manage the following objects: Nodes Cookbooks and recipes Roles Stores of JSON data (data bags), including encrypted data Environments Searching of indexed data User accounts and user data for the individuals who have permission to log on to and access Chef Infra Server
	Data bags store global variables as JSON data. Data bags are indexed for searching and can be loaded by a cookbook or accessed during a search.
	Policy defines how business and operational requirements, processes, and production workflows map to objects that are stored on the Chef Infra Server. Policy objects on the Chef Infra Server include roles, environments, and cookbook versions.

Policy

Policy maps business and operational requirements, process, and workflow to the following settings and objects stored on the Chef Infra Server:

Roles define server types, such as “web server” or “database server”.
Environments define process, such as “dev”, “staging”, or “production”.
Attributes define environment-specific details about a node that are included in a Policyfile.
Certain types of data—passwords, user account data, and other sensitive items—can be placed in data bags, which are located in a secure sub-area on the Chef Infra Server that can only be accessed by nodes that authenticate to the Chef Infra Server with the correct SSL certificates.
The cookbooks (and cookbook versions) in which organization-specific configuration policies are maintained.

Some important aspects of policy include:

Feature	Description
	A role is a way to define certain patterns and processes that exist across nodes in an organization as belonging to a single job function. Each role consists of zero (or more) attributes and a run-list. Each node can have zero (or more) roles assigned to it. When a role is run against a node, the configuration details of that node are compared against the attributes of the role, and then the contents of that role’s run-list are applied to the node’s configuration details. When a Chef Infra Client runs, it merges its own attributes and run-lists with those contained within each assigned role.
	An environment is a way to map an organization’s real-life workflow to what can be configured and managed when using Chef Infra. This mapping is accomplished by setting attributes and pinning cookbooks at the environment level. With environments, you can change cookbook configurations depending on the system’s designation. For example, by designating different staging and production environments, you can then define the correct URL of a database server for each environment. Environments also allow organizations to move new cookbook releases from staging to production with confidence by stepping releases through testing environments before entering production.
	A cookbook version represents a set of functionality that’s different from the cookbook on which it’s based. A version may exist for many reasons, such as ensuring the correct use of a third-party component, updating a bug fix, or adding an improvement. A cookbook version is defined using syntax and operators, may be associated with environments, cookbook metadata, and/or run-lists, and may be frozen (to prevent unwanted updates from being made). A cookbook version is maintained just like a cookbook, with regard to source control, uploading it to the Chef Infra Server, and how Chef Infra Client applies that cookbook when configuring nodes.
	A run-list defines all of the information necessary for Chef to configure a node into the desired state. A run-list is: An ordered list of roles and/or recipes that are run in the exact order defined in the run-list; if a recipe appears more than once in the run-list, Chef Infra Client won’t run it twice Always specific to the node on which it runs; nodes may have a run-list that’s identical to the run-list used by other nodes Stored as part of the node object on the Chef server Maintained using knife and then uploaded from the workstation to the Chef Infra Server, or maintained using Chef Automate

Conclusion

Chef is a thin DSL (domain-specific language) built on top of Ruby. This approach allows Chef to provide just enough abstraction to make reasoning about your infrastructure easy. Chef includes a built-in taxonomy of all the basic resources one might configure on a system, plus a defined mechanism to extend that taxonomy using the full power of the Ruby language. Chef chose Ruby because it provides the flexibility to use both the simple built-in taxonomy, as well as being able to handle any customization path your organization requires.