Part 2: Developing your first operator with Ansible.
In the previous post, we discussed the history of the Kubernetes operator, the problem that it's trying to solve, and why Ansible is such a natural fit for developing operators. In this post, we'll take a deeper look at the architecture of the Operator SDK and see just how easy it makes it for us to create an operator. Then we'll use the Operator SDK to create the scaffolding for an Ansible operator and discuss the layout.
In the previous post, we discussed the reconciliation loop, a recurring architectural concept in Kubernetes. The Ansible operator uses three primary components to execute a reconciliation loop, a playbook or role, the Operator SDK binary, and a watches file. If we were to look at a high level picture of how this is achieved, it would look something like this:
Image by Timothy Appnel. Used with permission.
Inside the dotted line is where the reconciliation loop occurs. Notice our three components. Of these three, the playbook or role is responsible for the primary action. This is where you will define exactly what needs to be done in response to the event in question. The Operator SDK binary is a generic operator written in Go; it will do all the heavy lifting in terms of executing the playbook or role when the event in question occurs. However, it does much more. The binary provides a reverse proxy service that handles caching and manages owner references for purposes of garbage collection. If we were to drill down a bit further we would see a more clear picture of how this comes together:
Image by Timothy Appnel. Used with permission.
Finally we have the watches file. The watches file allows the developer to define events and map them to specific playbook or roles. This is the glue that ties the Operator SDK binary together with your Ansible automation. This is where we will map our custom resources (identified by Group, Version, Kind, or GVK) to the playbook or role. Here is a simple example of a watches file:
--- version: v1alpha1 group: YourGroup.example.com kind: YourKind playbook: /path/to/playbook_or_role
From this point, the Operator SDK binary will monitor the cluster for the defined event, and execute the appropriate playbook or role when it identifies it. The watches file is not limited to one mapping, and it can do more things in addition to this. Here is an example of a more complex watches file:
--- # Simple example mapping Foo to the Foo role - version: v1alpha1 group: foo.example.com kind: Foo role: /opt/ansible/roles/Foo # Simple example mapping Bar to a playbook - version: v1alpha1 group: bar.example.com kind: Bar playbook: /opt/ansible/playbook.yml # More complex example for our Baz kind # Here we will disable requeuing and be managing the CR status in the playbook, # and specify additional variables. - version: v1alpha1 group: baz.example.com kind: Baz playbook: /opt/ansible/baz.yml reconcilePeriod: 0 manageStatus: false vars: foo: bar
The above example is taken from the Operator Framework Operator SDK user guide. If your interested in learning more about the watches file, you can do that here.
the best part
In the summary of the previous post, we saw that "the whole point" is that the Ansible operator reduces barriers to entry for those who want to develop operators but lack Go skills. After this discussion around the operator architecture, I hope that the way that is achieved has become more clear. This graphic does a great job of listing the components of the Ansible operator and highlighting exactly what is required of the developer:
Image by Timothy Appnel. Used with permission.
The image above has a grey box and a white box. In the grey box is everything that comes with the Operator SDK. In the white box is everything you need to provide as a developer: the Ansible automation and the watches file that ties everything together.
For the rest of this section we will be setting up the Operator SDK and building and examining the framework that it provides. Screenshots and examples will be provided for those who just want to read along, but the intent is to provide enough information for you to get set up on your own machine if you so desire. If you are interested in exploring the SDK, but don't want to set it up on your own machine, we'll talk about ways to do that in the next post.
set up go
In order to play with the Operator SDK, we're going to need to set up Go on your local machine. These instructions should work for anyone running linux, but if something doesn't work, please leave a comment below so we can improve the post.
The first thing to do is to set up your GOPATH and add it to your PATH. You can do that with the following commands:
mkdir -p $HOME/gopath/bin echo "export GOPATH=$HOME/gopath" >> $HOME/.bashrc echo "export PATH=$HOME/gopath/bin:/usr/local/go/bin:$PATH" >> $HOME/.bashrc
Don't forget to source your
.bashrc to make sure the changes take effect:
Next we need the Go binary from the official download page. As of the writing of this post, the most recent version is 1.13.4, and the examples below will reflect that, but be sure to check to make sure you have the right version. It's a tarball so we'll expand it, and we'll do that in
wget https://dl.google.com/go/go1.13.4.src.tar.gz tar -C /usr/local -xzf go1.13.4.src.tar.gz
Now you need to actually install Golang. You can do that with the following command:
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
You can take a look at the
install.sh script here.
We're going to need a nice directory for all of our operator development. Feel free to organize this in a way that suits you, but these examples work and you can customize them as you see fit.
mkdir -p $GOPATH/src/github.com/operator-framework
Change into your newly created directory:
Clone the Operator Framework Operator SDK git repository:
git clone https://github.com/operator-framework/operator-sdk
Then change into your operator-sdk directory and install:
cd gopath/src/github.com/operator-framework/operator-sdk git checkout master make tidy make install
initializing your operator
Now you're all set up and ready to go, you can create your first operator scaffolding and take a look at how everything is set up. It should be noted that so far, the steps to make any operator type (Go, Helm, or Ansible) are the same. You now have the SDK set up on your machine and ready to go. If you take a look at
operator-sdk --help you'll see all the great things you can do:
The command we're looking for is
new. As the output suggests, if you want to learn more about this command, or any of the others, simple use
operator-sdk [command] --help. You can initialize your operator with the name of your choice. We'll use tigeriq.
operator-sdk new tigeriq --api-version=shere.khan.com/v1alpha1 --kind=BigCat --type=ansible
One thing to note here is the last flag,
--type=ansible. This is what tells the SDK the type of operator you're building. The default is Go. After this command, you'll see a bit of output informing you about what the SDK is doing, and then if you list the contents of your directory, you'll see your new operator scaffolding, along with any other operators you've been working on:
examining the fundamentals
Let's take a look at what we have so far:
[krain@krain github.com]$ tree tigeriq/ tigeriq/ ├── build │ ├── Dockerfile │ └── test-framework │ ├── ansible-test.sh │ └── Dockerfile ├── deploy │ ├── crds │ │ ├── shere.khan.com_bigcats_crd.yaml │ │ └── shere.khan.com_v1alpha1_bigcat_cr.yaml │ ├── operator.yaml │ ├── role_binding.yaml │ ├── role.yaml │ └── service_account.yaml ├── molecule │ ├── default │ │ ├── asserts.yml │ │ ├── molecule.yml │ │ ├── playbook.yml │ │ └── prepare.yml │ ├── test-cluster │ │ ├── molecule.yml │ │ └── playbook.yml │ └── test-local │ ├── molecule.yml │ ├── playbook.yml │ └── prepare.yml ├── roles │ └── bigcat │ ├── defaults │ │ └── main.yml │ ├── files │ ├── handlers │ │ └── main.yml │ ├── meta │ │ └── main.yml │ ├── README.md │ ├── tasks │ │ └── main.yml │ ├── templates │ └── vars │ └── main.yml └── watches.yaml 17 directories, 25 files
build directory we see we get a Dockerfile along with all the necessary testing artifacts. It should be noted that the SDK doesn't require you to use Docker, there is also support for Buildah and Podman. If you're not familiar with those, this would be a great place to start.
deploy directory, we have what one would expect. Files with everything you need to create your necessary Kubernetes object. In short, this includes:
- The Kubernetes Custom Resource (CR)
- The Kubernetes Custom Resource Definition (CRD)
operator.yamlfile to define the operator
service_account.yamlfile to define the service account
role.yamlfile to define the role
role_binding.yamlfile to define the RoleBinding
For more information on Roles, RoleBindings, and Kubernetes RBAC in general, this page should prove quite useful.
The next directory is
molecule; this is the testing framework used with Ansible. It "provides support for testing with multiple instances, operating systems and distributions, virtualization providers, test frameworks and testing scenarios." While an in depth discussion of Molecule is outside the scope of this blog post, more information can be found here for those curious.
role directory is for your Ansible role. It's almost exactly what you would see, for example, if you created a role using
ansible-galaxy init [role] from the command line.
Finally, we have the
watches.yaml file, which we discussed in more detail above.
This is the basic layout of the Ansible Operator. After adding your specific information in the
watches.yaml file, adding your Ansible automation, and making minor adjustments to other files as necessary, you would move on to building and deploying your operator. If you would like to take a look at that process, the README page on the OperatorSDK GitHub is a great place to start.
In this post we took a closer look at the components of the Ansible Operator, installed the Operator SDK on a local machine, and created the scaffolding for a new operator. In the next and final post, we'll talk about resources that are available for anyone who would like to learn more about Ansible Operators and the operator community.