In today's digital age, software development holds immense relevance as the backbone of our interconnected world. From powering mobile applications and web platforms to driving innovation across industries, software development plays a crucial role in meeting the evolving needs and challenges of our society. It enables businesses to streamline operations, automate processes, and enhance customer experiences, fostering growth and competitiveness. Moreover, software development is instrumental in sectors such as healthcare, finance, education, and transportation, revolutionizing patient care, financial management, learning, and mobility. It also facilitates global connectivity, enabling remote collaboration and the exchange of ideas. As emerging technologies like artificial intelligence and blockchain continue to reshape our world, software development remains a vital discipline, driving their implementation, improvement, and integration.
Software development encompasses a wide range of activities involved in creating, designing, and maintaining software systems. This includes tasks such as writing code, designing software architectures, testing, debugging, and generating configuration files or deployment manifests. Recently while browsing through my tweeter feed, I came across an interesting tweet by Leigh Capili.
For Kubernetes to do its job for its users, we need to pass it some programs. In this case, the yaml manifests are the program for Kubernetes to perform certain actions. But, as we are aware writing yaml manifests is mostly a manual, tedious, and error-prone process. Of course, there are tools like Helm and Kustomize built to package and manage Kubernetes configurations.
In this post, I wanted to introduce an exciting tool for developing yaml manifests leveraging Cuelang. The process with Cuelang allows developers to define and automate the creation of deployment manifests, which are essential for deploying and configuring applications in various Kubernetes environments. By using Cuelang, developers can write logic and define rules to generate YAML manifests dynamically. This approach can improve efficiency, maintainability, and consistency in deploying applications, especially in complex or large-scale systems. It enables developers to abstract away repetitive or error-prone manual configuration tasks and instead relies on programmatic generation based on predefined rules and templates.
Let's begin!!
What is Cuelang
Cuelang is a configuration language and runtime environment designed for defining and validating data schemas, and configuration files, and generating configuration artifacts. It aims to provide a concise and expressive syntax for configuration management and offers advanced features like type checking, constraints, and code generation.
Well, that might sound a bit mouthful. Let's dig a bit deeper to understand what it actually means and later we'll see its capabilities in generating Kubernetes manifests.
Among many features of the language, Cuelang includes a robust type system that allows developers to define and enforce data types for their configuration files. By performing type checking, Cuelang helps improve the reliability and stability of the configuration. It also allows developers to define constraints or rules that the configuration data must adhere to. By enforcing constraints, Cuelang helps ensure that the configuration adheres to specific business requirements or predefined rules, improving the quality and correctness of the configuration.
Apart from the above features, The most interesting and relevant to this post are data validation, code generation, and extraction which provide the ability to generate configuration artifacts or code, based on predefined templates and constraints. By leveraging data validation and code generation, Cuelang reduces errors in configurations and enhances productivity and maintainability, particularly in scenarios where configurations need to be generated for multiple environments or instances.
Getting started with CUE
To start working with CUE, we need to install it on our machine. We can refer to this instruction set from the official documentation and install it. Once installed, we can confirm with the following command:
santosh@~:$ cue version
cue version v0.5.0
go version go1.20.2
-buildmode exe
-compiler gc
CGO_ENABLED 1
GOARCH amd64
GOOS linux
GOAMD64 v1
Cue's JSON-Like Syntax: Familiarity and Power Combined
Like Yaml, CUE is a superset of JSON. It takes all the good thing JSON provides and then adds some extra features like comments with //, we can omit quotes in field names that does not include special characters like blank spaces, omit comma at the end of each field, etc. For example, the following is a snippet from CUE's official documentation showing the similarity between JSON and CUE:
CUE comes with the concept of defining the schema, refining the schema with constraints, and filling the data. You can read more about types CUElang supports. First, we define the schema:
# shcema.cue
album: {
name: strings
age: int
canDrive: bool
}
Then we can place particular constraints on the schema. Cuelang places special significance on a concept called constraints, which place rules or restrictions on values. For example:
#constraints.cue
album: {
name: string
age: >20
canDrive: true
}
In the above constraints, we can see that the age field has a constraint that the value should be more than twenty. Any value that violates this constraint will be rejected and result in an error.
Now, we fill in the values:
# data.cue
album: {
name: "jhon Doe"
age: 19
canDrive: true
}
To see how constraints can help us in matching the patterns we intend the value to be. We can use cue vet and pass the constraints.cue and data.cue files to it like so:
santosh@cue:$ cue vet data.cue constraints.cue
album.age: invalid value 19 (out of bound >20):
./constraints.cue:5:10
./data.cue:3:10
In CUE, there is no differentiation made between types and values. The term "type" simply denotes the nature or category of a value, which can encompass both tangible instances and more abstract representations.
We can make a value concrete by suffixing an asterisk, * before the value *20, which makes it the default value. There are also Disjunctions, denoted with a pipe - | symbol. Disjunctions allow the summing of types or values like so:
age: *21 | >20
`
The above snippet evaluates if the age is 21, if provided OR any number more than 20.
I invite you to play with all the features and capabilities of this exciting language by making your hands dirty and playing with the tutorials at the official docs
Validating Kubernetes Manifests with Cuelang
Cue provides an extensive validation engine. We can design a schema and validate our configurations against it. For example, the fields in .containers is a map of lists, which contains many properties. We can build a schema for .containers and validate our configurations against it. In order to verify our Deployment configurations, we need to import the templates from the Kubernetes Go sources locally like so:
go get k8s.io/api/apps/v1
Then we are going to import the Kubernetes definitions and use them with an import statement in our container.cue file like so:
package kube
# containers.cue
import (
apps_v1 "k8s.io/api/apps/v1"
)
apps_v1.#Deployment & {
spec: template: spec: containers: [{
name: "nginx"
image: "nginx:alpine"
securityContext: {
allowPrivilegeescalation: false # <-- Error in field 'allowPrivilegeescalation'
privileged: true
readOnlyRootFilesystem: true
}
ports: [{
name: "nginx"
protocol: "TCP"
containerPort: 80
}]
...
}]}
In the above example, I am trying to validate a containers list in the Deployment manifests at deployment.spec.template.spec.container. I we run cue fmt now. We should see an error:
spec.template.spec.containers.0.securityContext: 2 errors in empty disjunction:
spec.template.spec.containers.0.securityContext: conflicting values null and {allowPrivilegeescalation:false,privileged:true,readOnlyRootFilesystem:true} (mismatched types null and struct):
<snip>
spec.template.spec.containers.0.securityContext.allowPrivilegeescalation: field not allowed:
<snip>
./containers.cue:7:1
./containers.cue:12:4
From the above logs, we can see that the allowPrivilegeescalation: field not allowed: ( I had deliberately misspelled allowPrivilegeescalation instead of allowPrivilegeEscalation). Similarly, we can go ahead and add more fields relevant to the Deployment spec and validate it against the Kubernetes definitions.
After we rectify the error in the configuration. We can run cue eval -c containers.cue --out yaml and see that the valid YAML manifest for the containers section of the manifests is printed out on the STDOUT.
This validation of our Kubernetes configurations against upstream Kubernetes definitions will weed out many errors while writing our configurations and even help in generating configurations, which is yet another powerful feature of the CUE.
Generating Kubernetes Manifests with Cuelang
As DevOps engineers, we need to generate a lot of yaml files. by leveraging CUE, we can remove most of the boilerplate from the config manifests and make our life easier. For example in this tutorial for Kubernetes use cases, the containerPort, targetPort, and spec.selector fields are populated by leveraging the powerful code generation capabilities of CUElang.
I invite everyone to explore the CUELang official documents for in-depth features and capabilities of this new and exciting language.
Conclusion
In conclusion, Cuelang presents a powerful and intuitive language for defining Kubernetes configurations and managing complex deployment scenarios. With its unique features like structural schema, parameterization, and dependency management, Cuelang empowers engineers to streamline their workflows and enhance collaboration within Kubernetes environments. By leveraging Cuelang's capabilities, teams can effectively define, validate, and manage configurations, ensuring consistency and reliability across their deployments. As the Kubernetes ecosystem continues to evolve, Cuelang serves as an invaluable tool for simplifying and enhancing the management of containerized applications, making it an essential addition to any DevOps engineer's toolkit.