Creating CI Pipelines with Tekton (Part 1/2)

In this blog post we're going to build a continuous integration (CI) pipeline with Tekton, an open-source framework for creating CI/CD pipelines in Kubernetes.

We're going to provision a local Kubernetes cluster via kind and install Tekton on it. After that we'll create a pipeline consisting of two steps which will run application unit tests, build a Docker image, and push it to DockerHub.

This is part 1 of 2 in which we will install Tekton and create a task that runs our application test. The second part is available here.

Creating the k8s cluster

We use kind to create a Kubernetes cluster for our Tekton installation: 

$ kind create cluster --name tekton

Installing Tekton

We can install Tekton by applying the release.yaml file from the latest release of the tektoncd/pipeline GitHub repo: 

$ kubectl apply -f https://github.com/tektoncd/pipeline/releases/download/v0.12.0/release.yaml

This will install Tekton into the tekton-pipelines namespace. We can check that the installation succeeded by listing the Pods in that namespace and making sure they're in Running state. 

$ kubectl get pods --namespace tekton-pipelines
NAME                                           READY   STATUS    RESTARTS   AGE
tekton-pipelines-controller-74848c44df-m42gf   1/1     Running   0          20s
tekton-pipelines-webhook-6f764dc8bf-zq44s      1/1     Running   0          19s

Setting up the Tekton CLI

Installing the CLI is optional but I found it to be more convenient than kubectl when managing Tekton resources. The examples later on will show both ways.

We can install it via Homebrew: 

$ brew tap tektoncd/tools
$ brew install tektoncd/tools/tektoncd-cli

$ tkn version
Client version: 0.9.0
Pipeline version: v0.12.0

Concepts

Tekton provides custom resource definitions (CRDs) for Kubernetes that can be used to define our Pipelines. In this tutorial we will use the following custom resources:

We will use the following two resources to define the execution of our Tasks and Pipeline:

For example, if we write a Task and want to test it we can execute it with a TaskRun. The same applies for a Pipeline: To execute a Pipeline we need to create a PipelineRun.

Application Code

In our example Pipeline we're going to use a Go application that simply prints the sum of two integers. You can find the application code, test, and Dockerfile in the src/ directory in this repo.

Creating our first task

Our first Task will run the application tests inside the cloned git repo. Create a file called 01-task-test.yaml with the following content: 

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: test
spec:
  resources:
    inputs:
      - name: repo
        type: git
  steps:
    - name: run-test
      image: golang:1.14-alpine
      workingDir: /workspace/repo/src
      command: ["go"]
      args: ["test"]

The resources: block defines the inputs that our task needs to execute its steps. Our step (named run-test) needs the cloned tekton-example git repository as an input and we can create this input with a PipelineResource.

Create a file called 02-pipelineresource.yaml: 

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: arthurk-tekton-example
spec:
  type: git
  params:
    - name: url
      value: https://github.com/arthurk/tekton-example
    - name: revision
      value: master

The git resource type will use git to clone the repo into the /workspace/$input_name directory everytime the Task is run. Since our input is named repo the code will be cloned to /workspace/repo. If our input would be named foobar it would be cloned into /workspace/foobar.

The next block in our Task (steps:) specifies the command to execute and the Docker image in which to run that command. We're going to use the golang Docker image as it already has Go installed.

For the go test command to run we need to change the directory. By default the command will run in the /workspace/repo directory but in our tekton-example repo the Go application is in the src directory. We do this by setting workingDir: /workspace/repo/src.

Next we specify the command to run (go test) but note that the command (go) and args (test) need to be defined separately in the YAML file.

Apply the Task and the PipelineResource with kubectl:

$ kubectl apply -f 01-task-test.yaml
task.tekton.dev/test created

$ kubectl apply -f 02-pipelineresource.yaml
pipelineresource.tekton.dev/arthurk-tekton-example created

Running our task

To run our Task we have to create a TaskRun that references the previously created Task and passes in all required inputs (PipelineResource).

Create a file called 03-taskrun.yaml with the following content: 

apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: testrun
spec:
  taskRef:
    name: test
  resources:
    inputs:
      - name: repo
        resourceRef:
          name: arthurk-tekton-example

This will take our Task (taskRef is a reference to our previously created task named test) with our tekton-example git repo as an input (resourceRef is a reference to our PipelineResource named arthurk-tekton-example) and execute it.

Apply the file with kubectl and then check the Pods and TaskRun resources. The Pod will go through the Init:0/2 and PodInitializing status and then succeed: 

$ kubectl apply -f 03-taskrun.yaml
pipelineresource.tekton.dev/arthurk-tekton-example created

$ kubectl get pods
NAME                READY   STATUS      RESTARTS   AGE
testrun-pod-pds5z   0/2     Completed   0          4m27s

$ kubectl get taskrun
NAME      SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
testrun   True        Succeeded   70s         57s

To see the output of the containers we can run the following command. Make sure to replace testrun-pod-pds5z with the the Pod name from the output above (it will be different for each run).

$ kubectl logs testrun-pod-pds5z --all-containers
{"level":"info","ts":1588477119.3692405,"caller":"git/git.go:136","msg":"Successfully cloned https://github.com/arthurk/tekton-example @ 301aeaa8f7fa6ec01218ba6c5ddf9095b24d5d98 (grafted, HEAD, origin/master) in path /workspace/repo"}
{"level":"info","ts":1588477119.4230678,"caller":"git/git.go:177","msg":"Successfully initialized and updated submodules in path /workspace/repo"}
PASS
ok  	_/workspace/repo/src	0.003s

Our tests passed and our task succeeded. Next we will use the Tekton CLI to see how we can make this whole process easier.

Using the Tekton CLI to run a Task

The Tekton CLI provides a faster and more convenient way to run Tasks.

Instead of manually writing a TaskRun manifest we can run the following command which takes our Task (named test), generates a TaskRun (with a random name) and shows its logs: 

$ tkn task start test --inputresource repo=arthurk-tekton-example --showlog
Taskrun started: test-run-8t46m
Waiting for logs to be available...
[git-source-arthurk-tekton-example-dqjfb] {"level":"info","ts":1588477372.740875,"caller":"git/git.go:136","msg":"Successfully cloned https://github.com/arthurk/tekton-example @ 301aeaa8f7fa6ec01218ba6c5ddf9095b24d5d98 (grafted, HEAD, origin/master) in path /workspace/repo"}
[git-source-arthurk-tekton-example-dqjfb] {"level":"info","ts":1588477372.7954974,"caller":"git/git.go:177","msg":"Successfully initialized and updated submodules in path /workspace/repo"}

[run-test] PASS
[run-test] ok  	_/workspace/repo/src	0.006s

Conclusion

We have successfully installed Tekton on a local Kubernetes cluster, defined a Task, and tested it by creating a TaskRun via YAML manifest as well as the Tekton CLI tkn.

All example code is available here.

In the next part we're going to create a task that will use Kaniko to build a Docker image for our application and then push it to DockerHub. We will then create a Pipeline that runs both of our tasks sequentially (run application tests, build and push).

Part 2 is available here.