Add lastly duties to Tekton pipelines – IBM Developer

Tekton Pipeline 0.14 launched the lastly clause within the Pipeline specification. This part is right for operating something simply earlier than exiting the pipeline, comparable to cleansing up any acquired sources, sending notifications, rolling again deployments, and extra.

As outlined on the Tekton web site, the lastly part takes an inventory of a number of duties which can be all executed in parallel in spite of everything duties are completed executing — no matter a hit or a failure. The next code reveals the lastly part of a pipeline.

apiVersion: tekton.dev/v1beta1
type: Pipeline
metadata:
  identify:build-deploy-cleaup-pipeline
spec:
  params:
    - identify: api-url
    - identify: cloud-region
  duties:
    - identify: clone
       taskRef:
         identify: git-clone
    - identify: construct
       taskRef:
         identify: construct
       runAfter:
         - clone
    - identify: deploy
       taskRef:
         identify: deploy
       runAfter:
         - construct
  lastly:
    - identify: cleanup
       taskRef:
         identify: cleanup

Additionally, the execution standing of a lastly job impacts the general pipelineRun standing. A pipelineRun is said failed if a number of lastly duties fail.

The neighborhood has applied many options over the previous few releases to assist design lastly duties for frequent use circumstances.

Use circumstances for lastly

A Tekton Pipeline is applied as a Kubernetes controller and executes a group of pods in your Kubernetes cluster. You may configure a pipeline utilizing a directed acyclic graph (DAG) and a set of lastly duties primarily based in your steady integration and steady supply (CI/CD) workflow. The commonest workflow for CI/CD is build-test-deploy, which we lengthen right here to incorporate lastly situations.

Ship notification

As the next picture reveals, a CI/CD pipeline defines a lastly job named send-notification that references send-to-channel-slack and/or sendmail to inform the group and sends a hit or failure notification relying on the execution standing of the construct course of. The construct course of contains construct and check duties within the duties part of the pipeline.

Cleanup sources

A few cleanup situations:

• A pipeline creates a challenge or acquires sources and sends the identify of the challenge or sources to the lastly job. The lastly job can then clear up these challenge sources no matter a failure within the duties part, as proven within the following picture:

  

• A pipeline provisions a brand new Kubernetes cluster. The pipeline begins executing and stops for some purpose. The pipeline has outlined lastly duties to unencumber the Kubernetes cluster. Although the pipelineRun stops, the lastly part executes to unencumber the sources.

Terminate slightly than cancel

You might be operating a pipeline that executes just a few Extract, Remodel, and Load (ETL) processes in a sequence, as proven within the following picture. Due to useful resource constraints, you wish to cancel scheduling any additional processes however compelete executing the present operating course of and analyze the outcomes from that course of, as proven on this picture:

These are the most typical use circumstances that require the performance of lastly or an exit handler in a pipeline. Subsequent, let’s have a look at the checklist of the options that we launched to perform these use circumstances.

Use lastly with if

The lastly duties are assured to execute earlier than the pipelineRun exits. Nonetheless, there are use circumstances the place the pipeline wants the pliability to not run a lastly job primarily based on a sure situation (for instance, the person needs to ship a Slack notification provided that a construct fails). We’ve got applied two options to make this use case doable: TEP-0045 and TEP-0028 have the main points. The general thought right here is to allow specifying when expressions in a lastly job and supply a way to entry the execution standing of a job in a lastly job at runtime so as to consider specified when expressions. You may entry an execution standing of a job utilizing a context variable $(duties.<task-name>.standing):

spec:
  pipelineSpec:
    duties:
      - identify: golang-build
        taskRef:
          identify: golang-build
      - identify: unit-test
        taskRef:
          identify: golang-unit
        runAfter: [ golang-build ]
      - identify: deploy
        taskRef:
          identify: deploy
        runAfter: [ unit-test ]
    lastly:
      - identify: notify-build-failure
        when:
          - enter: $(duties.golang-build.standing)
            operator: in
            values: ["Failed"]
        taskRef:
          identify: send-to-slack-channel

Now, as an alternative of notifying just for a construct failure, you wish to ship a Slack notification for any failure. TEP-0049 implements accessing an mixture standing of the duties part in lastly utilizing a context variable $(duties.standing):

spec:
  pipelineSpec:
    ...
    lastly:
      - identify: notify-any-failure
        when:
          - enter: $(duties.standing)
            operator: in
            values: ["Failed"]
        taskRef:
          identify: send-to-slack-channel

Join duties and at last sections

In case you are a pipeline creator, now you can ship the execution outcomes of a job to any lastly job in order that the challenge sources created/acquired by the pipeline are assured to be cleaned up by the lastly job. TEP-0004 explains the design particulars of this characteristic.

spec:
 duties:
   - identify: purchase
     taskRef:
       identify: purchase
 lastly:
   - identify: launch
     taskRef:
       identify: launch
     params:
       - identify: leased-resource
         worth:  $(duties.purchase.outcomes.leased-resource)

Gracefully terminate a pipeline

Rafal Bigraj spent a number of effort in designing and implementing a characteristic (TEP-0058), which made one of many frequent machine studying use circumstances doable utilizing Tekton.

We applied an answer to allow swish termination (cancellation) of a pipeline the place the present operating duties are cancelled or terminated and the cleanup is scheduled. When a pipelineRun is executing, you’ll be able to replace its definition to cancel the duties, which deletes all of the respective pods and schedules the lastly part.

apiVersion: tekton.dev/v1beta1
type: PipelineRun
metadata:
  identify: etl-processes
spec:
  standing: "CancelledRunFinally"

We applied yet one more resolution the place the present operating duties should not interrupted till they’re completed and no new duties are scheduled. After the present operating duties end, the lastly part is executed to research the outcomes from already executed duties.

apiVersion: tekton.dev/v1beta1
type: PipelineRun
metadata:
  identify: etl-processes
spec:
  standing: "StoppedRunFinally"

Now you know the way to design a lastly part of a pipeline to implement varied use circumstances. When you’ve got a use case or want a brand new characteristic in Tekton, attain out to the Tekton neighborhood over Slack or open a dialogue within the tektoncd/pipeline repo.