An acyclic directed graph allows the various stages of the pipeline to be executed out of order, ignoring the sequencing of the stages and allowing the tasks to be linked directly to each oth
Recently, GitLab released a great feature that helps reduce pipeline execution time and provides more flexibility in the order of task execution. This functionality: the directed acyclic graph (DAG: Directed Acyclic Graph ) is now available for free on froggit.fr, and on the self-hosted version of GitLab.
Tasks and stages of the pipeline
In a typical CI / CD pipeline, there are multiple stages representing the automation of the DevOps process such as compilation, testing, packages, configuration and deployment. Each step consists of one or more tasks. In the CI / CD configuration file: .gitlab-ci.yml you define the order of your steps. Usually, the pipeline starts with the compilation tasks, once all these tasks are completed, the tests begin, then the next step, and so on.
While this order makes perfect sense, in some cases it can slow down execution time when considered as a whole. Imagine that the compilation step is made up of task A which takes a minute and task B which is very slow (say it takes 5 minutes). Task C belongs to the test step, but it only depends on task A. Task C will still have to wait 5 minutes before being executed, which represents a loss of 4 minutes.
Get to know the directed acyclic graph
DAG allows you to run pipeline stages out of order, breaking the sequencing of stages and allowing tasks to jump back to each other directly regardless of which stage they belong to.
With the DAG, tasks can start directly after the tasks on which they depend are completed, even if other tasks from the previous step are still in progress. This new functionality speeds up the CI / CD process and enables faster deployment.
In the example below, a project generates both Android, iOS, and web apps through a multi-stage pipeline. IOS testing begins as soon as the iOS compilation is complete instead of waiting for the Android and web builds to complete as well. The same goes for the iOS deployment: it launched after the iOS tests were finished, without waiting for the other tests to finish. The total compute time may be the same, but the actual elapsed time is different. In more complex cases, it is possible to reduce the actual elapsed time for the pipeline significantly by declaring which task is dependent on which another task.
Declaration of dependencies between tasks
The .gitlab-ci.yml file introduces a new keyword: needs which takes as parameter an array of tasks on which it depends.
ios:
stage: build
script:
- echo "build ios..."
ios_test:
stage: test
script:
- echo "test something..."
needs: ["ios"]The ios_test task , which is part of the test stage , will start immediately after the ios task which is in the build stage and it will run regardless of the status of other tasks in the build stage .
When is this useful?
This can be very useful under the increasingly popular mono repository model where you have multiple folders in your repository that can compile, test, and possibly even deploy independently, just like in our example above where iOS, Android and web apps can be compiled, tested and deployed individually.
Another use could be when your pipeline has very heavy tests taking a long time to run. It would make sense to run these tests as early as possible rather than waiting for unrelated tasks to complete before finally launching them.
