--- marp: true --- # DUCT ``` Doing Unique CI Things ``` A pattern for creating per patch test environments, running end-to-end-tests, and reporting to GitLab or Gerrit ![bg right:40% 90%](images/low_poly_duct.png) --- # Stef Dunlap (she/her) staff software engineer in test at WMF, embedded on Abstract Wikipedia team | | Get in touch | |---------------------------------------------|--------------| | | sdunlap@wikimedia.org | | | KindRowboat (volunteer) | | | SDunlap-WMF (Wikimedia Staff) | | | https://tilde.town/~kindrobot | | | kindrobot@tiny.tilde.website | ![bg left:40% ](images/stef.jpg) --- # Problem Statement It's currently hard to automate creating per-patch test environments for Wikimedia extensions or services that integrate with other Wikimedia services ![bg right:40% 80%](uml/wikifunctions.png) --- # Previous failed approaches | Approach | Why didn't it work? | | ---- | ------------------- | | _The Pipeline_ (jenkins/zuul) | tests are designed to spin up and test one container per job | | [Patch demo](https://patchdemo.wmflabs.org/) | Cannot deploy custom/specified Wikimedia services | | Daily Beta Cluster selenium tests | shared tenant environment where we don't control the deploy schedule | ![bg right:30% 90%](images/sad_jenkins.svg) --- # DevEx Demo: (0) the page before ![height:600px](images/demo_0000_before.png) --- # DevEx Demo: (1) publish a patch for review ![height:600px](images/demo_0001_commit.png) --- # DevEx Demo: (1) publish a patch for review ![height:600px](images/demo_0002_commit.png) --- # DevEx Demo: (2) results ![height:600px](images/demo_0004_comment.png) --- # DevEx Demo: (3) visit environment ![height:600px](images/demo_0006_test-env.png) --- # DevEx Demo: (4) check pipeline ![height:600px](images/demo_0005_pipeline.png) --- # DevEx Demo: (5) check test results ![height:600px](images/demo_0007_test_results.png) --- # DUCT Component Diagram ![](uml/components.png) --- # DuctTape [DuctTape](https://gitlab.wikimedia.org/repos/abstract-wiki/aw-gerrit-gitlab-bridge) is the only custom code in DUCT. It's a rust app that runs on [Toolforge](https://toolforge.org), and is designed to be obsolete when Gerrit to GitLab migration completes ![height:450px](uml/components.png) --- # Component diagram after Gitlab migration ![height:350px](uml/components_2.png) After all of our gerrit repos [are moved to Gitlab](https://www.mediawiki.org/wiki/GitLab/Roadmap), Gitlab CI can interface directly with Gitlab, and there's no need for DuctTape. --- # Helm chart [Helm](https://helm.sh/) is a Kubernetes configuration and deployment tool. Your app may already have a production Helm chart that your can adopt for CI. You can also define tests to run against a deployment with `helm test`. [Wikifunction Helm chart for CI](https://gitlab.wikimedia.org/repos/abstract-wiki/aw-ci-chart) is on GitLab. ![bg right:50%](images/helm.png) --- # Relevant repos | name | description | |------|-------------| | [duct](https://gitlab.wikimedia.org/repos/abstract-wiki/duct) | this presentation, documentation | | [aw-ci-chart](https://gitlab.wikimedia.org/repos/abstract-wiki/aw-ci-chart) | the Helm chart for Wikifunctions CI environments | | [aw-e2e](https://gitlab.wikimedia.org/repos/abstract-wiki/aw-e2e) | the GitLab CI pipeline for deploying that ducttape interfaces with | --- # Thank you :D