Creating Dynamic Test Environments with GitHub Actions

How to optimize a test pipeline so teams don’t run into concurrency problems when testing their features and modules is a subject that keeps coming up, both in stuff I’ve covered in the past and more recently.

I’ve actually given a few talks on the subject already, and I even have a sample repository using Azure DevOps as the CI tool. You can check out the slides and the video below!

So here’s the question: how do we let multiple dev teams test their features in a completely separate environment, quickly and easily?

The answer, of course, is containers. When you combine Kubernetes with Helm and a CI tool, you can do a lot of things dynamically. In this article, I’m going to update my previous talk and show the same application, but this time running on a GitHub Actions pipeline. To make the scenario more realistic, we’ll use Azure with Azure Kubernetes Service, pulling images from a private Azure Container Registry already privately integrated with the cluster. All the data will be stored in a Mongo-flavored CosmosDB.

Let’s go!

Before we start#

We’ll have to set up the environment before I can show you how the dynamic part works. Since that’s not really the point of this post, I’ll just leave the command references for what we’re doing here, but you can find all the documentation you need directly in each service’s docs.

First, you need an Azure account. Once you have one, install the Azure CLI, since we’re only going to use the command line.

The first command is az login, to log in to your account and pick which subscription you’ll use to create the resources. Once you’re logged in, let’s start by creating the first resource: the resource group.

1
az group create -l eastus -n ship-manager-pipeline

Now let’s create our ACR so we have somewhere to store our images:

1
az acr create -n shipmanager --sku Basic -g ship-manager-pipeline

Wait until the CR is created, then run the following command to enable login via username and password. That’s the only way our CI will be able to log in and build the images:

1
az acr update -n shipmanager --admin-enabled true

Now let’s move on to CosmosDB. A single command creates the whole structure:

1
az cosmosdb create --kind MongoDB -n ship-manager-db -g ship-manager-pipeline

This command takes a bit longer to run, so be patient. It’s worth saying that, as amazing as CosmosDB is, it’s not really recommended for this specific case of spinning up databases for ephemeral environments like these, since it’s more complex to tear down later. But to keep things simple, we’ll use it anyway, and we’ll look at alternatives to this approach later in the article.

Finally, let’s create our AKS, which ties everything together:

1
az aks create -n ship-manager -g ship-manager-pipeline \
2
--enable-addons http_application_routing \
3
--attach-acr shipmanager \
4
--vm-size Standard_B2s \
5
--generate-ssh-keys \
6
--node-count 2

This creates our AKS already attached to the ACR, so we don’t need to create a secret in every namespace with our Docker login file to pull images, and we don’t need to bind a service account either.

Grab the AKS credentials with the following command:

1
az aks get-credentials -n ship-manager -g ship-manager-pipeline --admin

Keep in mind you need kubectl installed on your machine for this command to work. If you don’t have it, run az aks install-cli to install it.

Creating the chart#

The first step in building the pipeline is figuring out how it’s going to work. To start, we’ll only deal with two environments: the first is production, the second is test.

The production environment gets published whenever a push with a v* tag happens. The test environment gets published on a push to any other branch that isn’t master or main (depending on the case).

So you can follow along, I’ve put together this sample repository, which has both the application code and the actions code.

Tip

If you want to follow along step by step, fork the repository, but don’t forget to remove the .github folder so the actions get removed too.

Before creating the pipeline files, let’s create the Helm files so we can build our chart! Create a folder called kubernetes at the root of the repository, then create a second folder called ship-manager.We could create the Helm chart automatically via the CLI, but it generates a bunch of files we don’t need, so let’s create it by hand to keep things simple.

Inside the ship-manager folder, create two more folders: templates and charts. Now create two files at the same level as the templates folder: one called Chart.yaml and the other values.yaml.

Now, go into the charts folder, create a backend folder inside it, and inside that one add a Chart.yaml file plus a templates folder.

The final structure should look like this:

1
kubernetes
2
└── ship-manager
3
    ├── Chart.yaml
4
    ├── charts
5
    │   └── backend
6
    │       ├── Chart.yaml
7
    │       └── templates
8
    │
9
    ├── templates
10
    └── values.yaml

In the Chart.yaml file inside the ship-manager folder, let’s write the following:

1
apiVersion: v2
2
name: ship-manager
3
description: Chart for the ship manager app
4
version: 0.1.0

And the one in the backend folder will look like this:

1
apiVersion: v2
2
name: backend
3
description: Chart for the backend part of the ship manager app
4
version: 0.1.0

What we just did was create Helm’s equivalent of a package.json: the file that defines the package we’re going to install on our cluster.

Helm works based on a hierarchy, and what we just created here is a dependency order. In other words, we just said that the frontend of the application, which lives in ship-manager, depends on a backend located in the charts folder. If we created another charts folder inside backend, we’d be saying the backend depends on that one, and so on. This way, with a single command, Helm installs all the dependencies in order for us.

Let’s create our first template. Create a frontend.yaml file inside the templates folder that’s inside ship-manager. This template is what actually gets created inside the cluster. In it, we’ll have all the Kubernetes resources, starting with the Deployment:

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  name: ship-manager-frontend
5
spec:
6
  replicas: 1
7
  selector:
8
    matchLabels:
9
      app: ship-manager-frontend
10
  template:
11
    metadata:
12
      labels:
13
        app: ship-manager-frontend
14
    spec:
15
      containers:
16
        - image: {{ required "Registry is required" .Values.global.registryName }}/{{ required "Image name is required" .Values.frontend.imageName }}:{{ required "Image tag is required" .Values.global.imageTag }}
17
          name: ship-manager-frontend
18
          resources:
19
            requests:
20
              cpu: 100m
21
              memory: 128Mi
22
            limits:
23
              cpu: 250m
24
              memory: 256Mi
25
          ports:
26
            - containerPort: 8080
27
              name: http
28
          volumeMounts:
29
            - name: config
30
              mountPath: /usr/src/app/dist/config.js
31
              subPath: config.js
32
      volumes:
33
        - name: config
34
          configMap:
35
            name: frontend-config

Notice we’re using Helm placeholders to mark the parts that can change, and that’s exactly what makes all of this possible. Creating environments and changing variables at CLI time, instead of after compiling, means we can pass whatever values we want to those variables when we create the environment.

Next, we have the rest of the configuration:

1
apiVersion: v1
2
kind: Service
3
metadata:
4
  name: ship-manager-frontend
5
spec:
6
  selector:
7
    app: ship-manager-frontend
8
  ports:
9
    - name: http
10
      port: 80
11
      targetPort: 8080
12
---
13
apiVersion: networking.k8s.io/v1beta1
14
kind: Ingress
15
metadata:
16
  name: ship-manager-frontend
17
  annotations:
18
    kubernetes.io/ingress.class: addon-http-application-routing
19
spec:
20
  rules:
21
    - host: {{ default "ship-manager-frontend" .Values.frontend.ingress.hostname }}.{{ .Values.global.dnsZone }}
22
      http:
23
        paths:
24
          - path: /
25
            backend:
26
              serviceName: ship-manager-frontend
27
              servicePort: http
28
---
29
apiVersion: v1
30
kind: ConfigMap
31
metadata:
32
  name: frontend-config
33
data:
34
  config.js: |
35
    const config = (() => {
36
      return {
37
        'VUE_APP_BACKEND_BASE_URL': 'http://{{ default "ship-manager-backend" .Values.backend.ingress.hostname }}.{{ .Values.global.dnsZone }}',
38
        'VUE_APP_PROJECT_VERSION': '{{ .Values.global.imageTag }}'
39
      }
40
    })()

Notice I’m pulling everything from .Values. That’s the values.yaml file we’ll get to shortly. Also notice that most of the things that can and should change, like the image name, the tag, the hostname, and the database, are variables too.

In these cases, using configmaps and secrets goes a long way toward keeping the pipeline simple.

The final file looks like this:

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  name: ship-manager-frontend
5
spec:
6
  replicas: 1
7
  selector:
8
    matchLabels:
9
      app: ship-manager-frontend
10
  template:
11
    metadata:
12
      labels:
13
        app: ship-manager-frontend
14
    spec:
15
      containers:
16
        - image: {{ required "Registry is required" .Values.global.registryName }}/{{ required "Image name is required" .Values.frontend.imageName }}:{{ required "Image tag is required" .Values.global.imageTag }}
17
          name: ship-manager-frontend
18
          resources:
19
            requests:
20
              cpu: 100m
21
              memory: 128Mi
22
            limits:
23
              cpu: 250m
24
              memory: 256Mi
25
          ports:
26
            - containerPort: 8080
27
              name: http
28
          volumeMounts:
29
            - name: config
30
              mountPath: /usr/src/app/dist/config.js
31
              subPath: config.js
32
      volumes:
33
        - name: config
34
          configMap:
35
            name: frontend-config
36
---
37
apiVersion: v1
38
kind: Service
39
metadata:
40
  name: ship-manager-frontend
41
spec:
42
  selector:
43
    app: ship-manager-frontend
44
  ports:
45
    - name: http
46
      port: 80
47
      targetPort: 8080
48
---
49
apiVersion: networking.k8s.io/v1beta1
50
kind: Ingress
51
metadata:
52
  name: ship-manager-frontend
53
  annotations:
54
    kubernetes.io/ingress.class: addon-http-application-routing
55
spec:
56
  rules:
57
    - host: {{ default "ship-manager-frontend" .Values.frontend.ingress.hostname }}.{{ .Values.global.dnsZone }}
58
      http:
59
        paths:
60
          - path: /
61
            backend:
62
              serviceName: ship-manager-frontend
63
              servicePort: http
64
---
65
apiVersion: v1
66
kind: ConfigMap
67
metadata:
68
  name: frontend-config
69
data:
70
  config.js: |
71
    const config = (() => {
72
      return {
73
        'VUE_APP_BACKEND_BASE_URL': 'http://{{ default "ship-manager-backend" .Values.backend.ingress.hostname }}.{{ .Values.global.dnsZone }}',
74
        'VUE_APP_PROJECT_VERSION': '{{ .Values.global.imageTag }}'
75
      }
76
    })()

Let’s do the same thing with the backend, creating a backend.yaml file in the charts/backend/templates folder:

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  name: ship-manager-backend
5
spec:
6
  replicas: 1
7
  selector:
8
    matchLabels:
9
      app: ship-manager-backend
10
  template:
11
    metadata:
12
      labels:
13
        app: ship-manager-backend
14
    spec:
15
      containers:
16
        - image: {{ required "Registry is required" .Values.global.registryName }}/{{ required "Image name is required" .Values.imageName }}:{{ required "Image tag is required" .Values.global.imageTag }}
17
          name: ship-manager-backend
18
          resources:
19
            requests:
20
              cpu: 100m
21
              memory: 128Mi
22
            limits:
23
              cpu: 250m
24
              memory: 256Mi
25
          ports:
26
            - containerPort: 3000
27
              name: http
28
          env:
29
            - name: DATABASE_MONGODB_URI
30
              valueFrom:
31
                secretKeyRef:
32
                  key: database_mongodb_uri
33
                  name: backend-db
34
            - name: DATABASE_MONGODB_DBNAME
35
              value: {{ default "ship-manager" .Values.global.dbName }}
36
---
37
apiVersion: v1
38
kind: Service
39
metadata:
40
  name: ship-manager-backend
41
spec:
42
  selector:
43
    app: ship-manager-backend
44
  ports:
45
    - name: http
46
      port: 80
47
      targetPort: 3000
48
---
49
apiVersion: networking.k8s.io/v1beta1
50
kind: Ingress
51
metadata:
52
  name: ship-manager-backend
53
  annotations:
54
    kubernetes.io/ingress.class: addon-http-application-routing
55
spec:
56
  rules:
57
    - host: {{ default "ship-manager-backend" .Values.ingress.hostname }}.{{ .Values.global.dnsZone }}
58
      http:
59
        paths:
60
          - path: /
61
            backend:
62
              serviceName: ship-manager-backend
63
              servicePort: http
64
---
65
apiVersion: v1
66
kind: Secret
67
metadata:
68
  name: backend-db
69
type: Opaque
70
stringData:
71
  database_mongodb_uri: {{ required "DB Connection is required" .Values.global.dbConn | quote }}

Notice I’m also using a few functions, like required, default, and quote. These are native Helm functions and they really save the day when you need more complex functionality.

The values file#

Just like the charts, the values.yaml file is based on a hierarchy of scopes. Take our structure as an example:

1
global:
2
  chave: # Accessible to all charts, both frontend and backend, as `.Values.global.chave`
3

4
backend:
5
  chave: # Accessible only to frontend and backend, but for frontend it's `.Values.backend.chave` and backend uses it as `.Values.chave`
6

7
frontend:
8
  chave: # Accessible by frontend as `.Values.frontend.chave`, but not by backend
9

10
chave: # Accessible only to frontend as `.Values.chave`

Notice there’s a scope break inside the values file. Keys that share a name with their dependent charts are only accessible to those charts and to higher-order charts. Since our frontend is the highest-order chart, it has access to all the values, while the backend only has access to the keys defined under backend:.

Also notice that inside backend the scope gets “leveled”, meaning the scope gets stripped from inside backend. So you don’t need to access the value as .Values.backend.chave if you’re inside the backend chart, just as .Values.chave.You can have more values files inside dependent charts and the rule stays the same, the difference is that the higher-order chart changes. That said, this pattern makes maintenance pretty complex.

Our values file needs the same keys we defined inside our templates, so they’ll look like this:

1
global:
2
  registryName:
3
  imageTag:
4
  dbName: ship-manager
5
  dbConn:
6
  dnsZone:
7

8
backend:
9
  imageName: ship-manager-backend
10
  ingress:
11
    hostname:
12

13
frontend:
14
  imageName: ship-manager-frontend
15
  ingress:
16
    hostname:

The keys I’m leaving blank are the ones that will get filled in either by the CLI or by the default functions.

Creating the pipeline#

To create the pipeline, we’ll go the manual route: create a .github folder, and inside it a workflows folder. The first workflow will be the simplest one, production.

Inside the workflows folder, let’s create a deploy-production.yml file (it can honestly be any name) and start by writing the name of our pipeline and the triggers that will make it run.

1
name: Build and push the tagged build to production
2

3
on:
4
  push:
5
    tags:
6
      - 'v*'

Here we’re saying our action runs on every push with a v* tag, meaning v1.0.0 and even vabc. If you want to narrow the possibilities down, you can use a regex like v[0-9]\.[0-9]\.[0-9].

Next, let’s create our first job and define a shared variable:

1
name: Build and push the tagged build to production
2

3
on:
4
  push:
5
    tags:
6
      - 'v*'
7

8
env:
9
  IMAGE_NAME: ship-manager
10

11
jobs:
12
  build_push_image:
13
    runs-on: ubuntu-20.04

We created a job called build_push_image that runs on Ubuntu 20, plus a shared variable that will be the image’s base name. Now let’s get to the actual action: let’s start creating our job’s steps, beginning with two super important ones:

1
name: Build and push the tagged build to production
2

3
on:
4
  push:
5
    tags:
6
      - 'v*'
7

8
env:
9
  IMAGE_NAME: ship-manager
10

11
jobs:
12
  build_push_image:
13
    runs-on: ubuntu-20.04
14

15
    steps:
16
      - uses: actions/checkout@v2
17

18
      - name: Set env
19
        id: tags
20
        run: echo tag=${GITHUB_REF#refs/tags/} >> $GITHUB_ENV

The first step is a checkout of our repository. It’s present in practically every action and it’s always the first step. The second one defines a second variable: the tag name.

By default, $GITHUB_REF is either the branch name or the tag name, like /refs/heads/main or /refs/tags/v1.0.0. We need to strip out the /refs/* part and keep only what’s left, so we’re using a shell substitution to add it to the global variables. Note that this variable only works inside this job.We can’t define the variable inside env because that key doesn’t run any kind of shell, so we can’t use value substitution or expansions there.

Now let’s build the Docker part of the pipeline: building and pushing the backend and frontend images.

1
name: Build and push the tagged build to production
2

3
on:
4
  push:
5
    tags:
6
      - 'v*'
7

8
env:
9
  IMAGE_NAME: ship-manager
10

11
jobs:
12
  build_push_image:
13
    runs-on: ubuntu-20.04
14

15
    steps:
16
      - uses: actions/checkout@v2
17

18
      - name: Set env
19
        id: tags
20
        run: echo tag=${GITHUB_REF#refs/tags/} >> $GITHUB_ENV
21

22
      - name: Set up Buildx
23
        uses: docker/setup-buildx-action@v1
24

25
      - name: Login to ACR
26
        uses: docker/login-action@v1
27
        with:
28
          # Username used to log in to a Docker registry. If not set then no login will occur
29
          username: ${{secrets.ACR_LOGIN }}
30
          # Password or personal access token used to log in to a Docker registry. If not set then no login will occur
31
          password: ${{secrets.ACR_PASSWORD }}
32
          # Server address of Docker registry. If not set then will default to Docker Hub
33
          registry: ${{ secrets.ACR_NAME }}
34

35
      - name: Build and push frontend image
36
        uses: docker/build-push-action@v2
37
        with:
38
          # Docker repository to tag the image with
39
          tags: ${{secrets.ACR_NAME}}/${{ env.IMAGE_NAME }}-frontend:latest,${{secrets.ACR_NAME}}/${{ env.IMAGE_NAME }}-frontend:${{env.tag}}
40
          labels: |
41
            image.revision=${{github.sha}}
42
            image.release=${{github.ref}}
43
          file: frontend/Dockerfile
44
          context: frontend
45
          push: true
46

47
      - name: Build and push backend image
48
        uses: docker/build-push-action@v2
49
        with:
50
          # Docker repository to tag the image with
51
          tags: ${{secrets.ACR_NAME}}/${{ env.IMAGE_NAME }}-backend:latest,${{secrets.ACR_NAME}}/${{ env.IMAGE_NAME }}-backend:${{env.tag}}
52
          labels: |
53
            image.revision=${{github.sha}}
54
            image.release=${{github.ref}}
55
          file: backend/Dockerfile
56
          context: backend
57
          push: true

There are 3 Docker actions in play here. The first sets up buildx, Docker’s image build utility. The second logs in to a registry, in our case our ACR, and here we hit our first secret, which we’ll create inside our repository: the ACR login data.

Finally, we’re building the image and pushing it, tagging it as latest plus the GitHub tag name. That way we know which images are the “production” ones and which are the test ones. We’re also adding two labels to each image: one has the revision, our commit’s sha, and the other has the tag name.

That wraps up our first job. The second one is the part where we deploy to the cluster. The beginning is the same, so I’ll omit the content up to this point so we can focus just on this part:

1
# start of the file
2
jobs:
3
  build_push_image:
4
    # image push job
5

6
  deploy:
7
      runs-on: ubuntu-20.04
8
      needs: build_push_image
9

10
      steps:
11
        - uses: actions/checkout@v2
12

13
        - name: Set env
14
          id: tags
15
          run: echo tag=${GITHUB_REF#refs/tags/} >> $GITHUB_ENV
16

17
        - name: Install Helm
18
          uses: Azure/setup-helm@v1
19
          with:
20
            version: v3.3.1

Notice we’re creating a relationship between the two jobs with the needs key. That means the second job only runs if the first one passes. We do the checkout, copy over the variable creation, then run a simple step to install Helm on the machine.

1
# start of the file
2
jobs:
3
  build_push_image:
4
    # image push job
5

6
  deploy:
7
      runs-on: ubuntu-20.04
8
      needs: build_push_image
9

10
      steps:
11
        - uses: actions/checkout@v2
12

13
        - name: Set env
14
          id: tags
15
          run: echo tag=${GITHUB_REF#refs/tags/} >> $GITHUB_ENV
16

17
        - name: Install Helm
18
          uses: Azure/setup-helm@v1
19
          with:
20
            version: v3.3.1
21

22
      - name: Get AKS Credentials
23
        uses: Azure/aks-set-context@v1
24
        with:
25
          creds: ${{ secrets.AZURE_CREDENTIALS }}
26
          # Resource group name
27
          resource-group: ship-manager-pipeline
28
          # AKS cluster name
29
          cluster-name: ship-manager
30

31
      - name: Run Helm Deploy
32
        run: |
33
          helm upgrade \
34
            ship-manager-prd \
35
            ./kubernetes/ship-manager \
36
            --install \
37
            --create-namespace \
38
            --namespace production \
39
            --set global.registryName=${{ secrets.ACR_NAME }} \
40
            --set global.dbConn="${{ secrets.DB_CONNECTION }}" \
41
            --set global.dnsZone=${{ secrets.DNS_NAME }} \
42
            --set global.imageTag=${{env.tag}}

Finally, we have the command to grab the Kubernetes credentials and deploy with Helm. Notice we’re deploying to a production namespace and setting the values.yaml values through --set flags. This makes everything easier when we need to tear things down, since we only need to delete the namespace and everything gets deleted with it.

The test pipeline is almost identical. The difference is we’re setting more variables, and we change the publish namespace and the trigger too. The other file, which I called deploy-test, looks like this:

1
name: Build and push the tagged build to test
2

3
on:
4
  push:
5
    branches-ignore:
6
      - 'main'
7
      - 'master'
8

9
env:
10
  IMAGE_NAME: ship-manager
11

12
jobs:
13
  build_push_image:
14
    runs-on: ubuntu-20.04
15

16
    steps:
17
      - uses: actions/checkout@v2
18

19
      - name: Set env
20
        id: tags
21
        run: echo tag=${GITHUB_REF#refs/heads/} >> $GITHUB_ENV
22

23
      - name: Set up Buildx
24
        uses: docker/setup-buildx-action@v1
25

26
      - name: Login to ACR
27
        uses: docker/login-action@v1
28
        with:
29
          # Username used to log in to a Docker registry. If not set then no login will occur
30
          username: ${{secrets.ACR_LOGIN }}
31
          # Password or personal access token used to log in to a Docker registry. If not set then no login will occur
32
          password: ${{secrets.ACR_PASSWORD }}
33
          # Server address of Docker registry. If not set then will default to Docker Hub
34
          registry: ${{ secrets.ACR_NAME }}
35

36
      - name: Build and push frontend image
37
        uses: docker/build-push-action@v2
38
        with:
39
          # Docker repository to tag the image with
40
          tags: ${{ secrets.ACR_NAME }}/${{ env.IMAGE_NAME }}-frontend:${{env.tag}}
41
          labels: |
42
            image.revision=${{github.sha}}
43
          file: frontend/Dockerfile
44
          context: frontend
45
          push: true
46

47
      - name: Build and push backend image
48
        uses: docker/build-push-action@v2
49
        with:
50
          # Docker repository to tag the image with
51
          tags: ${{ secrets.ACR_NAME }}/${{ env.IMAGE_NAME }}-backend:${{env.tag}}
52
          labels: |
53
            image.revision=${{github.sha}}
54
          file: backend/Dockerfile
55
          context: backend
56
          push: true
57

58
  deploy:
59
    runs-on: ubuntu-20.04
60
    needs: build_push_image
61

62
    steps:
63
      - uses: actions/checkout@v2
64

65
      - name: Set env
66
        id: tags
67
        run: echo tag=${GITHUB_REF#refs/tags/} >> $GITHUB_ENV
68

69
      - name: Install Helm
70
        uses: Azure/setup-helm@v1
71
        with:
72
          version: v3.3.1
73

74
      - name: Get AKS Credentials
75
        uses: Azure/aks-set-context@v1
76
        with:
77
          creds: ${{ secrets.AZURE_CREDENTIALS }}
78
          # Resource group name
79
          resource-group: ship-manager-pipeline
80
          # AKS cluster name
81
          cluster-name: ship-manager
82

83
      - name: Run Helm Deploy
84
        run: |
85
          helm upgrade \
86
            ship-manager-${{env.tag}} \
87
            ./kubernetes/ship-manager \
88
            --install \
89
            --create-namespace \
90
            --namespace test-${{env.tag}} \
91
            --set global.registryName=${{ secrets.ACR_NAME }} \
92
            --set global.dbConn="${{ secrets.DB_CONNECTION }}" \
93
            --set global.dbName=ship-manager-test-${{env.tag}} \
94
            --set global.dnsZone=${{ secrets.DNS_NAME }} \
95
            --set backend.ingress.hostname=ship-manager-backend-${{env.tag}} \
96
            --set frontend.ingress.hostname=ship-manager-frontend-${{env.tag}} \
97
            --set global.imageTag=${{env.tag}}

Secrets#

Now that we’ve got the pipelines created, let’s go to GitHub and create our secrets! Open the repository in your browser, navigate to the Settings tab and then secrets, then click New repository secret to create a new local secret.

Let’s create a secret called ACR_LOGIN, which is the name of our ACR: shipmanager. Another one called ACR_NAME, which isn’t really a secret since it’s just our CR’s DNS, but this way we avoid a hardcoded value in our action. This value is shipmanager.azurecr.io.Both pieces of information can be found in the Azure portal. Once you know the ACR name, the login is the same and the DNS is always <name>.azurecr.io

The ACR password can be grabbed with an AZ CLI command: az acr credential show -n shipmanager --query "passwords[0].value" -o tsv, and it should go into another secret called ACR_PASSWORD.

To get our AKS key, we’ll need service principal access on Azure, which you can get with the az ad sp create-for-rbac --sdk-auth command. This command returns a JSON: copy the entire JSON and paste it into the secret called AZURE_CREDENTIALS.

The database connection for the secret called DB_CONNECTION can also be grabbed with the command az cosmosdb keys list -n ship-manager-db -g ship-manager-pipeline --type connection-strings --query "connectionStrings[0].connectionString".

And the final secret can be obtained through a query on the list of enabled AKS add-ons. Since we turned on HTTP Application Routing, we get a DNS zone freed up for us, which we can grab with the command az aks show -n ship-manager -g ship-manager-pipeline --query "addonProfiles.httpApplicationRouting.config.HTTPApplicationRoutingZoneName and put in the secret called DNS_NAME.

Testing it out#

We commit our changes and now let’s create a tag with git tag -a v<version> -m'new version and then git push --tags to trigger our build. There’ll be a small delay, and then an output like this:

If we check out our application after a few minutes (DNS takes a while to propagate), we’ll see we have an address matching our ingress (you can get the frontend’s address with kubectl get ing -n production). Accessing it, we’ll have our application up and running:

Our application is on a production branch and will be accessible and updated to the latest version whenever we push with a specific tag. The same thing happens when we create a new branch and push. Go ahead and try creating any branch and pushing some code!

Conclusion and improvements#

Setting up a dynamic environment isn’t simple, but it can be the difference between a team that’s slow to test its features and a team that can be a lot more efficient. In this example we got to about 50% of what’s needed. The other important part of the pipeline is removing its resources whenever they’re no longer being used.

That’s why using a separate database instead of the same instance is a lot more preferable. The ideal setup would be creating a dependency in the backend chart for a completely empty MongoDB chart. That way we can be sure this environment is fully isolated, and we can tear down the whole environment with no problems.

Leave your comments and who knows, maybe we can continue this series!