Running TestProject OpenSDK Tests With Github Actions

The previous parts of my TestProject OpenSDK tutorial described how you can write tests for web applications by using both TestProject OpenSDK and JUnit 5.

However, the problem is that automated tests aren't very useful if they aren't run on a regular basis. This blog post describes how you can solve this problem by using Github Actions.

After you have read this blog post, you:

• Understand why you should create a continuous integration (CI) pipeline that runs your tests.
• Know how you can get your TestProject API key.
• Can create a simple CI pipeline which runs your tests by using Github Actions.

Let's begin.

Why Should You Create a CI Pipeline?

Wikipedia defines the term continuous integration as follows:

In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day.

If you are using this approach, it's important that you follow these two rules:

• Before you can merge your code to the shared mainline, you must ensure that your code is working as expected. This rule is typically enforced by running automated tests and reviewing the code before it's merged to the shared mainline.
• After you have merged your code to the shared mainline, you must verify that the shared mainline is working as expected. This rule is typically enforced by running automated tests after someone has merged code to the shared mainline.

These rules are important because they ensure that the code that's found from the shared mainline is working and can be released at any time. Also, because the shared mainline is working, your team members don't have to waste any time for fixing broken code. Instead, they can concentrate on building new features.

Also, it's important to understand that even though these rules are clearly useful, following them isn't enough. For example, you can run automated tests manually before you merge code to the shared mainline and after you have merged code to the shared mainline. This approach has three problems:

• It's extremely irritating and slow to review code if you have to run tests before you can review the "production code". Unfortunately, this means that people might ignore the tests and review only the production code. In other words, the review process is unreliable and you cannot trust that the code which passes the review process is working.
• You cannot trust that the code found from the shared mainline is working because sooner or later someone merges broken code to it because he/she forgot to run the tests. In other words, if you have to be 100% sure that the code is working, you must run the tests yourself.
• Running tests manually is very boring and demotivating work. To make matters worse, the person who is running tests for the code found from the shared mainline cannot really do anything else because he/she must be ready to fix possible problems as soon as he/she notices them.

Luckily, the solution is quite obvious. Because no one should waste their time doing work that can be automated, you must create a CI pipeline which runs your tests. Because this blog post assumes that you are using Github Actions, your CI pipeline must run your tests when you create a new pull request or merge code to the main (former master) branch.

Before you can create a new CI pipeline by using Github Actions, you have to create a new TestProject API key. Next, you will find out how can do it.

Creating a New TestProject API Key

When you want to run TestProject OpenSDK tests by using Github Actions, you have to run the TestProject agent with Docker. When you start the Docker container which runs the TestProject agent, you must specify the TestProject API key which is used by the TestProject agent when it communicates with the TestProject platform.

You can create your TestProject API key by following these steps:

First, you have to start the create API key wizard by following these steps:

1. Log in to the app.testproject.io website.
2. Open the 'Integrations' page.
3. Open the 'API Keys' section.
4. Click the 'Create API Key' button.

The following figure illustrates this step:

Second, you have to configure the name of your API key. After you have configured the name of your API key, you can move on to the next step of the wizard.

The following figure illustrates this step:

Third, you have to specify the projects which can be accessed by using the created API key. You can either grant access to all projects or grant access only to the specified projects. After you are ready to create your API key, you have to click the 'Finish' link.

The following figure illustrates this step:

After you have created a new API key, the created API key is shown on the 'API Keys' page and you can copy it to the clipboard by clicking the 'Copy' link.

The following figure illustrates the layout of the 'API Keys' page:

After you have obtained your new TestProject API key, you have to figure out how you can pass environment variables to your Github Actions workflow. Let's move on and find out how you can solve this problem.

Creating the Required Github Actions Secrets

Secrets are basically encrypted environment variables which can be used in Github Actions workflows. Before you can create the Github Actions workflow that runs your tests, you must create two secrets:

• The TP_API_KEY secret contains your TestProject API key. The TestProject agent uses this API key when it communicates with the TestProject platform.
• The TP_DEV_TOKEN secret contains your TestProject developer token. As you remember, when you configure TestProject OpenSDK, you must set the TestProject developer token as the value of the TP_DEV_TOKEN environment variable.

When you want to create a new repository secret, you have to follow these steps:

1. Open the main page of your Github repository.
2. Open the 'Settings' page.
3. Open the 'Secrets' section.
4. Click the 'New repository secret' button which is found from the top right corner of the 'Secrets' page. This button opens a new page which allows you to create a new repository secret.

The following figure illustrates the layout of the 'Secrets' page:

Next, you will find out how you can create the Github Actions workflow that runs your tests.

Creating the Github Actions Workflow Which Runs Your Tests

You can create the Github Actions workflow which runs your tests by following these steps:

First, you have to create a YAML file which configures your Github Actions workflow and put this file to the /.github/workflows directory.

Second, you have to configure the name of your workflow. After you have done this, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

Third, you have to configure the events which trigger your workflow. Because you must create a CI pipeline which runs your tests when you create a new pull request or merge code to the main (former master) branch, you have ensure that your Github Actions workflow is triggered when you create a new pull request to the main branch or push code to the main branch.

After you have configured the events which trigger your Github Actions workflow, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

Fourth, you have to add a new job to your Github Actions workflow and ensure that this job is run by using the latest Ubuntu version which is supported by Github Actions.

After you have added this job to your Github Actions workflow, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

Fifth, you have to add the steps section to your workflow file. This section configures the steps of the job which runs your tests. After you have done this, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:

Sixth, you have to check out the code. This ensures that your Github Actions workflow can access the code. After you have added the required step to your Github Actions workflow, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

Seventh, you have to configure the JDK version (15) which is used by your workflow. After you have configured the JDK version, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Configure JDK version
        uses: actions/setup-java@v1
        with:
          java-version: 15

Eighth, you have to run the TestProject agent by following these steps:

1. Create a Docker compose file which runs the TestProject agent. The easiest way to do this is to use the Docker compose file that's provided by TestProject. If you use the file provided by TestProject, you can pass the TestProject API key to the started TestProject agent by using envsubst.
2. Store the value of the TP_API_KEY repository secret in the TP_API_KEY environment variable.
3. Replace the ${TP_API_KEY} variable found from the Docker compose file with the value of the TP_API_KEY environment variable.
4. Start the Docker container which runs the TestProject agent.

After you have created a new step which runs the TestProject agent, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Configure JDK version
        uses: actions/setup-java@v1
        with:
          java-version: 15
      - name: Run TestProject agent
        env:
          TP_API_KEY: ${{ secrets.TP_API_KEY }} 
        run: |
          envsubst < .github/ci/docker-compose.yml > docker-compose.yml
          docker-compose -f docker-compose.yml up -d

Ninth, you must ensure that your workflow waits until the TestProject agent is ready to run your tests.

When you want to get the status of a TestProject agent, you have to send a GET request to the url: http://[host]:[port]/api/status. When you send a GET request to the specified url, the TestProject agent returns the following JSON document:

{
	"installPath":"/Applications/TestProject Agent.app/Contents/Resources",
	"registered":true,
	"operatingSystem":"Mac OS X",
	"machineName":"Petris-MacBook-Pro.local",
	"dataPath":"/Users/loke/Library/Application Support/TestProject/Agent",
	"logevents":false,
	"authorized":true,
	"guid":"nA2o8Exc7UykZN-z6LQwLg",
	"alias":"Petri's Agent",
	"ipAddresses":["192.168.0.3"],
	"tag":"0.66.2",
	"state":"Idle",
	"id":585304,
	"storedIdentities":["hyuGRTfdsCVRscxRTEwesd"],
	"fsmState":"Idle"
}

When you want to ensure that your TestProject agent is ready to run your tests, you have to check the value of the registered attribute. When the value of this attribute is true, the TestProject agent can run your tests.

When you want to create a step which waits until the TestProject agent is ready to run your tests, you have to follow these steps:

1. Use the Wait for API Action.
2. Send a GET request to the url: http://localhost:8585/api/status.
3. Wait until the value of the registered attribute is true.
4. Send one HTTP request per second.
5. If the TestProject agent isn't ready to run your tests after 60 seconds have passed, continue the workflow.

After you have added a new step to your Github Actions worfklow, your workflow file looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Configure JDK version
        uses: actions/setup-java@v1
        with:
          java-version: 15
      - name: Run TestProject agent
        env:
          TP_API_KEY: ${{ secrets.TP_API_KEY }} 
        run: |
          envsubst < .github/ci/docker-compose.yml > docker-compose.yml
          docker-compose -f docker-compose.yml up -d
      - name: Sleep until the TestProject agent can run tests
        uses: mydea/action-wait-for-api@v1
        with:
          url: http://localhost:8585/api/status
          expected-response-field: registered
          expected-response-field-value: true
          interval: 1
          timeout: 60       

Tenth, you have to run your tests with Maven by following these steps:

1. Store the value of the TP_DEV_TOKEN repository secret in the TP_DEV_TOKEN environment variable. This ensures that TestProject OpenSDK can access your TestProject developer token.
2. Run the command: mvn clean test.
3. Configure the working directory which contains your POM file.

After you have added a new step to your workflow file, it looks as follows:

name: Run TestProject OpenSDK Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Configure JDK version
        uses: actions/setup-java@v1
        with:
          java-version: 15
      - name: Run TestProject agent
        env:
          TP_API_KEY: ${{ secrets.TP_API_KEY }} 
        run: |
          envsubst < .github/ci/docker-compose.yml > docker-compose.yml
          docker-compose -f docker-compose.yml up -d
      - name: Sleep until the TestProject agent can run tests
        uses: mydea/action-wait-for-api@v1
        with:
          url: http://localhost:8585/api/status
          expected-response-field: registered
          expected-response-field-value: true
          interval: 1
          timeout: 60       
      - name: Run tests with Maven
        env:
          TP_DEV_TOKEN: ${{secrets.TP_DEV_TOKEN}}
        run: mvn clean verify
        working-directory: opensdk/code-reuse    

Let's move on and find out how you can see the results of your Github Actions workflow on the Github UI.

Investigating the Results of Your Github Actions Workflow

When you create a new pull request, the main page of the pull request has a section which displays the results of the invoked Github Actions workflows. This section looks as follows:

If you want take a closer look at the checks which were done to your pull request, you have to open the 'Checks' tab by either clicking the 'Checks' or 'Show all checks' link. The 'Checks' tab has two areas:

• The area shown on the left side of the 'Checks' tab allows you to select the displayed check.
• The area shown on the right side of the 'Checks' tab displays the configuration and output of the selected check.

The following figure illustrates the layout of the 'Checks' tab:

When you merge your PR to the main branch, Github runs the 'Run TestProject OpenSDK Tests' workflow. If the tests fail, it will send you an email which looks as follows:

You can also take a closer at the execution history of your Github Actions workflows. If you want to do this, you have to click the 'Actions' link found from the main navigation menu of your Github repository.

The following figure identifies the location of this link:

The 'Actions' page allows you to select the workflow whose execution history is displayed on the 'Actions' page. If you want to take a closer look at a specific workflow run, you have to click the title of the workflow run.

The following figure illustrates the layout of the 'Actions' page:

You have now created a simple CI pipeline which runs your TestProject OpenSDK tests by using Github Actions, and you understand how you can take a closer look at the results of your Github Actions workflows. Let's summarize what you learned from this blog post.

Summary

This blog post has taught you five things:

• You must create a CI pipeline if you want to ensure that the code that’s found from the shared mainline is working and can be released at any time.
• When you start the Docker container which runs the TestProject agent, you must specify the TestProject API key which is used by the TestProject agent when it communicates with the TestProject platform.
• When you want to specify environment variables which can be used in your Github Actions workflows, you must use Github Actions secrets.
• When you want to create a new Github Actions workflow, you have to describe your workflow by using YAML.
• You have to put your workflow files to the /.github/workflows directory.

P.S. You can get the example workflow from Github.