Automation Testing Guide for Startups - Level 3: Integrate With Your CI/CD

If you followed our level 1 and level 2 of this startup guide, you already know how to create and maintain hundreds of automated tests for your web app. In this episode, we will show you how to introduce automated regression testing into your software development process. Integration with CI/CD is a professional way to achieve high-quality code and prevent bugs before they are released to customers.

What will you achieve on this level?

Once you finished this level you will:

• collaborate with all team members on test automation
• run automated regression tests with every build
• see test results directly in your CI/CD tool

⚠️ Technical skills required!

To achieve this level you need to have software engineering knowledge and experience with development tools. If you’re not a technical person, ask your developers for help.

TLDR:

• Use BugBug CLI tool to integrate with your CI/CD tool
• Run tests every time an environment is deployed
• Invite all your team members to BugBug, so that anyone can view test results
• Copy & paste links to test results and share them with colleagues

Run tests with every deployment

Your goal is to make sure that every time someone makes changes to the code, this new code is automatically checked against bugs before releasing to clients. If you use Github, Gitlab, Bitbucket or similar, you can configure these tools to run a “build” every time someone merges new code to a specific git branch. Our goal is to make sure that every time the build is run it will also trigger an automated regression tests suite. If the suite fails, the build also fails. If the build failed, you should not merge the code to master. This approach lowers the risk of introducing bugs to production.

Check the ID of test suites

Simply go to “Suites”, and select “Run via CLI”. You will see terminal commands that you can copy and paste into your CI/CD scripts.

Integrate test suites with your existing CI/CD pipeline

You probably already have a build script. It’s very easy to initiate BugBug tests from it. You just need to add a couple of lines of code that will trigger the tests and wait for the results. Learn more in the documentation of BugBug CLI and ask your DevOps for help if needed.

Once you successfully integrate your build scripts with BugBug, you are able to see the tests results directly in your build status.

Share links to test results

You noticed that a test failed and you wonder how can you send the report to your colleague? The easiest way is to simply copy the URL and send it via chat. Each item from the “Runs history” has a unique URL address.

To access it the recipient needs to be invited to your organization. BugBug allows you to invite an unlimited number of people to your organization free of charge. Testing automation is more powerful if you make it available to the whole team!

Don’t forget about "Level 2" recommendations

CI/CD integration doesn’t make sense if your tests are not resilient to changes and easy to maintain. See our previous tips in the level 2 test automation guide, where we covered topics such as custom attributes, reusable components and variables.