![holiday-logo-wide (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/holiday_logo_wide_1_3b4d04928e.png)



## What is Holiday Park? 
Holiday Park is the biggest chain of resorts on the Polish side of the Baltic Sea. The chain offers stays in luxurious chalets and apartments for families on holidays with children. Located just a few steps from the sea, the resorts offer relaxation and comfort to kids and their parents alike. There are a number of activities at Holiday Park, such as swimming pools, playgrounds, beach courts, game rooms, animation for children, and sport classes for guests.

## **Problem:** bugs prevented users from booking
The company created an original booking system for luxury resorts located throughout Poland. As time went on, the simple booking system was significantly expanded, causing the **existing manual testing to become inefficient**. With a limited IT team, the company began looking for a solution that would **reduce the amount of work required for testing**; in turn, a suitable tool was chosen. It has happened that **undetected errors prevented reservations** for accommodation, negatively affecting the business as well as the company's image.

## **Solution:** Reducing costs with BugBug
In order to reduce the IT team's workload, the company was looking for a solution that would stabilize the testing of holiday booking systems. One solution was to use Protractor's framework. Yet the costs and workload associated with creating and maintaining tests exceed the expected figures. 

**The simplicity and rapidity of BugBug.io led to its selection.**

Another reason was **the possibility of adapting the tests** to the developments in the tested application. Holiday Park was particularly interested in **connecting the tests to the internal software development process** (pipeline) - and that’s also enabled by BugBug.

As a comprehensive SaaS tool enabling users to easily create and sustain regression tests, **BugBug.io was a perfect fit for Holiday Park’s needs**.

And in the words of Marcin Baran, team leader at Holiday Park, BugBug is additionally: *“a very user-friendly and useful tool.”*


## How BugBug resolved the issue
Using BugBug, **fundamental tests of a booking system were created in only a few minutes**. The test recorder turned out to **work flawlessly**. 

Then, very quickly, more complicated tests of a multistage booking form were created. The BugBug team recommended that the tests be prepared using reusable components to **make upkeep easier and more efficient**. Consequently, a graph showing the whole project's tested business path was produced. Testing is associated with a process in which they can be run in cloud storage after every release of a new version of the holidaypark.pl application. As a result of this feature, **the company does not have to maintain its own IT infrastructure for this specific purpose**.

Marcin, the IT Team Leader at Holiday Park, appreciates **how easy it is to create and set up BugBug**.

According to him, it is essential that** tests are quickly adapted to changes** made in the application. As well as being able to re-record the steps at any stage of the existing tests, a visual editor can be used to aid in this process.

![hp-cs-img2.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/hp_cs_img2_dfa1aebe5b.png)


### Now it’s your turn
**BugBug.io can help your brand, too**. Facilitate regression testing with our solution, saving time and resources. Sign up for free!

thumbnail_holiday-park-resort-logo-s (1).png

holiday-park-resort-logo-s (1).png

thumbnail_holidaypark-grayscale300x140.jpg

holidaypark-grayscale300x140.jpg

Reducing costs of regression testing of a hotel booking platform

holiday-park

![brand-24-logo-wide (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/brand_24_logo_wide_1_3dc557fb31.png)




In brief:
---------

-   Brand24 is a popular SaaS media monitoring platform

-   They use BugBug to check every hour if their website works properly

-   The Product Manager implemented the test automation all by himself, without software developers

-   The cost of implementation was significantly lower than Selenium

-   BugBug has already detected more than 5 real issues and alerted the Brand24 team
<br/><br/>

* * * * *




**Challenge:**\
Brand24 needed a reliable way to ensure their website always works properly
---------------------------------------------------------------------------------------

Brand24 is a Software-as-a-Service startup that provides a platform for online reputation management & media monitoring. They have clients in more than 110 countries, including Intel, Stanford University and Twitch. They rely heavily on web technologies.

### Bugs that are detected too late are a waste of time and money

"Each minute of our website being down costs us not only money but also reputation." - says Krzysztof Studniarek, Product Owner/Manager of Brand24. Imagine spending money on a marketing campaign that leads to your blog: if the blog is down, you just waste your budget and lose a potential customer.

### Traditional WWW monitoring? Not good enough

Brand24 already tried a classic SEO monitoring tool that checks if specific URLs work. "These tools didn't detect real issues, for example, when the page loads partially, without CSS. We wanted to ensure that the complete user paths are working correctly." 

### Selenium? Too expensive

Brand24 also considered using Selenium. However, it requires coding skills and the involvement of software developers. "We realized that setting up Selenium would not be profitable. We needed to find an alternative."

![brand-24-screen (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/brand_24_screen_1_ea71f0255d.png)



**Solution:**\
Automated browser tests with BugBug, executed every hour
-------------------------------------------------------------------

### Fast implementation

Brand24 created their first automated tests using BugBug in a matter of minutes, without having to write any code. "I used [BugBug tests recording](https://docs.bugbug.io/creating-tests/recording-the-tests#record-the-first-test), and it worked surprisingly well," says Krzysztof, the Product Manager. "Since April 2021, we run these tests every hour to check if Brand24 website works as it should".

### Reduced costs

Compared to Selenium, BugBug showed a dramatically lower overall cost. Brand24 implemented hourly monitoring without the involvement of the IT department. Krzysztof, the Product Manager with a technical background, simply did it by himself! 

### Productive collaboration

BugBug allowed Brand24 to share responsibilities on test maintenance with other team members. They currently have 8 active users who receive hourly notifications about the website status.

### Cost-efficient maintenance

Since the initial setup, the only recurring cost for Brand24 is the monthly premium subscription fee ([see pricing](http://bugbug.io/pricing)). The maintenance is also simple, thanks to BugBug's priority customer support. "BugBug experts proactively help me with fixing tests. They also provide tips on how to make them more stable."

![b24_casestudy_p9z4rhj7a7osqgtaui3ty08ctn85l22tkg9d5upb8w_0779b13eda (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/b24_casestudy_p9z4rhj7a7osqgtaui3ty08ctn85l22tkg9d5upb8w_0779b13eda_1_6f39ea1dcb.png)


**Impact:**\
BugBug successfully detected several critical issues and immediately notified Brand24 
-----------------------------------------------------------------------------------------------

Since April 2021, BugBug has run the hourly monitoring more than 100'000 times and detected the following issues:

1.  **Prevented slow page loading caused by a third-party plugin**\
    BugBug Support noticed that some tests began to work slower than expected. After the investigation, it turned out that an analytics library caused a significant lag on the website. In consequence, Brand24 removed the library.

1.  **Detected issues with login after a failed release**\
    BugBug detected that Brand24 users could not log in after a failed release

1.  **Detected that the website is down**\
    Brand24 received an alert from BugBug, that multiple automated tests failed. They quickly reacted and fixed the problem in a matter of minutes.

1.  **Detected that the pricing page is empty**\
    BugBug notified Brand24 that the pricing page has an issue. BugBug took a screenshot that the page was blank.

![failed-test.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/failed_test_e6b0cedba0.png)

-----------------------------------------------------------------------------------------------
<br/><br/>
BugBug provided a meaningful business value, without complexity, and without high costs. If you want to learn more how BugBug can help your business, [start a free trial](https://app.bugbug.io/sign-up/) or [book a demo](https://bugbug.io/book-a-demo/).

thumbnail_brand-24-logo-s (1).png

brand-24-logo-s (1).png

b24-logo-1.png

How a popular SaaS set up test automation without engineers

brand24

Customer Success Stories

bugbug_logo_1.png

bugbug_logo_5.png

bugbug_logo_4.png

bugbug_logo_3.png

<div class="cms-section">
      <div class="cms-wrapper">
        <div class="cms-testimonials-row">
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">“Recording tests in BugBug works surprisingly well. I've tested various competing solutions, but none did as smoothly as BugBug.”</div>
              <div class="cms-testimonials-item__author">— Krzysztof Studniarek, Product Manager, BRAND24</div>
            </div>
          </div>
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">“In two days we managed to automate test cases that took us weeks to write up and execute using other software. It is easy to use, yet can still perform advanced testing should the tester want to.”</div>
              <div class="cms-testimonials-item__author">— Panayiotis, Software Tester, ATM Marketing</div>
            </div>
          </div>
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">"Very good and fast support from the BugBug team. Have clear documentation which really helps you to get started. Features like scheduling test cases and getting results are perfect for our needs."</div>
              <div class="cms-testimonials-item__author">— Imalka, QA Engineer, Calcey.com</div>
            </div>
          </div>
        </div>
        <div class="cms-testimonials-row">
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">"BugBug is super intuitive and lowers the barrier of becoming an end-to-end tester. It reduces development time and significantly lowers the cost of software testing. "</div>
              <div class="cms-testimonials-item__author">— Wojciech, CEO Davinci-Studio.com</div>
            </div>
          </div>
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">"We always wanted to automate, but didn't have resources. BugBug empowered us to do so. Very simple, quick to use, and looks really cool!"</div>
              <div class="cms-testimonials-item__author">— Kris, CEO, Innential</div>
            </div>
          </div>
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">"It works great! BugBug saves me hours of monotonous work writing and maintaining tests."</div>
              <div class="cms-testimonials-item__author">— Martin, Python developer</div>
            </div>
          </div>
        </div>
        <div class="cms-testimonials-row">
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">"BugBug is a great choice for our business because it is easily accessible and intuitive. In the free plan, it offers tools that meet our daily needs."</div>
              <div class="cms-testimonials-item__author">— Mariana Barbosa, Azzo Tecnologia</div>
            </div>
          </div>
          <div class="cms-testimonials-item">
            <div class="cms-testimonials-item__icon"><img src="https://assets.website-files.com/62826448121477c0eb54a773/633456764c0c618517865056_bugbug-quote-icon.png" loading="lazy" alt="" class="cms-testimonials-item__icon-img"></div>
            <div class="cms-testimonials-item__texts">
              <div class="cms-testimonials-item__main-text">“A very interesting software testing tool that allows you to run automated tests without any programming knowledge.”</div>
              <div class="cms-testimonials-item__author">— Michał, QA Engineer</div>
            </div>
          </div>
        </div>
      </div>
    </div>

<style>

.cms-testimonials-row {
  display: -webkit-box;
  display: -webkit-flex;
  display: -ms-flexbox;
  display: flex;
  padding-bottom: 32px;
  grid-column-gap: 32px;
  grid-row-gap: 32px;
}

.cms-testimonials-item {
  display: -webkit-box;
  display: -webkit-flex;
  display: -ms-flexbox;
  display: flex;
  -webkit-box-align: start;
  -webkit-align-items: flex-start;
  -ms-flex-align: start;
  align-items: flex-start;
  -webkit-box-flex: 0;
  -webkit-flex: 0 33.33%;
  -ms-flex: 0 33.33%;
  flex: 0 33.33%;
  grid-column-gap: 16px;
}

.cms-testimonials-item__main-text {
  min-height: 100px;
  padding-bottom: 16px;
  color: #666;
  font-size: 18px;
  line-height: 1.6em;
  font-style: italic;
  letter-spacing: 0em;
}

.cms-testimonials-item__icon {
  width: 64px;
  height: 64px;
  -webkit-box-flex: 0;
  -webkit-flex: 0 0 auto;
  -ms-flex: 0 0 auto;
  flex: 0 0 auto;
}

.cms-testimonials-item__author {
  min-height: 100px;
  color: #666;
  font-size: 16px;
  line-height: 1.6em;
  text-align: right;
  letter-spacing: 0em;
}

@media screen and (max-width: 991px) {
  .cms-testimonials-row {
    -webkit-flex-wrap: wrap;
    -ms-flex-wrap: wrap;
    flex-wrap: wrap;
  }

  .cms-testimonials-item {
    -webkit-flex-basis: 100%;
    -ms-flex-preferred-size: 100%;
    flex-basis: 100%;
  }
}


@media screen and (max-width: 479px) {
  .cms-testimonials-item__main-text {
    min-height: 0px;
    font-size: 18px;
  }

  .cms-testimonials-item__author {
    min-height: 0px;
    font-size: 13px;
  }
}
</style>


Case studies

Features

Pricing

Roadmap

Privacy policy

Terms of service

About us

Book a demo

Contact

Get more tips and product related content. Zero spam.

Don't miss any updates

End-to-end testing made simple! BugBug is a new, more intuitive tool for test automation of web apps. Get all the benefits of test automation without hiring new people or infrastructure changes.

BugBug.io | Intuitive and reliable test automation for startups

large_Open Graph 1200x630 3.png

medium_Open Graph 1200x630 3.png

small_Open Graph 1200x630 3.png

thumbnail_Open Graph 1200x630 3.png

Open Graph 1200x630 3.png

Blog about test automation

Guides to automation

small_pawel-about-512.png

thumbnail_pawel-about-512.png

pawel-about-512.png

Paweł Bylina

Are you a tech leader, CEO or CTO in a tech startup? You want to implement automation testing for your SaaS product? We've written this guide especially for you. It's packed with actionable tips on how to approach test automation in an agile way.

We're going to tackle the following questions:

-   What is the business goal of automation testing?

-   How to implement automation incrementally?

-   What resources do you need to develop automated tests?

-   What are typical problems and obstacles and how to overcome them?


The strategic approach to automation testing
--------------------------------------------

Automation testing is a very broad term. There is no single goal of automation. Depending on the level of automation you will get different benefits for your organization.

*Figure 1: Strategic matrix of different goals of test automation*
![bugbug-goals-of-automation-testing.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_goals_of_automation_testing_9308b9cb1b.png)

See this strategic matrix above. You can achieve all of these goals, but not all at once. You should pick the right order of goals, starting with the ones that are low-effort and high-value. 

To achieve each of these goals, you need a different combination of resources and skills. We'll guide you through each of the goals, explaining the approach, time & resources and how to achieve it with BugBug.

Step 1: low cost & high impact goals
------------------------------------

### **Goal:**

-   Prevent loss of business due to production downtimes

### **Approach**

-   You don't need to hire new people, you can do it yourself

-   Create 5-10 automated tests and run them every hour to monitor your production, for things like login, landing pages, registration form, contact form, basic app navigation

### **Time & Resources**

-   2-4 hours of your own time

### **BugBug features that you should use**

-   [Recording tests](https://docs.bugbug.io/creating-tests/recording-the-tests)

-   [Cloud runs](https://docs.bugbug.io/running-tests/in-cloud-on-server)

-   [Schedules](https://docs.bugbug.io/running-tests/schedules)

-   [Notifications](https://docs.bugbug.io/collaboration/notifications)

-   [Re-recording test steps](https://docs.bugbug.io/recording-tests-steps/re-recording-steps)

Step 2: low hanging fruit
-------------------------

### **Goal:**

-   Improve team collaboration on quality assurance

### **Approach**

-   Give all the scrum teams access to the testing platform

-   Share direct links to failed tests in your tickets or on a chat

-   Configure who gets notifications

### **Time & Resources**

-   Inviting people only takes 5 - 10 minutes, but you should spend 2-4 hours on communicating its purpose

### **BugBug features that you should use**

-   [Invite users](https://docs.bugbug.io/collaboration/inviting-team-members)

-   [Project settings > Notifications](https://docs.bugbug.io/collaboration/notifications#choose-when-people-get-email-notifications)

Step 3: incremental development of automated tests
--------------------------------------------------

### **Goals**

-   Save time on manual regression testing

-   Release faster

-   Prevent cost of regression bugs

### **Approach**

-   Talk to the whole team about what you want to achieve and what's the incremental plan

-   Choose 1-2 people responsible for automated tests 

-   Create dedicated test user accounts that will be used for automation only

-   Run automation on some testing environment (staging)

-   Create separate test suites for staging and prod (run all tests on staging, but only essential smoke tests on prod)

-   Add more tests incrementally, 2-3 test cases per week (after 3 months you should have about 50 test cases automated)

-   Start adding [data-test-id attributes](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors) in your HTML

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [New test from here](https://docs.bugbug.io/creating-tests/new-test-from-here)

-   [Duplicating tests](https://docs.bugbug.io/creating-tests/duplicating-tests)

-   [Components](https://docs.bugbug.io/editing-tests/wip-make-component)

-   [Variables](https://docs.bugbug.io/editing-tests/variables)

-   [Profiles](https://docs.bugbug.io/editing-tests/variables#profiles) to [test on different environments](https://docs.bugbug.io/editing-tests/variables#work-with-different-development-environments)

-   [Suites](https://docs.bugbug.io/organizing-tests/suites)

-   [Manually editing steps](https://docs.bugbug.io/editing-tests/manually-creating-the-test)

-   [Testing registration](https://docs.bugbug.io/editing-tests/variables#test-user-registration-and-login-using-variables)

-   [Custom javascript](https://docs.bugbug.io/editing-tests/custom-javascript-actions) might be needed

Step 4: integrate automation with CI/CD
---------------------------------------

### **Goal:**

-   Improve software development lifecycle, being sure that no code is merged without tests

### **Approach**

-   Integrate automation with your development pipelines

-   Trigger automated tests with every pull request

-   Show test results directly in your repo tool (Bitbucket, GitHub, GitLab, etc.)

-   Ask developers to run automated tests locally on the localhost environment before they commit code

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [CLI](https://docs.bugbug.io/integrations/api)

-   [Overriding variables](https://docs.bugbug.io/integrations/api)

-   [Running on localhost](https://docs.bugbug.io/running-tests/test-in-your-browser)

Step 5: increase coverage
-------------------------

### **Goal:**

-   Get 100% test coverage, be confident that all the features, all the browsers, all the devices work correctly

### **Approach**

-   Pick your battles, choose what is the most essential to automate

-   We recommend you keep adding 1-2 test cases every week and don't try to achieve the impossible 100% coverage

### **Time & Resources**

-   To automate everything you would need an Infinite amount of time :)

-   To automate 2-3 new test cases per week you still need 2-3 hours weekly plus a bit more experience with tests and javascript.

-   Note that the more test cases you have, the longer the maintenance. The trick is to find the right balance.

### **BugBug features that you should use**

-   Project settings > Browser width (for [testing mobile screens](https://docs.bugbug.io/workflow-tips/mobile-version-testing))

-   Lots of custom Javascript

-   [Javascript variables](https://docs.bugbug.io/editing-tests/variables#javascript-variables)

-   [Javascript assertions](https://docs.bugbug.io/editing-tests/custom-javascript-actions#advanced-javascript-assertions)

-   [Parallel running](https://docs.bugbug.io/running-tests/parallel-mode)

Common problems and solutions in implementing automation testing:
-----------------------------------------------------------------

Here are some typical problems you may encounter when you want to start automating tests for a SaaS product. We've made a short list with possible solutions that worked for other [BugBug clients](https://bugbug.io/case-study/).

### **Captcha**

You need to [disable the captcha only for automation testing](https://docs.bugbug.io/troubleshooting/captcha-in-automation-testing). Unfortunately, it's not trivial and you need to read through captcha docs or ask a developer for help.

### **Date-based features, calendars** 

Is your app searching train timetables for tomorrow? It's difficult to automate because the date and the expected outcome can change every day.

Solution A: In the automated test, pick a date that's far in the future. Then update the tests regularly, changing this date and the expected outcome.

Solution B: create a dedicated environment for automation with a mocked-up hardcoded date on the backend

### **Dynamic lists such as search results, product listings**

It's hard to automate a test if some data is changing all the time. You need to create independent tests that are always sure what to expect, for example, if you're checking a list of promoted products on your homepage, your test should first go to the admin panel and mark such products as promoted.

You can also have a separate dedicated test automation environment where you have a hardcoded set of data.

### **Can't use the same user email twice**

Create a new user with each test - use the [built-it variable "testRunId"](https://docs.bugbug.io/editing-tests/variables#use-built-in-variables-for-dynamic-or-random-values) in combination with a [temporary email catcher](https://www.google.com/search?q=temporary+email&ei=ARVtY_X8IomKrwS_i4k4&ved=0ahUKEwi1gZ2u86P7AhUJxYsKHb9FAgcQ4dUDCA8&uact=5&oq=temporary+email&gs_lcp=Cgxnd3Mtd2l6LXNlcnAQAzIECAAQQzIFCAAQkQIyBQgAEJECMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDoKCAAQRxDWBBCwA0oECE0YAUoECEEYAEoECEYYAFDhAVjhAWC1CmgCcAF4AIABa4gBa5IBAzAuMZgBAKABAcgBCMABAQ&sclient=gws-wiz-serp) always to run tests with a fresh new user. You can also delete these users at the end of the test by recording test steps in an admin panel (if you have one) Read more in how to test user registration.

### **Selectors are not reliable**

The only way to make sure your tests are stable and easy to update is to start using "data-testid" attributes in your HTML structure, [dedicated attributes for automation testing selectors](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors). You need to communicate that to the front-end developer and make sure you remember about it during the code review. No worries, adding additional attributes have no risk, it's just additional data for automation with no other function, it will not introduce new bugs.

### **Multilanguage**

Your page can run in different languages and you want to verify all of them. Try to assess the risk of having a bug only in one language without affecting the others. Perhaps it's not likely that such a bug would occur?

If you still need to test different languages, you have two options:

Option A: Record separate tests for each language. Hard to maintain on a bigger scale

Option B: Use variables & profiles to run the same test on different languages

-   Add data-testid to all of your app elements, so that the selectors are independent of the app language and the displayed test 

-   Add a way in your app to change the language via a URL, for example, "myapp.com/?lang=fr"

-   Create a variable "language code"

-   Create different profiles with different language codes

-   Replace the language code in the "Go to URL" step with the variable

-   Create separate suites for all the languages, overriding their profile with a chosen language

**Tests take long time to finish**

[Upgrade your BugBug plan](https://bugbug.io/pricing/) to be capable of [running cloud tests in parallel](https://docs.bugbug.io/running-tests/parallel-mode)

### **Downloading files**

Currently BugBug can't verify if the files have been download. Soon we'll add a new type of assertion for that, [talk to our support](https://bugbug.io/contact/) if you need it ASAP.

### **Drag & drop interactions**

BugBug support for drag&drop is in beta, you can achieve it by combining the "Mouse press", "Hover" and "Release mouse button" [actions](https://docs.bugbug.io/editing-tests/actions). Talk to our support for details.

### **A recorded selector doesn't work**

Sometimes it's not possible to record a reliable selector and you need to use your human intelligence and debug HTML a bit. 

Use our [No-code XPath Selector Builder](https://bugbug.io/xpath-selector-builder/) which makes it easy to craft a good selector manually and can teach you in-depth about [best practices in selectors](https://bugbug.io/xpath-selector-builder/#How-to-create-reliable-XPath-selectors-for-test-automation).

Success in test automation depends on the regularity of your progress
---------------------------------------------------------------------

Implementing automation testing in a SaaS company is a long-term process. You should do it step by step. The biggest challenge for tech leaders is lack of time to dedicate on QA area. The truth is you don't need a lot to push it forward. Better slow than never. [😉](https://emojipedia.org/winking-face/) Block yourself a slot in calendar every week to work on QA automation and keep the momentum going.

medium_bugbug-matrix-illustration.png

small_bugbug-matrix-illustration.png

thumbnail_bugbug-matrix-illustration.png

bugbug-matrix-illustration.png

Tech leader's guide to automation testing

tech-leaders-guide-to-automation-testing

Software Testing Knowledge

Test Automation Expert and Test Architect. He is a seasoned tester who can’t stay long in one place. He is looking for perfection and a place to challenge his skills.  He gathered experience working for different companies with different working models, From small to big corporations, From Products via In-house development to software house. Thanks to that he has a wide perspective on testing quality and delivering value.\

[linkedin.com/in/wyrodek/](https://linkedin.com/in/wyrodek/){ref="nofollow"}

large_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

medium_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

small_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

thumbnail_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

Maciej Wyrodek

Test automation, especially end-to-end, is currently enjoying a revival. A lot of new tools and approaches have emerged over the last few years. Selenium has finally launched version 4, Cypress and Playwright have brought coded automation to a new level and tools like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim.io](https://www.testim.io/), have revitalized the idea of record and play automation. Since a lot is happening, let's take a look at automation and compare the strengths and weaknesses of 3 approaches to Automation: Custom Made Frameworks, Out of the box framework and Record and play. We will look at the pros and cons of all the solutions and when they work best. However, first we need to discuss the reasons for using test automation.

Why we automate tests
-----------------------------------

If you ask typical testers why they automate tests, they can provide many reasons: from shortening the Regression time, improvement of test coverage, to providing faster feedback. Ultimately, if we look hard enough, all goes back to the one and only goal of companies, as stated by [Eliyahu M. Goldratt in The Goal](https://en.wikipedia.org/wiki/The_Goal_(novel)) - Making money. Testing by itself often is considered a cost - as it does not bring more money. But let's not forget that more money can be made not only by selling more - but by increasing the margin - either by making the product worth the price or by reducing the cost. Testing can help to protect the bottom line, by reducing the cost of mistakes and finding them more easily and faster. This also leads to reducing the cost of customer support, refunds, etc. But as [DevOps](https://itrevolution.com/the-devops-handbook/) and [Modern Testing](https://www.moderntesting.org/) state - testing can synergize with a great experimentation culture, leading to possible delivery of a quality product in a shorter time and cost-savings on the development cycles. But this cannot be achieved without robust test automation. And automation needs to follow a few rules. For that, we need to consider the properties of great automation.

Traits of good automation
-------------------------

For automation to be useful in the DevOps world, it needs to be: 

-   Reliable 

-   Fast

-   Easy to write and maintain new test 

-   Easy to run

### Reliability

If test automation is not reliable, we won`t believe its results. As a result, we will still depend heavily on manual rechecking of everything and that is time-consuming. Or even worse, it can lead to wasting time as now no one will even bother to check the results.

### Fast

The development cycle is fast. If developers need to wait for the results of their work more than a coffee break, they will switch to the next task. Now the results will need to wait which leads to the cost of context switching. And longer [lead time of a task](https://kanbantool.com/kanban-library/analytics-and-metrics/kanban-definition-of-lead-time-and-cycle-time).

### Ease of Writing and Maintaining

If automation should be a regular part of a software development lifecycle, their benefit must justify the cost. If writing a test takes 4 days, it'd better be worth it - especially if developing  the tested feature takes less than that. The same holds true for maintenance: we need to be able to easily understand why a test has failed and whether this is a code issue or a test issue and how to fix it.

### Ease of running

This relates to being fast: anybody who needs to use a test to check a code should be able to do it without extra hoops, for example automatically running them locally or via CI/CD.

How to choose the best automation approach
------------------------------------------

There are a lot of ways of choosing tools for a project. When looking for tools that resolve a problem, we need to choose a tool that contributes the coveted value. The list of questions requiring answers is long, but today we will look at only one aspect, namely automation development and maintenance cost. We will look at the 3 most popular approaches, starting from ones requiring most coding and maintenance effort, like "Custom Frameworks", through usage of "Out of the box" Frameworks and finishing with Record and Play tools. We will look at the pros and cons. As it is my belief, all these approaches have made a name for themselves on the market and suit different project needs.

**Pros**

-   Great Customizability

-   Ownership

-   "On-site support"

**Cons**

-   Huge Maintenance Cost

-   Time Consuming

-   Requires lots of skill to Maintain

When I was starting test automation, this was the way to go. Mainly because Selenium Web Driver was the tool of choice back then. And as its description stated, [Selenium automates browsers](https://web.archive.org/web/20120208115222/http://seleniumhq.org/). Selenium was just a tool that did exactly what it said it delivered.

It didn't provide any functions for running test, or making assertions. 

At that time, practically every company had to build its own framework.

Often it was the role of test automation testers. In most cases, they were just learning how to automate. Making lots of mistakes in the process. But some companies managed to make great frameworks and, as a result, decided to share them with others. Some of them are still in use and developed but other have not stood the test of time. [Ocaramba](https://github.com/ObjectivityLtd/Ocaramba),[Selino](https://github.com/TestStack/TestStack.Seleno), [Serenity](https://serenity-bdd.github.io/theserenitybook/latest/index.html),[Selendie](https://selenide.org/), [Protractor](https://www.protractortest.org/#/)

This approach is still used today. Why?

### Pros

The main advantage of this approach is control. It gives a company the greatest level of control in customization. If a specific feature for a project is required, it is relatively easy to find someone who can develop this functionality.

Additionally, it gives a company considerable independence; there is no risk that at one day, the tool support will end or [repo will be deleted](https://nakedsecurity.sophos.com/2022/01/11/javascript-developer-destroys-own-projects-in-supply-chain-lesson/), leading to massive issues.

What is more, there is someone who knows how the tool works on site and can teach new team members.

### Cons

The main drawback of this solution is the cost - maintaining internal test automation framework costs time and money. Many companies underestimate the cost of developing test automation.

Time is of essence here: this automation needs a long time to be developed,  from creating a framework to writing a test. And this is the aspect that can be most damning. As discussed before, for automation to bring value, it needs to provide fast feedback.

Creating a custom framework is as big a project as any and it requires a good dose of skills, ideas, and planning. This means a lot of expertise while typically, companies are not willing to spend precious resources on test automation.

Out of the box frameworks: Pros & Cons
--------------------------------------

**Pros**

-   Shorter time needed to write first test

-   Much less need for framework maintenance

-   Lots of available online trainings and knowledge

**Cons**

-   Often only limited support from community

-   Writing and Test Maintenance takes time

-   Huge dependence on framework developers
As mentioned before, the growth of Selenium was accompanied by a lot of frameworks created on its base. Over time, the issues with the web driver became increasingly prominent, leading to the re-emergence of new automation frameworks not based on Selenium: [Cypress](https://www.cypress.io/), [Puppeter](https://github.com/puppeteer/puppeteer), [Playwright](https://playwright.dev/), to name a few most recent ones.

All these tools had one thing in common - they provided the user with out-of-the-box, ready options to write automation. No matter if this was Ocarmaba based on Selenium or Cypress: the goal of the solution was the minimalism needed for framework maintenance.

These tools usually provide built-in test runners, an extensive assertions library, boiler plates for starting writing project and tests, and - most importantly- a built-in mechanism to deal with test flakiness and general instability.

Additionally, these frameworks usually provide their own philosophy of writing test automation. As long as what we need to do is with accordance to their approach, everything goes well. Problems start when we need to go beyond what is suggested.

Until recently, the best example has been a test covering multiple domains in Cypress, but as of [version 9.6](https://docs.cypress.io/guides/references/changelog#9-6-0), it is no longer such a grievous case.

In another scenario, we need to rely heavily on framework developers, if they introduce breakthrough changes. Usually, we are left with two options: update and fix the code. Alternatively, don't update and hope that there won't be any issues in the framework that need fixing.

### Pros

Compared to the custom frameworks, we can start writing tests almost immediately without the need of spending weeks or months on writing the toolset first. Similarly, we don't need to wait whenever the operating system or browsers have an update. The tool maintainer will update the framework for us. And since most tools in this category are open-source ones, there is lots of information easily available online.

### Cons

As mentioned before, these tools have their own test philosophy, which is much more restrictive. Quite often support is limited - these tools are usually open-source and support in case of issues can be unavailable where we need to resort to forums and chats where we hope someone will answer a question; sometimes we get the answer in minutes but it may never come just as well.

However, the greatest risk of all is that we are at the mercy of framework developers - these tools become a part of our code base and if we want to keep support, sooner all later we will need to update even if we don't want to. A case in point is Cypress 10 - [it introduced breakthrough changes](https://docs.cypress.io/guides/references/changelog#10-0-0). As a result, many plugins and customer functionalities won't work without extensive updating.

Record and Play Testing Tools: Pros & Cons
------------------------------------------

**Pros**

-   Fast Test Writing Time

-   Low Entry Level

-   Professional Customer Support

-   Easy Scalability

**Cons**

-   Usually cloud-based, requires access to test environments

-   Still young, "untested" technology

-   Requires a new mindset

-   Data management can be quite complicated

-   Usually a commercial paid tool


The idea of record and play automation is not new, even Selenium has Selenium IDE which, after a few years on developments lately, has won some users' recognition.

Microsoft had Coded UI Test as part of the Visual Studio from 2010 and only in [2019 announced it deprecation](https://devblogs.microsoft.com/devops/changes-to-coded-ui-test-in-visual-studio-2019/)

Back then this technology was long before its time, the frameworks were very unstable and the maintenance of this test was problematic.

But in the last 4-5 years, a lot has changed. Solutions like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim](https://www.testim.io/) have taken over what modern technology has to offer and reinvented the concept.

### Pros

Most coded automation requires great investment of time into writing and maintaining test - usually even a trivial change costs around half an hour. In case of Automation Records and Play, test are usually meant to be disposed of  after they  stop working, as it is much easier to re-record a test instead of fixing it. This is of special importance for startups and projects with a lot of changes in the UI layer. When a team is spread thin, it is hard to find the time to write a test. But usually, you can record a few tests while checking if what has been done even works.

On the other hand,e out-of-the-box coded frameworks offer usually an easier learning curve. Record And Play tools usually do not require coding knowledge - although in more complicated cases some JavaScript knowledge can prove very helpful.

Since most of the record and play tools are paid solutions, it also means that companies have professional Trainers and Customer Support to help when issues arise. Usually, it means that in the case of young projects, we can even influence their feature roadmap.

These modern record and play solutions come with their own cloud runners, making scalability of test automation unbelievingly easy. Typically, paralyzing a test takes a few clicks.

### Cons

Unfortunately, Cloud Running is also a huge Con. First and foremost, we need to give this tool access to our test environments. 

An additional issue related to the Cloud is Term of Service - not all projects and customers can afford cloud services.

Most of these Record & Play tools are relatively young and not battle-tested. For long-term projects, this uncertainty can be very risky.

In my opinion, the biggest issue is Data Management. So far, no Record and Play solution has provided an easy way to manage test data. And this can be a huge blocker. Consequently, we need to change our approach to E2E test automation, with a desire to cover all the important and easy paths as fast as possible, to make this automation possibly most valuable.

Lastly, there is the price - cloud solutions are not cheap and if we need to run a lot of tests, it may be quite expensive. But some tools like [BugBug](https://bugbug.io) are on the other hand very cheap and offer unlimited cloud runs for a fixed mohtly fee. Arguably, it is still less expensive than maintaining coded automation and its infrastructure.

No clear-cut distinction
------------------------

Before the final thought, some clarification is required:

As much as we humans like clear categorizing, the real-word borders are much blurred.

For example: should Playwright be in the Custom frameworks or in the Out-of-the-box category? After all, its strength is the ability to use its own test runners, reporting services and easy building of the new features we need.

Similarly, it has the Record and Play functionally and so did Cypress (at the moment of writing this post, the latest version of Cypress is 10.3 and since 10 Cypress Studio, which allowed test recording, has been deactivated).

There are also tools that I like to refer to as combine-harvesters: basically, they do everything and claim that they can test anything, from mobile via web to desktop and many others -- the examples include TOSCA and Test Complete.

That's why we need to look at the numerous different dimensions of automation - and how much coding is just one of them.

Summary
-------

It is my belief that over time, both no-code development and record and play automation become the dominant players on the market. Yet there is a lot of room for improvement. All the types of approaches to automation, starting from custom frameworks, or using out-of-the-box coded solutions ending on Record and Play tools, have their uses on the market.

They answer to different needs and solve different problems. So, before selecting any tool, it is always good to ask yourselves: "What are we trying to gain?"

large_image (3).jpg

medium_image (3).jpg

small_image (3).jpg

thumbnail_image (3).jpg

image (3).jpg

Record and Play vs Coded Automation testing - how to choose what works for your team

record-and-play-vs-coded-automation-testing

We’ve prepared a short guide with essential recommendations on test automation that we created specifically for startups. We divided the guide into 3 levels:

[★☆☆ Level 1: get the most out of basic automation](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/)

[★★☆ Level 2: automate hundreds of regression tests](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)

[★★★ Level 3: integrate with CI/CD builds](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-3/)
  
Also see our guide dedicated to non-technical people: [beginner's tutorial to automation testing with BugBug](https://docs.bugbug.io/in-depth-guides/beginners-tutorial-to-automation-testing)
 
### What will you achieve on Level 1?
After finishing this level you will run 5-10 automated tests every hour and check if your core features are working properly. You will detect production bugs with big impact on users sooner than before.

### TLDR:
-   Start with short and simple tests that check your core features
-   Remember about recording “hover”
-   Create special test user accounts dedicated to automation
-   Schedule the tests to run every hour
-   Incrementally add more tests

### Focus on the most important features

![80-20 rule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/80_20_rule_1_c2126d9815.png)

Many companies spend endless hours on a neverending pursuit to automate everything. This leads to very costly automation that requires a whole team to create and maintain. Don’t be like these companies and keep it simple. Agile startups should start with automated testing of the most essential functionality. 

> 👍 **Rule of thumb:**
> Usually, it just takes 5 to 10 automated tests to be sure that most users are able to achieve their goals.

What should you test? Think of your features. What percentage of users are using a particular feature? If the answer is higher than 80%, then this is a core feature. The goal of automating tests of such important features is to make sure that you can detect bugs that have a huge impact on your business as soon as possible (new user registration is for sure a core feature, but it’s a little bit more tricky to automate, which we will cover in the next episode)






### Start with short & atomic tests

When you [record your first tests](https://docs.bugbug.io/creating-tests/recording-the-tests) start with something very simple and atomic. Atomic means that the test is small, independent and checks just one thing.

A good idea is to start with testing the basic navigation of your landing page. Click “record” and create a test that just clicks one item of your navigation link in the menu. Add [assertion](https://docs.bugbug.io/recording-tests-steps/recording-assertions) for some text in a heading. Finish the test and run it. Awesome! You’ve created your first automated test that touches the most essential area of your business!

![Screenshot 2022-07-25 at 13.44.12 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_12_1_0ebde0ddbb.png)

Now create a separate test for each navigation link. Here are screenshots of how BugBug team set up their landing page tests.

![Screenshot 2022-07-25 at 13.44.15 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_15_1_74fb39e9de.png)


### Don’t forget about recording hovers

If your website navigation appears on hover, you need to tell BugBug that you want to record a mouse moving to a specific element. To do this you need to click “Hover” during the recording and then click an element. When running the test, BugBug will move the mouse cursor to this element.

![bugbug record hovers.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_record_hovers_81639fbd30.png)


### Create special test users

Another idea is to test if login works. Create a separate test that only checks that one thing.

To achieve that you need to prepare a “test user” - a user account that is dedicated only to testing if login works properly

-   Manually register a new user with a new unique email, for example with john.smith+testautomation@email.com (you can use [disposable mail](https://www.google.com/search?q=disposable+mail) or [plus aliasing)](https://support.google.com/a/users/answer/9308648?hl=en)
-   Go to BugBug and record a test that clicks the login button, enters the user login and password and submits the form
-   Add an assertion for some text that confirms that the login was successful
-   Finish your test
    

  

You might be tempted to test other features in the same test, but don’t do this now. We want to be sure that the login test is atomic and independent from other features.

  

Remember that you can create as many test users as needed. For example, if your product offers multiple combinations of preferences, you can create a separate user for each combination.

  

> 👍 **Rule of thumb:**
> Always create new test users if this leads to less test steps

  

### Schedule all tests to run every hour

With BugBug you just need a couple of clicks to start running the tests every hour. Just go to “Schedules”, select the “Hourly” one and add “All tests”.

![hourly schedule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/hourly_schedule_1_445d7fe266.png)


  

That’s it, now you will start getting emails every hour with the results of your test automation. Tip: create a filter on your email account so that you avoid inbox clutter or [configure notifications to only get an email on failures.](https://docs.bugbug.io/running-tests/schedules#configure-email-notifications)

![email notifications (1) (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/email_notifications_1_1_a23f96ba47.png)


  

### Incrementally add more tests

You’ve successfully created test automation that monitors your production! Now keep adding more short tests one by one. Don’t forget to focus on being cost-effective - if you have trouble automating a use case, skip it and go to the next one or find a different way of checking if it works. And remember: you can always [ask BugBug support to help you with any troubles](https://bugbug.io/contact).

Read next: [next level guide on how to easily maintain hundreds of automated tests ](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)
  
  

medium_illustration - guide - level1.png

small_illustration - guide - level1.png

thumbnail_illustration - guide - level1.png

illustration - guide - level1.png

Automation testing guide for startups - Level 1: get the most out of basic automation

automation-testing-guide-for-startups-level-1