Software Testing Knowledge

Test Automation Expert and Test Architect. He is a seasoned tester who can’t stay long in one place. He is looking for perfection and a place to challenge his skills.  He gathered experience working for different companies with different working models, From small to big corporations, From Products via In-house development to software house. Thanks to that he has a wide perspective on testing quality and delivering value.\

[linkedin.com/in/wyrodek/](https://linkedin.com/in/wyrodek/){ref="nofollow"}

large_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

medium_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

small_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

thumbnail_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

Maciej Wyrodek

As a movement, DevOps is over 10 years old, and it is one of the most dominating approaches to modern software development. In some companies, however, testing in DevOps has become a critical issue.

[TABLE_OF_CONTENTS]

Lots of old methods and techniques are no longer good enough. DevOps requires changes in testers’ mindsets. Not everybody could adopt to this new era. Today we will try to help. We will look at some practices and techniques that actually work! We will also talk about quality mindset and look at automation. But first, we need to answer a few questions about DevOps and Testing, starting with:

## What do we mean by DevOps?

As with everything, DevOps has become a generic term. Everybody understands it differently:
- for some, it is about tools like CI/CD,
- for others it is a role in a company - usually a cloud and a CI/CD expert.

We will use a definition by [Gene Kim](https://itrevolution.com/faculty/gene-kim/){rel="nofollow"}, author of books like [Phoneix project](https://itrevolution.com/the-phoenix-project/){rel="nofollow"} and a [DevOps handbook](https://itrevolution.com/the-devops-handbook/){rel="nofollow"}, one of the greatest evangelists of DevOps.
In short: DevOps is a culture which helps to create learning organizations which can outperform competition, thanks to fast learning and experimentation enabled by fast delivery of the product.

In [Beyond the Phoenix project](https://itrevolution.com/book/beyond-phoenix-project-audiobook/){rel="nofollow"}, Gene Kim summed it up as 3 ways of DevOps:
- All about Flow,
- All about Feedback
- All about Learning

In short: You need to deliver a product fast to see how it works and learn what to improve, then repeat the cycle.

## What About testing?
Testing is good about collecting feedback and giving information, so it can be helpful with the feedback part.
But what about speed? Management Often sees testing as cost and a bottleneck.
Some even say you can’t have a good quality product if you are going fast. As we learn later, this is not true, but first, let’s discuss another important topic:

## What is quality?
I want you to answer one question. Which product is of better quality:
 - both are doing similar things,
- both have been on the market for some time
- one is a popular product with millions of users and known bugs in hundreds.
- the other one has less than a thousand users but virtually no bugs.

*So which is of better quality?*
At first glance, the other one feels as a better answer since it has fewer bugs.
But commercially it is a failure, it barely has any users! Users chose the other one  - maybe it was first on the market, maybe has had better marketing, maybe a better price.
The fact remains that, despite its all flows, the product is good enough for the users. We call this shippable quality, a product that good enough for users doesn’t need to be perfect as long as it satisfies the users’ needs.
The first thing you need to understand about Testing in DevOps is that **shippable quality today is better than perfect quality tomorrow**!

## Quality at Speed in DevOps
There is also one interesting paradox: the faster and more often you deploy, the better your product quality.
Why? In short:
1. First, you deploy frequently,
2. Each deployment contains less code, instead of months ita takes weeks, instead of weeks it takes days or even a single story.
3. Less code means fewer merging issues, fewer dependencies to check.
4. It also means smaller changes that are easier to test.
5. Resulting in fewer bugs.

It means you can also deploy quick fix if we find a serious issue.

## The Goal of Testing in DevOps
Summing all the concepts, we add all this together. The goal of testing in DevOps is clear: Empower teams to achieve testing as fast as possible.

## Ways of Testing in Devops

### Shift Left
To put it simply, Shift left is about testing as early as possible - and it doesn’t mean testing the finished product as early as possible. It means that, if you can find a partial concept or even an idea that you can verify before everything is done, you should do it at first chance.

A few Examples:
Start with the documentation. If the team has refinements, or there is a meeting where new features/stories are discussed, this is the place to put on the testing hat and do some lateral thinking. Pinpointing the issues here will save time later.

This is the time to ask a few important questions:
- How are we going to test it?
- What can we do to test it as fast as possible?

If the team is too big and the entire team meeting is out of the question, it is a good idea to have a 3 amigos meeting with the developer, the tester and the product owner/analyticto discuss how to develop a story and how to test it. The goal of this discussion should be to decide what needs to be automated and what to check manually.

Another example of Shift left is Dev Testing - instead of having a dedicated testing expert, developers with enough training can very well test their own code. It requires practice and discipline but canal in all, it saves a lot of time on the hand over imagine standard approach:
1.  developers write a code;
2.  the code waits for testing
3.  the testers test and find bug
4.  the bug waits for the developer to have the time to fix it
5.  the developer fixes it
6.  the bug waits for retest.

You get the idea. Now let’s look at a much shorter loop:
1. the developers write a code
2. they test it and fix issues immediately
3. Deployment

[As The Goal](https://www.amazon.com/Goal-Process-Ongoing-Improvement/dp/0884271951){rel="nofollow"} has showed us, a physical product wastes a lot of time just waiting to be worked on by different stations. Our work is not so different.
Of course developers need to be first taught how to test - and there are ways to do it: Dev-Test Pairing, or [Session Based Testing](https://www.satisfice.com/download/session-based-test-management){rel="nofollow"} with debriefings.
There is a lot more to shift left, but in general you can describe it in one phase: test as soon as possible.

### No code automation
Automation is a vast topic which we won’t cover completely here. Test automation should be a part of shift left, meaning: Automate as early as possible and only as much as you need. There are two tools that help in deciding what and how should be automated – a [Test Pyramid](/blog/software-testing/testing-pyramid-a-successful-approach-to-test-automation/) and Testing Quadrants.

However, the basic rule of thumb isthat if something can be done as unit test simply it should be done.

Unfortunately, even using this rule, we still need some end-to-end testing - this type is difficult to write and hard to maintain. Flaky(unstable) test are the bone of the industry.  Even the best companies, like Google, have problems with achieving 0% of flaky tests when creating end-to-end tests using tools like Selenium Web Driver or Cypress.

Usually, writing and stabilizing an end-to-end test takes a long time. Now imagine that you want to deploy often, but practically each new story requires fixing/changing the test automation, which takes a long time and slows you down.

Fortunately, the popularity of no-code leads to the rise of a new generation of Test Automation Tools like [BugBug.io](https://bugbug.io) and Mabl. Both allow to create a stable test in minutes!
Previously, the biggest issue related to this tool was that if a test was failed, your only option was to rerecord it. While fixing these test is much simpler, remember that Testing in DevOps is about speed!

Even if you would have to rerecord the test, it still will be faster than fixing a coded end to end test Written in Selenium Web Driver or Cypress.

### Monitoring and Testing in production
Unit and integration testing is all very nice but your system testing is also important. You need to make sure that everything works nicely together. There is one problem: the test environments. 

Usually they are not really on a par with production - they may have too low resources, run on different OS and many, many other issues. Even in the staging environment, often it is not a good representation of the production environment. Yes, in theory they should but in reality, if you want to test in an environment resembling production, there is no better place than production itself!

Ok, but how do you do that?
There are lots of approaches.
For example, you can do [cannery releases](https://martinfowler.com/bliki/CanaryRelease.htm){rel="nofollow"} of your application to see how it works in the wild.

If you have [a/b testing](https://hbr.org/2017/06/a-refresher-on-ab-testing){rel="nofollow"} - you can use these tools to hide all the new good things behind the a/b test and, after team ensures it is working, you can slowly start to show it to more and more users.
Another approach is [blue/green deployment](https://martinfowler.com/bliki/BlueGreenDeployment.html){rel="nofollow"}.

However, first YOU NEED TO HAVE GOOD MONITORING - with good observability. especially of user activity.
Remember what DevOps is about: “all about flow, all about feedback, and all about learning” - monitoring helps with both feedback and learning!
With a good rollback strategy or feature flags, you can safely deploy a code to production. If something goes wrong, swiftly disable it.

If you want to learn more, Cindy Sridharan runs an amazing blog [post](https://copyconstruct.medium.com/testing-microservices-the-sane-way-9bb31d158c16){rel="nofollow"} [series](https://copyconstruct.medium.com/testing-in-production-the-safe-way-18ca102d0ef1){rel="nofollow"} on [this topic](https://copyconstruct.medium.com/testing-in-production-the-hard-parts-3f06cefaf592){rel="nofollow"}.

### Quality Culture
We have talked about DevOps and we have talked about testing in it; there is one more issue to address as even the best tools and technical solutions won’t save the day if there is no culture promoting quality.  In order to create this culture, we need good principles.
Fortunately, [Modern Testing](https://www.moderntesting.org/){rel="nofollow"} comes to the rescue, trying to address the necessary cultural changes for testing in DevOps.
The basic principles include:

1. Our priority is improving the business.
2. We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system.
3. We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures.
4. We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.
5. We believe the customer is the only one capable of judging and evaluating the quality of our product
6. We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact.
7. We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist.

## Summary

Today we started with the same definition of DevOps - culture concentrating on flow, feedback and learning.

We have explained the goal of testing in DevOps - empowering teams to achieve shippable quality as fast as possible.

Then we looked at a different way to help us achieve this goal from shifting left, and no-code test automation, to testing on production, ending on testing culture.

Notably, at first many of the discussed issues were not directly related to testing.

Most of the technical solutions require changes either in the process the deployment strategy or the code structure.

This is another important aspect of Testing in DevOps - testing cannot be left to the end. Just like quality, it needs to be built into both the product and the process. Since the beginning, it cannot be an afterthought.

If there is one thing that can bring your team closer to excellent DevOps testing, start with asking these questions: How can we test it? When is the first opportunity we can test it?

large_bug-g27e425b83_1280.jpg

medium_bug-g27e425b83_1280.jpg

small_bug-g27e425b83_1280.jpg

thumbnail_bug-g27e425b83_1280.jpg

bug-g27e425b83_1280.jpg

Testing in DevOps

testing-in-devops

thumbnail_https://media-exp1.licdn.com/dms/image/C4E03AQFhPAvVKTYptQ/profile-displayphoto-shrink_800_800/0/1575468541681?e=1646265600&v=beta&t=y7yyvxBb2bUGOo5TrJ4VmRjRrjcjfooYSAbit9n25F0

https://media-exp1.licdn.com/dms/image/C4E03AQFhPAvVKTYptQ/profile-displayphoto-shrink_800_800/0/1575468541681?e=1646265600&v=beta&t=y7yyvxBb2bUGOo5TrJ4VmRjRrjcjfooYSAbit9n25F0

Mark Macznik

If you want to make it big as a software developer, you need to deliver your clients a product that is stable and of high quality. Programming is a complicated process, and very often many programmers work on the same project. The more complex the software, the more bugs and flaws will creep their way into it – and it is absolutely crucial to beat them out before the final build reaches your client. 

Smoke testing is one of the methods of preliminary testing, often performed as functional tests or unit tests during the software production process. A smoke test is a critical test type, and passing it is one of the key requirements for moving on with the testing process. 

But what exactly is smoke testing and how is it performed? Is it as important as we say? We will try to answer these questions and more with this short yet informative guide on smoke testing with examples.

## How does Smoke Testing work?

Smoke testing, being one of the critical testing methods, aims to confirm whether the basic functionalities of your software are working as intended. Smoke testing is most often performed on initial builds when the software is at its most unstable. The results of smoke testing can give you a good idea of what’s working and what’s not in your app, allowing you a smooth transition into regression testing in the future.

Hiring a QA team to perform detailed tests on your software can be expensive. If you decide to keep your own QA team on a payroll every month, you can quickly find yourself bleeding money, and the team might not have that much work to do – especially if you’re a small developer. Outsourcing your QA might lower your costs, but still won’t be cheap. Performing a basic smoke test will give you an idea of which areas you will need to focus on during future testing. Keep in mind that smoke tests aren’t meant to be thoroughgoing – make sure to choose test cases that will only target the most crucial functionalities of your software. We can check everything in more detail during the later testing stages.

## What’s the difference between Smoke Testing and Sanity Testing?

If you know your way around software testing, you might have already spotted the similarities that smoke testing has when compared to sanity testing. Both tests mean to evaluate basic functionality to determine whether or not the app should move ahead to further tests, or whether it still needs more work at its most fundamental level. So what’s exactly the distinction between the two? 

While some argue that the two terms can be used interchangeably, there is actually a bit of difference. Sanity testing refers to testing new functionality and fixing bugs caused by patches or updates. Smoke testing, in turn, focuses on the key functionalities of the software.

## The benefits of performing a Smoke Test

While you might think smoke testing is only a rudimentary way of software testing, the benefits it gives the development team are undeniable. Apart from allowing you to save money on further QA by identifying flaws in basic functionality, smoke testing can save you tons of effort and time. If you should skip smoke testing and proceed directly to regression testing, you can easily get stuck in a testing cycle loop, moving on to thorough tests and going back a few steps each time a major error is found.

Smoke testing is fast, efficient, and effective. It allows you to find out any flaws and defects in the earliest stages of software production, reducing the risk of releasing a faulty product. Once all of the major bugs and problems have been solved you can then move on to more detailed testing methods, but in the beginning, smoke testing gives you the ability to swiftly get rid of critical errors that would otherwise hinder your progress.

So what could exactly happen if you forget about ensuring a proper smoke testing stage? The most common results of skipping this preliminary test are major setbacks in the future, as major defects go on undetected and make a mess out of your functional testing stage. Having to go back to basic functions and gradually fix them one by one, instead of figuring out all of them right from the get-go, will ultimately result in a waste of time, money, and effort.

## Smoke testing examples and sample test cases

How exactly smoke testing will look depends heavily on the type of software you are developing. If you’re running a simple website that has register and login functions, an example of smoke testing would be verifying that functionality. The result you’re going to usually expect from such a test would be to simply access your website and log in without any issues. The steps would look similar to this:

1. Open up your browser’s executable
2. Navigate to your website
3. Open the login panel
4. Enter your credentials and log in/register
5. Finish the process

If the results are positive – as in you actually managed to log in or register, then the test was successful. You can create similar test cases for other basic functionality of your software, like verifying whether a cart system is working properly at your online store or checking if your app doesn’t crash after you open the executable.

## Summary

All in all, you can see why smoke testing is so important. Can you imagine forwarding your build to QA only to find out the app crashes on start? Avoiding smoke testing can lead to a tremendous waste of time and resources, delaying newer iterations of your software and putting you in danger of missing deadlines.

If you care about the quality of your software, smoke testing and sanity testing are the way to go. Always perform a set of smoke testing cases at the beginning of the development process and some additional sanity testing in future builds to ensure stability and minimize your team’s effort during the testing phases.

large_smoke-tests-blog.jpg

medium_smoke-tests-blog.jpg

small_smoke-tests-blog.jpg

thumbnail_smoke-tests-blog.jpg

smoke-tests-blog.jpg

A concise introduction to Smoke Testing

a-concise-introduction-to-smoke-testing

Testing is an ever-present part of a software developer’s job, performed at every stage of the production process. There is a wide range of test types used for software testing, some performed at various points during production, and some performed after production to confirm that the software functions as intended.

Regression testing is performed on a product that has already been finished and released. How exactly is it performed? What are the objectives and how does the procedure look? If you’re keen to find out, keep on reading as we explain everything you should know about regression testing.

[TABLE_OF_CONTENTS]

## The definition of Regression Testing

**Regression testing** is a software testing method that aims to investigate and locate any possible flaws or bugs in software that has already been fully developed. Regression Testing is performed to find out whether your software has regressed (surprise, surprise) as you continue introducing new updates and bug fixes. Whenever a piece of software is changed in any way, new bugs might emerge and some of the older ones re-emerge. 

Most often, regression testing is performed immediately after introducing a new update, but it isn’t the only factor that might call for the test. If any software or hardware-related factor has changed, regression testing should be committed to find out whether everything performs as it has before the change. This can include any sort of bug fixes and software changes, but regression might also be caused by hardware configuration changes and electronic component degradation over time. Regression testing isn’t easy – it requires cooperation and diligence out of all of the team members, which makes it perfect for Agile Software Development practices.

## Regression Testing techniques

Regression testing can include various functional and non-functional tests in its process to thoroughly investigate the software in search of flaws. There are also many techniques with which developers and QA specialists perform regression testing, including:

- Retest All
- Regression Test Selection
- Test Case Prioritization
- Hybrid approach

## Retest All – an accurate yet expensive solution

The **Retest All** technique is the most accurate and all-around method of regression testing, but it is also the most expensive one. In Retest All, every test case is checked again on the software to confirm if all of its parts are working as intended. That’s all there is to Retest All – it might be simple in definition, but it is definitely the hardest and most costly technique to actually perform.

## Regression Test Selection – when you realize you might not have enough for a Retest All

**Regression Test Selection** has been designed as a solution to the expensive nature of the Retest All technique. In **RTS** we don’t run all of the tests in our test suite, but instead we divide all of the tests into three categories – reusable test cases, retestable test cases, and obsolete test cases. In addition, RTS might actually create new test cases that haven’t been in the suite before, but might be relevant for regression tests.

If you’re interested in finding out more about regression testing examples, we recommend taking a look at *Understanding Regression Testing Techniques* by Gaurav Duggal and Bharti Suri. The authors further divide RTS techniques, naming two most important categories – coverage techniques and minimization techniques, and explain what are safe techniques.

**Coverage techniques** feature specific coverage criteria, often provided by the client who wants their software regression tested. These techniques sift through all test cases, selecting only the ones that are relevant for the covered parts of your software. **Minimization techniques** work in a way similar to coverage techniques, but instead of selecting cases based on set criteria, you select a minimum set of cases that you think will do the job.

## Test Case Prioritization – a rational approach

**Test Case Prioritization** aims to maximize the rate of fault detection, creating the most thorough regression test for the lowest cost. Every test case is assigned a priority, after which all of them are performed in order, starting from the most important one and finishing on the least important one. Thanks to that, even if your resources aren’t enough to perform all of the tests in your suite, you will still perform the ones that should detect the most flaws.

**TSP** might be performed in two ways – the first one, called **General Prioritization**, selects test cases that should be effective (on average) on all versions of the software you’re working with. The other way, called **Version Specific Prioritization**, only takes into account a specific version of the software, without considering other existing or potential future versions.

## Hybrid approach – the best of both worlds

When choosing one technique of regression testing is impossible – why not combine the best features of Regression Test Selection and Test Case Prioritization? The **hybrid approach** is still being developed, and every researcher gives a personal spin on the idea. Since there isn’t an exact definition of what a hybrid approach actually is, there are tons of versions available – each one with its own upsides and downsides. 

## Conclusion – is Regression Testing necessary?

As you have already learned, regression tests are performed to keep the quality of your software as high as possible for as long as possible. Updates and changes to your software might introduce new problems, which is exactly why regression testing is so important – to keep user experience high throughout all iterations of your program. 

Not all projects require regression testing, of course. In projects that are already finished and won’t be further updated, regression testing is mostly irrelevant. However, if you are planning to continue development further – never forget to properly regression test your software before releasing an update.

large_regression-tests-blog.jpg

medium_regression-tests-blog.jpg

small_regression-tests-blog.jpg

thumbnail_regression-tests-blog.jpg

regression-tests-blog.jpg

Everything you need to know about Regression Testing

everything-you-need-to-know-about-regression-testing

When working on a piece of software, testing is one of the most crucial elements of the production process. It allows the development team to find out what works and what doesn’t, and doing it in a proper, standardized manner will give your entire work crew a better sense of communication. With proper testing practices, you can ensure your product stays at its highest quality, minimizing the number of bugs and flaws the users may experience and speeding up the development process.

Functional testing is one of the general software testing categories, meant to verify the functionality of an app or a website. Testing isn’t something done only at the very end of development – in fact, in the early planning stages of the project, you should create a list of functional specifications. Simply put, make a list of what your software is supposed to accomplish.

Implementing a proper functional testing phase can be very beneficial to your development process. How exactly to implement it properly? Let’s see what functional testing is all about with some examples to make it easier to understand.

## Who can perform functional testing?

Since functional testing is a black-box software testing technique, the tester doesn’t need any knowledge about the inner workings of your software. In fact, the tester doesn’t even need to know anything about programming – functional testing is all about user experience, so the tester shouldn’t need any more knowledge than an average user is privy to. In fact, functional tests are most effective when performed by someone without deeper knowledge of the system they are testing.

With functional testing, you can easily check the main functionality of your software, confirming whether all required features work as intended. It also gives you the ability to see how accessible the app is to standard users – as a developer and (most probably) a programming specialist, your knowledge allows you to instinctively use an app or a system in a way that a standard user wouldn’t be able to. Functional testing lets you see exactly how usable the software is for such users, letting you see whether the end-user can use the software comfortably and what exactly happens when an error occurs.

## What are the types of Functional Testing?

Since functional testing is more of a category of software tests than an individual testing methodology, there are a variety of types of functional tests. The most common types of functional testing are:

- Sanity Testing
- Smoke Testing
- Integration Testing
- System Testing
- Component Testing
- Unit Testing
- User Acceptance Testing

**Sanity Testing** is performed whenever your software gets updated with new bug fixes and changes. It verifies all of the newly introduced functionalities, making sure they work as intended. Most often, sanity testing is considered as a type of regression testing, and many programmers get it confused with smoke testing – let’s see why.

**Smoke Testing** is very similar to sanity testing, but there is a definitive difference. Smoke testing is performed on initial builds of the software, verifying that basic functionality works. This type of testing isn’t exhaustive – and it isn’t supposed to be. It aims to go over only the most fundamental components of your software. 

**Integration Testing** gives you the ability to confirm whether individual functionalities and components of your software perform well together. It is performed after all of the individual modules have been confirmed to work without flaws and is one of the most popular types of functional testing. Integration testing can be easily automated, and often is, making it a simple step in the QA process.

Once you are sure that all functionalities operate fine individually and when integrated together, it’s time for **System Testing**. System testing is performed not to confirm whether something works, but whether your software complies with specific requirements. These are most often requirements given to you by your client, and system testing gives you the ability to easily measure how well you have achieved what your client wanted of you.

**Component Testing** is performed before integration testing to verify the individual performance of every component of your software. Keep in mind that these components aren’t simply consisting of a single functionality. For example, a component test for the login component of a website might include testing the visibility of the UI, page loading, SQL performance, and end-user experience.Finally, **Unit Testing** is a very thorough test that directly examines the code of your software’s components to make sure it is doing exactly what it is supposed to do. Unit testing can provide valuable documentation of your code development as you progress with the software’s production, which can be extremely useful to communicate between teams and speed up the work. Without unit testing, failed tests can be much harder to explain, and finding error sources will take much more time.

## Conclusion – how to understand Functional Testing?

Since there are so many types of functional testing tests, it’s impossible to provide an example that would fit all of them. Instead, we want to demonstrate to you first a simple functional testing framework. In its simplest form, functional testing can be said to have three steps.

1. Prepare input values and categorize them into test cases
2. Verify your test cases
3. Compare the results to the software requirements and expected results

However, functional tests can have varying degrees of complexity. Let’s consider the example of a login functionality test. Unit testing would require dozens of test cases and many scenarios, including every part of the login process. In turn, a simple smoke test would require much fewer cases, only verifying the functionalities critical for the end-user to log in.

As you can see, functional testing is a fascinating and very comprehensive subject, with many software testing types that will surely be useful to any software developer. With proper testing practices, your software can achieve better quality much faster, fostering communication and cooperation between different teams working on the same project.

large_functional-testing.jpg

medium_functional-testing.jpg

small_functional-testing.jpg

thumbnail_functional-testing.jpg

functional-testing.jpg

What is Functional Testing? Types and examples.

what-is-functional-testing-types-and-examples

Before you release your product into the market, it is extremely important to properly test whether it’s functioning as intended. To do that, software developers utilize a variety of testing techniques that verify the software’s integrity. The IT industry makes use of dozens of various test suites – there are over 40 different testing methods, counting only the most popular ones! To thoroughly check their software, developers implement **End-to-End testing**, often simply called **E2E** testing – an elaborate technique that mimics end-user experience. 

There are many layers to End to End testing, as it is one of the most complex testing methods, requiring you to check every part of your software that the end-user interacts with in any imaginable way. 

With this guide, we want to show you the importance of E2E tests, as well as tell you precisely what they are, including a few specific examples.

**Also see:** if you're new to software automation, take a look at this [beginner's guide to automation with step-by-step screenshots](https://docs.bugbug.io/in-depth-guides/beginners-tutorial-to-automation-testing).

[TABLE_OF_CONTENTS]

## What exactly is End-to-End testing?

**End-to-End testing** is one of the most useful and informative techniques for software testing, providing valuable data about the inner workings of the program from the end-user perspective. With E2E testing, developers can thoroughly verify the behavior of their software. During the test, you will be trying to replicate key user behavior scenarios when using your software. The actual list of things to do varies, depending on the type of software you’re testing – but in all cases, it is rather extensive.

When performing a software testing method, like an End-to-End test on a web application, you would start with opening your browser and navigating to the proper URL, and then proceeding to use the software as intended for the end-user.

Similarly, in the case of a regular desktop application or mobile application, you would open the app just like a user would, checking out every bit of functionality that is intended for the users to work with. Keep in mind that during E2E testing you won’t need to check developer-specific aspects of the software, only the end-user perspective if it is important to you.

During an E2E test, your goal is to verify all of the dependencies and data integrity that directly or indirectly influence the reception of the software by the end-user. This usually includes any external services, as well as databases used by the software and metrics, as well as any other system dependencies. E2E tests will ideally be performed in the production environment, but this isn’t always possible. The tests can, of course, be performed in an independent End-to-End testing environment, but you should make sure it is as close to the production environment as possible.

Within the SDLC process, E2E tests are performed in the Testing phase, which is situated after Coding and before Deployment. 

## Why is End-to-End testing so useful?

Is End-to-End testing necessary? It is true that more and more developers make sure to implement an E2E testing phase before the final deployment of their software. There are many benefits to E2E tests, but the most crucial ones are:

- The ability to verify software performance from the end-user perspective
- Testing not only the software itself but also any interrelated sub systems, increasing test coverage
- Validation of back-end systems
- Immediate end-user-like feedback on test failing

While tests such as unit testing or integration testing are very useful, allowing you to verify the working of each component and providing quick results, they don’t give you the whole picture. By not performing E2E tests, you won’t know how the experience of using your software will look like from the end-user perspective.

For example, individual component testing (integration tests) might show promising results and full functionality. However, when combined, something goes wrong and produces a bug. By only checking individual components, you wouldn’t be able to find out about the bug. With a proper End-to-End test, you can verify whether everything works correctly together as it should. Even integration testing doesn’t cover all of the system components of your software, and you might never know whether every aspect of your app clicks together.

E2E testing doesn’t necessarily have to discover bugs – it can also give you useful insight into the user scenario of your product. Overall, all types of tests play their respective roles in the whole testing process. Just because E2E tests give you a more complex understanding of your software as a whole doesn’t mean that other test types are less valuable. Before releasing your app, always make sure to proceed with a thought-through testing strategy. However, while they’re certainly extremely useful, keep in mind that E2E tests occupy the top of the testing pyramid, making them the most expensive and most difficult to perform.

End-to-End testing is also one of the few functional types of tests that can be performed manually, as well as automated - which is done especially often in Continuous Integration. In such a scenario, a chosen set of tests can be run automatically each time a new iteration is uploaded to the testing server.

## End-to-End Testing Framework

Alright, so now you know what End-to-End testing is – but how to exactly implement it? There is a wide variety of checks and individual tests that must be covered by your end-to-end testing plan, some of which include:

- Establishing test requirements in the preparation phase (including software and hardware requirements)
- Creating proper documentation
- Deciding what type of test environment will be used (production environment or independent test environment)
- Preparation of test cases and general E2E design
- Tracking of data input and output
- Test execution and evaluation

When executing E2E testing methods, developers usually follow an established E2E testing framework that guides you throughout the process, enumerating various tasks and responsibilities. The framework is relatively simple and consists of only three elements:

### Establish user functions

The User Functions relates to listing the features of your software, as well as all of its interconnected software systems. To properly measure and test software functionality, you need to first think about what you’re going to test. After you’re finished with listing user functions, remember to record everything relevant to each of them, including both the input and output data. Since E2E testing can be considered holistic, you also need to pay attention to all of the relations between every individual function – how they work together and whether there are any clashes.

### Come up with specific conditions

Now that you have established the user functions you’re going to test, you need to think about the testing conditions. Try to come up with as many quantifiable indicators that can be measured for each of the functions. These can range from response time to consistency of data flow, to simple convenience of use. Remember – you’re trying to imagine yourself as an end-user, so try to keep track of anything that might be an issue for a non-developer.

### Create individual test cases

Finally, before the E2E testing can commence, you need to prepare specific test cases for each of the conditions. At this point, you have prepared a list of user functions to test, as well as data conditions that will be measured during the test. Now, you need to create at least one test case for each of the conditions. By test cases, we mean user scenarios during which specific indicators will be able to be measured. 

And by test case preparation status, we mean the precise location of the test instances being developed, as compared to intended test cases.

## Examples of E2E tests

Congratulations, you’ve almost reached the end! At this point, you should pretty much be a walking fountain of knowledge about End-to-End tests. However, the raw theory is one thing, and implementing those types of tests for system testing in reality is another. To make the process a bit more digestible, consider this End-to-End testing example for a website with shopping functionality:

- Go to the specified URL
- Open a list of products
- Check whether the information on the list correlates with specific product pages
- Try to add the product to the cart and look for visible confirmation
- Navigate to the cart and check whether the product has been successfully added
- Double-check all of the relevant information (price, product info, etc.)
- Proceed with the check-out and verify payment options integration

See how simple it is in the end? Now you can implement E2E testing methodology in your software to ensure better end-user reception, increase your product quality, and build a positive brand image.

## Frequently Asked Questions

1. **What is automated testing?** \
Automated testing in software development is a form of software testing that uses the execution of predefined sets of operations to probe and validate the software system. The fundamental purpose of test automation in software development is to generate objective feedback about the quality of the product. One such tool is Selenium. It can run UI tests.
The ideal situation is to run E2E tests with both manual testing and automated testing approaches. If your application has a lot of complicated user interfaces, you'll need more end-to-end testing.

2. **What is software development lifecycle?** \
Software development lifecycle (SDLC) is the process used by developers to create, deploy and maintain software. This process includes several steps that are all closely connected with one another. These steps include requirements gathering, analysis and design, coding and testing.

3. **What is a testing pyramid?** \
In software development projects, we have limited time and resources. This is why it's important to consider our test efforts carefully and find the right balance between different types of testing.
Basically, there are 3 layers in the testing pyramid: Unit Tests, Integration Tests,  and E2E Tests.

3. **What is a selector?** \
A selector is a small piece of code that describes a specific button in your app. It allows you to clearly define what should be clicked in automated end-to-end tests. You can learn more about selectors using [this interactive XPath learning tool](https://bugbug.io/xpath-selector-builder/?ref=blogarticle)


large_testowanie-end-to-end.png

medium_testowanie-end-to-end.png

small_testowanie-end-to-end.png

thumbnail_testowanie-end-to-end.png

testowanie-end-to-end.png

What is End-to-End (E2E) testing?

what-is-end-to-end-testing

**Nowadays, the profession of a software tester has become a symbol of entering the IT world without elbow grease. Why can some people pull it off and some cannot? Is there a golden rule for securing a job as a tester? Let’s find out.**

A good software tester has not only undergone special training crowned by a respective diploma. If you want to do this job successfully, you will appreciate natural predispositions and skills that you can acquire single-handedly. You may wonder if pure skills will suffice. Well, it depends …

## Who can work as a software tester?

Speaking in most general terms: anyone. There are no age or “trade” limitations; for many people who want to branch out the profession of a tester marks the first step in a career in IT. Many candidates fall prey to a stereotype, thinking that this is a simple, easy and pleasant profession that anyone can practise. Here come the details that you need to be aware of if you are  contemplating the job of a tester. As it is the case with other professions, some predispositions are also welcome, not only personality-related. These predispositions include:

### Analytical thinking 
A good tester needs to be able to combine facts, view an application and its possible uses in the context of its product environment and to draw conclusions;

### Inquisitiveness


When, surfing the Internet, you ask questions like “why does it work this way?”, “I wonder why they used this solution”, you are on the right path. Even if people around you sometimes find your inquisitiveness irritating,  it will prove priceless in a tester’s work;

### Resilience to boredom

It comes in handy especially at the beginning of work as a manual tester, when you check how an application works and simply “click your way” through some software an unlimited number of times. A certain degree of repeatability is an inherent part of a tester’s job; 

### Openness to people 

Another stereotype worth debunking before stepping onto the path of software testing is an introvert hiding in his cubbyhole, a dead ringer of  a character from “IT Crowd”, avoiding people at all cost. The situation is quite the opposite! Yes, you spend some time on the testing itself but you need to communicate the results of your work to a developer, a project manager or an entire team. Sometimes you need to explain what is not working. Sometimes you need to ask why something is working the way it is ☺   This is not a job for people who would rather have their arm chopped off than talk to someone on the team.

### Creativity

It will come in handy on a higher level of advancement when you are responsible for creating testing scenarios. In some companies, testers cooperate closely with the entire project team and contribute to developing the assumptions behind a product which provides plenty of room for creativity.

The stereotype of a software tester has one “benefit”: people think that testers do not need to know programming. Yes and no – it all depends on your aspirations. 

## Do you want to be a manual tester?

At this stage,  you are typically in charge of “clicking your way” through an application and reporting possible errors. After several or over a dozen months of working as a tester you can also start to write scenarios and test cases. At this stage, in order to acquire new skills, you can also learn to make use of automation tools  or tools supporting manual testers. However, if you do not want to grow or learn the basis of a programming language – you hit the wall.   The same holds true for financial promotion – among testers, the manual ones are at the bottom of the pay brackets. 

## Do you want to be an automatic tester?

Knowing the basis of programming is imperative because in this job you write your own test-automatizing scripts and you fix possible operating errors. All the time, you supervise also the programmes’ operations and you enhance them, for example with new requirements that occurred in the process of creating a product. There is a reason why burnt-out developers or people who want to have a break in programming are good candidates for automatic testers. If you are literate in even one programming language (of course the more the better) you are good to go.

## How to start working as a programming tester

Ok, let’s assume that you remain firm in your decision to become a tester. You have all the predispositions and you want to grow in IT. Where do you start? Regular broadening of your horizons and self-learning are a must; it does not matter whether you  already know the job or are contemplating it. The Internet is a mine of knowledge; we recommend publications both in Polish and English  (there are many more materials in English). If you know some basis, you can apply right away for a manual testing internship programme. Junior positions are also within your reach – they are particularly attractive to people who branch out and cannot afford an unpaid or poorly paid internship. At the same time, you can try to take on single jobs on platforms like uTest – they will ensure an inflow of cash as well as experience, so valuable at the dawn of a professional career.

Read also: A manual tester: 10 reasons that make this job interesting

## Working as a software tester – with or without training?

As working in IT has become fashionable, training companies are flourishing. They offer not only trainings for developers but also testers. On the one hand, this is an interesting and developing option because a course poses an opportunity to acquire knowledge in a user-friendly way.  This is also a guarantee of regularity in learning which is of importance to less systematic people or ones having difficulties with self-motivation. On the other hand, a training is usually costly. Before you choose the school, take a good look of the market because some participants complain about the small number of practical classes and poor care on the part of the mentors. Whether you decide to embark on a  course or not, it is a good idea to find a mentor: a person who already works as a tester and who may lend a guiding hand in your search, recommend literature on the subject  or give you feedback on your work. Last but not least: even if you have the best teachers/mentors, without your determination and own work they will not be of great help. Acquisition of testing skills is, first and foremost,  a considerable effort and ensures regular enrichment of knowledge – a lot depends on you and your commitment.

## How to be an IT tester with a certificate

The schools mentioned here frequently offer their own diplomas but the market honours certificates issued by the International Software Testing Qualifications Board (ISTQB). While you can obtain certificates at any stage of professional development, a document on the basic level should be the first step to becoming a Certified Tester. If you are new to the industry, this item on your CV is bound to be welcome. On the other hand, some people are critical about the certificates and say that all you need to know as a tester can be learnt single-handedly.

## How to be a tester and earn real money 

This is a very important issue; after all, the time of poorly paid internship must come to an end. Big companies typically have a natural development and promotion path that you may follow after having worked for some time as an intern or an IT junior. However, continued development is key to promotion – if you are contemplating the position of an automatic tester, do not put off learning  programming. The more you know about testing and the industry the better your chances of becoming a QA (quality assurance) team member. It stands for a tester with considerable experience who is in charge of entire projects, also related to people management, writing testing scenarios, writing automatic tests and monitoring work within a project. Irrespective of the position you assume, when you work in IT you should know the methods of project management while words like “agile” and “waterfall” should definitely ring the bell. 

large_jak-zostac-testerem.png

medium_jak-zostac-testerem.png

small_jak-zostac-testerem.png

thumbnail_jak-zostac-testerem.png

jak-zostac-testerem.png

How you can become a software tester

how-you-can-become-a-software-tester

Blog about test automation

Features

Pricing

Roadmap

Case studies

Privacy policy

Terms of service

About us

Book a demo

Contact

Get more tips and product related content. Zero spam.

Don't miss any updates

End-to-end testing made simple! BugBug is a new, more intuitive tool for test automation of web apps. Get all the benefits of test automation without hiring new people or infrastructure changes.

BugBug.io | Intuitive and reliable test automation for startups

large_Open Graph 1200x630 3.png

medium_Open Graph 1200x630 3.png

small_Open Graph 1200x630 3.png

thumbnail_Open Graph 1200x630 3.png

Open Graph 1200x630 3.png

Updates

Product

Guides to automation

small_pawel-about-512.png

thumbnail_pawel-about-512.png

pawel-about-512.png

Paweł Bylina

Are you a tech leader, CEO or CTO in a tech startup? You want to implement automation testing for your SaaS product? We've written this guide especially for you. It's packed with actionable tips on how to approach test automation in an agile way.

We're going to tackle the following questions:

-   What is the business goal of automation testing?

-   How to implement automation incrementally?

-   What resources do you need to develop automated tests?

-   What are typical problems and obstacles and how to overcome them?


The strategic approach to automation testing
--------------------------------------------

Automation testing is a very broad term. There is no single goal of automation. Depending on the level of automation you will get different benefits for your organization.

*Figure 1: Strategic matrix of different goals of test automation*
![bugbug-goals-of-automation-testing.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_goals_of_automation_testing_9308b9cb1b.png)

See this strategic matrix above. You can achieve all of these goals, but not all at once. You should pick the right order of goals, starting with the ones that are low-effort and high-value. 

To achieve each of these goals, you need a different combination of resources and skills. We'll guide you through each of the goals, explaining the approach, time & resources and how to achieve it with BugBug.

Step 1: low cost & high impact goals
------------------------------------

### **Goal:**

-   Prevent loss of business due to production downtimes

### **Approach**

-   You don't need to hire new people, you can do it yourself

-   Create 5-10 automated tests and run them every hour to monitor your production, for things like login, landing pages, registration form, contact form, basic app navigation

### **Time & Resources**

-   2-4 hours of your own time

### **BugBug features that you should use**

-   [Recording tests](https://docs.bugbug.io/creating-tests/recording-the-tests)

-   [Cloud runs](https://docs.bugbug.io/running-tests/in-cloud-on-server)

-   [Schedules](https://docs.bugbug.io/running-tests/schedules)

-   [Notifications](https://docs.bugbug.io/collaboration/notifications)

-   [Re-recording test steps](https://docs.bugbug.io/recording-tests-steps/re-recording-steps)

Step 2: low hanging fruit
-------------------------

### **Goal:**

-   Improve team collaboration on quality assurance

### **Approach**

-   Give all the scrum teams access to the testing platform

-   Share direct links to failed tests in your tickets or on a chat

-   Configure who gets notifications

### **Time & Resources**

-   Inviting people only takes 5 - 10 minutes, but you should spend 2-4 hours on communicating its purpose

### **BugBug features that you should use**

-   [Invite users](https://docs.bugbug.io/collaboration/inviting-team-members)

-   [Project settings > Notifications](https://docs.bugbug.io/collaboration/notifications#choose-when-people-get-email-notifications)

Step 3: incremental development of automated tests
--------------------------------------------------

### **Goals**

-   Save time on manual regression testing

-   Release faster

-   Prevent cost of regression bugs

### **Approach**

-   Talk to the whole team about what you want to achieve and what's the incremental plan

-   Choose 1-2 people responsible for automated tests 

-   Create dedicated test user accounts that will be used for automation only

-   Run automation on some testing environment (staging)

-   Create separate test suites for staging and prod (run all tests on staging, but only essential smoke tests on prod)

-   Add more tests incrementally, 2-3 test cases per week (after 3 months you should have about 50 test cases automated)

-   Start adding [data-test-id attributes](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors) in your HTML

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [New test from here](https://docs.bugbug.io/creating-tests/new-test-from-here)

-   [Duplicating tests](https://docs.bugbug.io/creating-tests/duplicating-tests)

-   [Components](https://docs.bugbug.io/editing-tests/wip-make-component)

-   [Variables](https://docs.bugbug.io/editing-tests/variables)

-   [Profiles](https://docs.bugbug.io/editing-tests/variables#profiles) to [test on different environments](https://docs.bugbug.io/editing-tests/variables#work-with-different-development-environments)

-   [Suites](https://docs.bugbug.io/organizing-tests/suites)

-   [Manually editing steps](https://docs.bugbug.io/editing-tests/manually-creating-the-test)

-   [Testing registration](https://docs.bugbug.io/editing-tests/variables#test-user-registration-and-login-using-variables)

-   [Custom javascript](https://docs.bugbug.io/editing-tests/custom-javascript-actions) might be needed

Step 4: integrate automation with CI/CD
---------------------------------------

### **Goal:**

-   Improve software development lifecycle, being sure that no code is merged without tests

### **Approach**

-   Integrate automation with your development pipelines

-   Trigger automated tests with every pull request

-   Show test results directly in your repo tool (Bitbucket, GitHub, GitLab, etc.)

-   Ask developers to run automated tests locally on the localhost environment before they commit code

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [CLI](https://docs.bugbug.io/integrations/api)

-   [Overriding variables](https://docs.bugbug.io/integrations/api)

-   [Running on localhost](https://docs.bugbug.io/running-tests/test-in-your-browser)

Step 5: increase coverage
-------------------------

### **Goal:**

-   Get 100% test coverage, be confident that all the features, all the browsers, all the devices work correctly

### **Approach**

-   Pick your battles, choose what is the most essential to automate

-   We recommend you keep adding 1-2 test cases every week and don't try to achieve the impossible 100% coverage

### **Time & Resources**

-   To automate everything you would need an Infinite amount of time :)

-   To automate 2-3 new test cases per week you still need 2-3 hours weekly plus a bit more experience with tests and javascript.

-   Note that the more test cases you have, the longer the maintenance. The trick is to find the right balance.

### **BugBug features that you should use**

-   Project settings > Browser width (for [testing mobile screens](https://docs.bugbug.io/workflow-tips/mobile-version-testing))

-   Lots of custom Javascript

-   [Javascript variables](https://docs.bugbug.io/editing-tests/variables#javascript-variables)

-   [Javascript assertions](https://docs.bugbug.io/editing-tests/custom-javascript-actions#advanced-javascript-assertions)

-   [Parallel running](https://docs.bugbug.io/running-tests/parallel-mode)

Common problems and solutions in implementing automation testing:
-----------------------------------------------------------------

Here are some typical problems you may encounter when you want to start automating tests for a SaaS product. We've made a short list with possible solutions that worked for other [BugBug clients](https://bugbug.io/case-study/).

### **Captcha**

You need to [disable the captcha only for automation testing](https://docs.bugbug.io/troubleshooting/captcha-in-automation-testing). Unfortunately, it's not trivial and you need to read through captcha docs or ask a developer for help.

### **Date-based features, calendars** 

Is your app searching train timetables for tomorrow? It's difficult to automate because the date and the expected outcome can change every day.

Solution A: In the automated test, pick a date that's far in the future. Then update the tests regularly, changing this date and the expected outcome.

Solution B: create a dedicated environment for automation with a mocked-up hardcoded date on the backend

### **Dynamic lists such as search results, product listings**

It's hard to automate a test if some data is changing all the time. You need to create independent tests that are always sure what to expect, for example, if you're checking a list of promoted products on your homepage, your test should first go to the admin panel and mark such products as promoted.

You can also have a separate dedicated test automation environment where you have a hardcoded set of data.

### **Can't use the same user email twice**

Create a new user with each test - use the [built-it variable "testRunId"](https://docs.bugbug.io/editing-tests/variables#use-built-in-variables-for-dynamic-or-random-values) in combination with a [temporary email catcher](https://www.google.com/search?q=temporary+email&ei=ARVtY_X8IomKrwS_i4k4&ved=0ahUKEwi1gZ2u86P7AhUJxYsKHb9FAgcQ4dUDCA8&uact=5&oq=temporary+email&gs_lcp=Cgxnd3Mtd2l6LXNlcnAQAzIECAAQQzIFCAAQkQIyBQgAEJECMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDoKCAAQRxDWBBCwA0oECE0YAUoECEEYAEoECEYYAFDhAVjhAWC1CmgCcAF4AIABa4gBa5IBAzAuMZgBAKABAcgBCMABAQ&sclient=gws-wiz-serp) always to run tests with a fresh new user. You can also delete these users at the end of the test by recording test steps in an admin panel (if you have one) Read more in how to test user registration.

### **Selectors are not reliable**

The only way to make sure your tests are stable and easy to update is to start using "data-testid" attributes in your HTML structure, [dedicated attributes for automation testing selectors](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors). You need to communicate that to the front-end developer and make sure you remember about it during the code review. No worries, adding additional attributes have no risk, it's just additional data for automation with no other function, it will not introduce new bugs.

### **Multilanguage**

Your page can run in different languages and you want to verify all of them. Try to assess the risk of having a bug only in one language without affecting the others. Perhaps it's not likely that such a bug would occur?

If you still need to test different languages, you have two options:

Option A: Record separate tests for each language. Hard to maintain on a bigger scale

Option B: Use variables & profiles to run the same test on different languages

-   Add data-testid to all of your app elements, so that the selectors are independent of the app language and the displayed test 

-   Add a way in your app to change the language via a URL, for example, "myapp.com/?lang=fr"

-   Create a variable "language code"

-   Create different profiles with different language codes

-   Replace the language code in the "Go to URL" step with the variable

-   Create separate suites for all the languages, overriding their profile with a chosen language

**Tests take long time to finish**

[Upgrade your BugBug plan](https://bugbug.io/pricing/) to be capable of [running cloud tests in parallel](https://docs.bugbug.io/running-tests/parallel-mode)

### **Downloading files**

Currently BugBug can't verify if the files have been download. Soon we'll add a new type of assertion for that, [talk to our support](https://bugbug.io/contact/) if you need it ASAP.

### **Drag & drop interactions**

BugBug support for drag&drop is in beta, you can achieve it by combining the "Mouse press", "Hover" and "Release mouse button" [actions](https://docs.bugbug.io/editing-tests/actions). Talk to our support for details.

### **A recorded selector doesn't work**

Sometimes it's not possible to record a reliable selector and you need to use your human intelligence and debug HTML a bit. 

Use our [No-code XPath Selector Builder](https://bugbug.io/xpath-selector-builder/) which makes it easy to craft a good selector manually and can teach you in-depth about [best practices in selectors](https://bugbug.io/xpath-selector-builder/#How-to-create-reliable-XPath-selectors-for-test-automation).

Success in test automation depends on the regularity of your progress
---------------------------------------------------------------------

Implementing automation testing in a SaaS company is a long-term process. You should do it step by step. The biggest challenge for tech leaders is lack of time to dedicate on QA area. The truth is you don't need a lot to push it forward. Better slow than never. [😉](https://emojipedia.org/winking-face/) Block yourself a slot in calendar every week to work on QA automation and keep the momentum going.

medium_bugbug-matrix-illustration.png

small_bugbug-matrix-illustration.png

thumbnail_bugbug-matrix-illustration.png

bugbug-matrix-illustration.png

Tech leader's guide to automation testing

tech-leaders-guide-to-automation-testing

Test automation, especially end-to-end, is currently enjoying a revival. A lot of new tools and approaches have emerged over the last few years. Selenium has finally launched version 4, Cypress and Playwright have brought coded automation to a new level and tools like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim.io](https://www.testim.io/), have revitalized the idea of record and play automation. Since a lot is happening, let's take a look at automation and compare the strengths and weaknesses of 3 approaches to Automation: Custom Made Frameworks, Out of the box framework and Record and play. We will look at the pros and cons of all the solutions and when they work best. However, first we need to discuss the reasons for using test automation.

Why we automate tests
-----------------------------------

If you ask typical testers why they automate tests, they can provide many reasons: from shortening the Regression time, improvement of test coverage, to providing faster feedback. Ultimately, if we look hard enough, all goes back to the one and only goal of companies, as stated by [Eliyahu M. Goldratt in The Goal](https://en.wikipedia.org/wiki/The_Goal_(novel)) - Making money. Testing by itself often is considered a cost - as it does not bring more money. But let's not forget that more money can be made not only by selling more - but by increasing the margin - either by making the product worth the price or by reducing the cost. Testing can help to protect the bottom line, by reducing the cost of mistakes and finding them more easily and faster. This also leads to reducing the cost of customer support, refunds, etc. But as [DevOps](https://itrevolution.com/the-devops-handbook/) and [Modern Testing](https://www.moderntesting.org/) state - testing can synergize with a great experimentation culture, leading to possible delivery of a quality product in a shorter time and cost-savings on the development cycles. But this cannot be achieved without robust test automation. And automation needs to follow a few rules. For that, we need to consider the properties of great automation.

Traits of good automation
-------------------------

For automation to be useful in the DevOps world, it needs to be: 

-   Reliable 

-   Fast

-   Easy to write and maintain new test 

-   Easy to run

### Reliability

If test automation is not reliable, we won`t believe its results. As a result, we will still depend heavily on manual rechecking of everything and that is time-consuming. Or even worse, it can lead to wasting time as now no one will even bother to check the results.

### Fast

The development cycle is fast. If developers need to wait for the results of their work more than a coffee break, they will switch to the next task. Now the results will need to wait which leads to the cost of context switching. And longer [lead time of a task](https://kanbantool.com/kanban-library/analytics-and-metrics/kanban-definition-of-lead-time-and-cycle-time).

### Ease of Writing and Maintaining

If automation should be a regular part of a software development lifecycle, their benefit must justify the cost. If writing a test takes 4 days, it'd better be worth it - especially if developing  the tested feature takes less than that. The same holds true for maintenance: we need to be able to easily understand why a test has failed and whether this is a code issue or a test issue and how to fix it.

### Ease of running

This relates to being fast: anybody who needs to use a test to check a code should be able to do it without extra hoops, for example automatically running them locally or via CI/CD.

How to choose the best automation approach
------------------------------------------

There are a lot of ways of choosing tools for a project. When looking for tools that resolve a problem, we need to choose a tool that contributes the coveted value. The list of questions requiring answers is long, but today we will look at only one aspect, namely automation development and maintenance cost. We will look at the 3 most popular approaches, starting from ones requiring most coding and maintenance effort, like "Custom Frameworks", through usage of "Out of the box" Frameworks and finishing with Record and Play tools. We will look at the pros and cons. As it is my belief, all these approaches have made a name for themselves on the market and suit different project needs.

**Pros**

-   Great Customizability

-   Ownership

-   "On-site support"

**Cons**

-   Huge Maintenance Cost

-   Time Consuming

-   Requires lots of skill to Maintain

When I was starting test automation, this was the way to go. Mainly because Selenium Web Driver was the tool of choice back then. And as its description stated, [Selenium automates browsers](https://web.archive.org/web/20120208115222/http://seleniumhq.org/). Selenium was just a tool that did exactly what it said it delivered.

It didn't provide any functions for running test, or making assertions. 

At that time, practically every company had to build its own framework.

Often it was the role of test automation testers. In most cases, they were just learning how to automate. Making lots of mistakes in the process. But some companies managed to make great frameworks and, as a result, decided to share them with others. Some of them are still in use and developed but other have not stood the test of time. [Ocaramba](https://github.com/ObjectivityLtd/Ocaramba),[Selino](https://github.com/TestStack/TestStack.Seleno), [Serenity](https://serenity-bdd.github.io/theserenitybook/latest/index.html),[Selendie](https://selenide.org/), [Protractor](https://www.protractortest.org/#/)

This approach is still used today. Why?

### Pros

The main advantage of this approach is control. It gives a company the greatest level of control in customization. If a specific feature for a project is required, it is relatively easy to find someone who can develop this functionality.

Additionally, it gives a company considerable independence; there is no risk that at one day, the tool support will end or [repo will be deleted](https://nakedsecurity.sophos.com/2022/01/11/javascript-developer-destroys-own-projects-in-supply-chain-lesson/), leading to massive issues.

What is more, there is someone who knows how the tool works on site and can teach new team members.

### Cons

The main drawback of this solution is the cost - maintaining internal test automation framework costs time and money. Many companies underestimate the cost of developing test automation.

Time is of essence here: this automation needs a long time to be developed,  from creating a framework to writing a test. And this is the aspect that can be most damning. As discussed before, for automation to bring value, it needs to provide fast feedback.

Creating a custom framework is as big a project as any and it requires a good dose of skills, ideas, and planning. This means a lot of expertise while typically, companies are not willing to spend precious resources on test automation.

Out of the box frameworks: Pros & Cons
--------------------------------------

**Pros**

-   Shorter time needed to write first test

-   Much less need for framework maintenance

-   Lots of available online trainings and knowledge

**Cons**

-   Often only limited support from community

-   Writing and Test Maintenance takes time

-   Huge dependence on framework developers
As mentioned before, the growth of Selenium was accompanied by a lot of frameworks created on its base. Over time, the issues with the web driver became increasingly prominent, leading to the re-emergence of new automation frameworks not based on Selenium: [Cypress](https://www.cypress.io/), [Puppeter](https://github.com/puppeteer/puppeteer), [Playwright](https://playwright.dev/), to name a few most recent ones.

All these tools had one thing in common - they provided the user with out-of-the-box, ready options to write automation. No matter if this was Ocarmaba based on Selenium or Cypress: the goal of the solution was the minimalism needed for framework maintenance.

These tools usually provide built-in test runners, an extensive assertions library, boiler plates for starting writing project and tests, and - most importantly- a built-in mechanism to deal with test flakiness and general instability.

Additionally, these frameworks usually provide their own philosophy of writing test automation. As long as what we need to do is with accordance to their approach, everything goes well. Problems start when we need to go beyond what is suggested.

Until recently, the best example has been a test covering multiple domains in Cypress, but as of [version 9.6](https://docs.cypress.io/guides/references/changelog#9-6-0), it is no longer such a grievous case.

In another scenario, we need to rely heavily on framework developers, if they introduce breakthrough changes. Usually, we are left with two options: update and fix the code. Alternatively, don't update and hope that there won't be any issues in the framework that need fixing.

### Pros

Compared to the custom frameworks, we can start writing tests almost immediately without the need of spending weeks or months on writing the toolset first. Similarly, we don't need to wait whenever the operating system or browsers have an update. The tool maintainer will update the framework for us. And since most tools in this category are open-source ones, there is lots of information easily available online.

### Cons

As mentioned before, these tools have their own test philosophy, which is much more restrictive. Quite often support is limited - these tools are usually open-source and support in case of issues can be unavailable where we need to resort to forums and chats where we hope someone will answer a question; sometimes we get the answer in minutes but it may never come just as well.

However, the greatest risk of all is that we are at the mercy of framework developers - these tools become a part of our code base and if we want to keep support, sooner all later we will need to update even if we don't want to. A case in point is Cypress 10 - [it introduced breakthrough changes](https://docs.cypress.io/guides/references/changelog#10-0-0). As a result, many plugins and customer functionalities won't work without extensive updating.

Record and Play Testing Tools: Pros & Cons
------------------------------------------

**Pros**

-   Fast Test Writing Time

-   Low Entry Level

-   Professional Customer Support

-   Easy Scalability

**Cons**

-   Usually cloud-based, requires access to test environments

-   Still young, "untested" technology

-   Requires a new mindset

-   Data management can be quite complicated

-   Usually a commercial paid tool


The idea of record and play automation is not new, even Selenium has Selenium IDE which, after a few years on developments lately, has won some users' recognition.

Microsoft had Coded UI Test as part of the Visual Studio from 2010 and only in [2019 announced it deprecation](https://devblogs.microsoft.com/devops/changes-to-coded-ui-test-in-visual-studio-2019/)

Back then this technology was long before its time, the frameworks were very unstable and the maintenance of this test was problematic.

But in the last 4-5 years, a lot has changed. Solutions like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim](https://www.testim.io/) have taken over what modern technology has to offer and reinvented the concept.

### Pros

Most coded automation requires great investment of time into writing and maintaining test - usually even a trivial change costs around half an hour. In case of Automation Records and Play, test are usually meant to be disposed of  after they  stop working, as it is much easier to re-record a test instead of fixing it. This is of special importance for startups and projects with a lot of changes in the UI layer. When a team is spread thin, it is hard to find the time to write a test. But usually, you can record a few tests while checking if what has been done even works.

On the other hand,e out-of-the-box coded frameworks offer usually an easier learning curve. Record And Play tools usually do not require coding knowledge - although in more complicated cases some JavaScript knowledge can prove very helpful.

Since most of the record and play tools are paid solutions, it also means that companies have professional Trainers and Customer Support to help when issues arise. Usually, it means that in the case of young projects, we can even influence their feature roadmap.

These modern record and play solutions come with their own cloud runners, making scalability of test automation unbelievingly easy. Typically, paralyzing a test takes a few clicks.

### Cons

Unfortunately, Cloud Running is also a huge Con. First and foremost, we need to give this tool access to our test environments. 

An additional issue related to the Cloud is Term of Service - not all projects and customers can afford cloud services.

Most of these Record & Play tools are relatively young and not battle-tested. For long-term projects, this uncertainty can be very risky.

In my opinion, the biggest issue is Data Management. So far, no Record and Play solution has provided an easy way to manage test data. And this can be a huge blocker. Consequently, we need to change our approach to E2E test automation, with a desire to cover all the important and easy paths as fast as possible, to make this automation possibly most valuable.

Lastly, there is the price - cloud solutions are not cheap and if we need to run a lot of tests, it may be quite expensive. But some tools like [BugBug](https://bugbug.io) are on the other hand very cheap and offer unlimited cloud runs for a fixed mohtly fee. Arguably, it is still less expensive than maintaining coded automation and its infrastructure.

No clear-cut distinction
------------------------

Before the final thought, some clarification is required:

As much as we humans like clear categorizing, the real-word borders are much blurred.

For example: should Playwright be in the Custom frameworks or in the Out-of-the-box category? After all, its strength is the ability to use its own test runners, reporting services and easy building of the new features we need.

Similarly, it has the Record and Play functionally and so did Cypress (at the moment of writing this post, the latest version of Cypress is 10.3 and since 10 Cypress Studio, which allowed test recording, has been deactivated).

There are also tools that I like to refer to as combine-harvesters: basically, they do everything and claim that they can test anything, from mobile via web to desktop and many others -- the examples include TOSCA and Test Complete.

That's why we need to look at the numerous different dimensions of automation - and how much coding is just one of them.

Summary
-------

It is my belief that over time, both no-code development and record and play automation become the dominant players on the market. Yet there is a lot of room for improvement. All the types of approaches to automation, starting from custom frameworks, or using out-of-the-box coded solutions ending on Record and Play tools, have their uses on the market.

They answer to different needs and solve different problems. So, before selecting any tool, it is always good to ask yourselves: "What are we trying to gain?"

large_image (3).jpg

medium_image (3).jpg

small_image (3).jpg

thumbnail_image (3).jpg

image (3).jpg

Record and Play vs Coded Automation testing - how to choose what works for your team

record-and-play-vs-coded-automation-testing

We’ve prepared a short guide with essential recommendations on test automation that we created specifically for startups. We divided the guide into 3 levels:

[★☆☆ Level 1: get the most out of basic automation](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/)

[★★☆ Level 2: automate hundreds of regression tests](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)

[★★★ Level 3: integrate with CI/CD builds](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-3/)
  
Also see our guide dedicated to non-technical people: [beginner's tutorial to automation testing with BugBug](https://docs.bugbug.io/in-depth-guides/beginners-tutorial-to-automation-testing)
 
### What will you achieve on Level 1?
After finishing this level you will run 5-10 automated tests every hour and check if your core features are working properly. You will detect production bugs with big impact on users sooner than before.

### TLDR:
-   Start with short and simple tests that check your core features
-   Remember about recording “hover”
-   Create special test user accounts dedicated to automation
-   Schedule the tests to run every hour
-   Incrementally add more tests

### Focus on the most important features

![80-20 rule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/80_20_rule_1_c2126d9815.png)

Many companies spend endless hours on a neverending pursuit to automate everything. This leads to very costly automation that requires a whole team to create and maintain. Don’t be like these companies and keep it simple. Agile startups should start with automated testing of the most essential functionality. 

> 👍 **Rule of thumb:**
> Usually, it just takes 5 to 10 automated tests to be sure that most users are able to achieve their goals.

What should you test? Think of your features. What percentage of users are using a particular feature? If the answer is higher than 80%, then this is a core feature. The goal of automating tests of such important features is to make sure that you can detect bugs that have a huge impact on your business as soon as possible (new user registration is for sure a core feature, but it’s a little bit more tricky to automate, which we will cover in the next episode)






### Start with short & atomic tests

When you [record your first tests](https://docs.bugbug.io/creating-tests/recording-the-tests) start with something very simple and atomic. Atomic means that the test is small, independent and checks just one thing.

A good idea is to start with testing the basic navigation of your landing page. Click “record” and create a test that just clicks one item of your navigation link in the menu. Add [assertion](https://docs.bugbug.io/recording-tests-steps/recording-assertions) for some text in a heading. Finish the test and run it. Awesome! You’ve created your first automated test that touches the most essential area of your business!

![Screenshot 2022-07-25 at 13.44.12 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_12_1_0ebde0ddbb.png)

Now create a separate test for each navigation link. Here are screenshots of how BugBug team set up their landing page tests.

![Screenshot 2022-07-25 at 13.44.15 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_15_1_74fb39e9de.png)


### Don’t forget about recording hovers

If your website navigation appears on hover, you need to tell BugBug that you want to record a mouse moving to a specific element. To do this you need to click “Hover” during the recording and then click an element. When running the test, BugBug will move the mouse cursor to this element.

![bugbug record hovers.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_record_hovers_81639fbd30.png)


### Create special test users

Another idea is to test if login works. Create a separate test that only checks that one thing.

To achieve that you need to prepare a “test user” - a user account that is dedicated only to testing if login works properly

-   Manually register a new user with a new unique email, for example with john.smith+testautomation@email.com (you can use [disposable mail](https://www.google.com/search?q=disposable+mail) or [plus aliasing)](https://support.google.com/a/users/answer/9308648?hl=en)
-   Go to BugBug and record a test that clicks the login button, enters the user login and password and submits the form
-   Add an assertion for some text that confirms that the login was successful
-   Finish your test
    

  

You might be tempted to test other features in the same test, but don’t do this now. We want to be sure that the login test is atomic and independent from other features.

  

Remember that you can create as many test users as needed. For example, if your product offers multiple combinations of preferences, you can create a separate user for each combination.

  

> 👍 **Rule of thumb:**
> Always create new test users if this leads to less test steps

  

### Schedule all tests to run every hour

With BugBug you just need a couple of clicks to start running the tests every hour. Just go to “Schedules”, select the “Hourly” one and add “All tests”.

![hourly schedule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/hourly_schedule_1_445d7fe266.png)


  

That’s it, now you will start getting emails every hour with the results of your test automation. Tip: create a filter on your email account so that you avoid inbox clutter or [configure notifications to only get an email on failures.](https://docs.bugbug.io/running-tests/schedules#configure-email-notifications)

![email notifications (1) (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/email_notifications_1_1_a23f96ba47.png)


  

### Incrementally add more tests

You’ve successfully created test automation that monitors your production! Now keep adding more short tests one by one. Don’t forget to focus on being cost-effective - if you have trouble automating a use case, skip it and go to the next one or find a different way of checking if it works. And remember: you can always [ask BugBug support to help you with any troubles](https://bugbug.io/contact).

Read next: [next level guide on how to easily maintain hundreds of automated tests ](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)
  
  

medium_illustration - guide - level1.png

small_illustration - guide - level1.png

thumbnail_illustration - guide - level1.png

illustration - guide - level1.png

Automation testing guide for startups - Level 1: get the most out of basic automation

automation-testing-guide-for-startups-level-1

In the [first episode](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/) of our 3-part guide, we covered the basics and described how to get the most business value from simple test automation. Now it’s time to learn some more advanced concepts and increase coverage of your automated tests. Here we go!
 
  
### What will you achieve on this level?

Once you finished this level you will:
-   have 100+ nicely structured tests
-   be able to update multiple tests at once when needed
-   run tests on different environments
-   automate testing of new user registration
    

  

### TLDR:

-   Organize your test steps in groups
-   Convert some groups into shared components
-   Duplicate tests efficiently with “new test from here”
-   Set up a URL variable to quickly swap between environments
-   Set up a testing email inbox to automate tests of new user registration
-   Use [built-in variables](https://docs.bugbug.io/editing-tests/variables#use-built-in-variables-for-dynamic-or-random-values) to generate random user email aliases
    
### Scaling the test automation

![illustration-number-of-tests.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/illustration_number_of_tests_7ee4d49d68.png)


In this episode, we will focus on scaling your test automation to a larger number of test cases so that your regression tests also cover corner cases and niche features. This is “Level 2” so things will get more complicated, but BugBug makes everything easy to achieve.

Example: in [the episode 1 of this guide](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/) we’ve created just one test that checks if log-in works. Now we want to create several tests that check different outcomes of logging in, for example:

-   Log in - success
-   Log in - incorrect password
-   Log in - account blocked
-   Log in - password reset required
-   Log in - account doesn’t exist
    
You can see that we aim to increase the test coverage dramatically. Instead of 5-10 tests, we’re going to have hundreds of them.

### Keep tests nice and tidy

If you want to have more than 10 tests, you should start organizing the test steps and keep them in order. Why is it important? Imagine that you’ve created 100 tests, but later you changed something in the UI. All of your tests started to fail. If you haven’t organized test steps, you will need to go to each test individually, see what’s wrong and re-record the failed steps 100 times. Fortunately, BugBug allows you to easily avoid such situations.

### Group your test steps

Keep your steps tidy by grouping them. Give a clear name to each group, which will help you to navigate through long test cases. Move your cursor in between the steps to see a plus icon, then click “[split group here](https://docs.bugbug.io/editing-tests/grouping-steps)”.

![split-group-here.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/split_group_here_fce12649d5.png)


  

### Convert groups to shared components

Once you start grouping your test steps, soon you will realize that some groups are exactly the same in multiple tests. This makes them a perfect candidate for [components](https://docs.bugbug.io/editing-tests/wip-make-component). Components are magic groups: if you change it in one test, it will change in all other tests that have it.

![component-made (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/component_made_1_afa56da91d.png)
  

Start with creating a component for the first step that goes to your starting URL. Then go to each test and replace the normal group with the component by using “Insert existing component”.

![insert-existing (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/insert_existing_1_985f99b0c0.png)


  

### Duplicate the tests rather than create new ones

Since you already have components, you may not want to record each test case from scratch. A lot of your tests will start with the same components (for example: go to homepage, log in, enter a password, etc.). You can quickly create a new test “in the middle” of a different test. When you click “[new test from here](https://docs.bugbug.io/creating-tests/new-test-from-here)” you will create a duplicate of the test that keeps all the preceding test steps.
 
![new-test-from-here.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/new_test_from_here_5d79377e02.png)


  

If you use components in a smart way, you will be able to fix hundreds of tests with just one [re-recording](https://docs.bugbug.io/recording-tests-steps/re-recording-steps).

### Use data-testid attributes as selectors

Automated tests are most reliable if your [selectors](https://docs.bugbug.io/preventing-failed-tests/selectors) are clearly identifying an element. The best way is to clearly annotate such elements in your HTML structure. Ask your frontend developer to add `data-testid` HTML attributes to elements that you click in your automated tests. 

Why is it better? Let’s see an example. If you don’t use a dedicated attribute, your login button selector might look like `//SPAN[normalize-space(text())="Log in"]`. This selector points to any element with text "Log in". If you change the text in the button, such a selector will stop working and your test would fail. 

If you used an attribute, for example `[data-testid=”login-submit”]` your tests will be more resilient to changes. Even if you change the text in the button, you will not need to edit your automated test. What’s more, the developers will see this attribute in the code so they will be aware that this specific item is a subject of test automation. Once you have such attributes in your HTML, BugBug will automatically create selectors based on “data-testid” attributes during the recording. 

![data-testid.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/data_testid_669fc25baa.png)

### Build robust selectors without coding

Sometimes it's not possible to create a reliable selector with recording. Dynamic lists, tables and hover actions are notorious for being difficult to record. You just need to use your human intelligence and use a more complex XPath selector. We created a dedicated tool for that with an in-depth guide - see [No-code XPath Builder](https://bugbug.io/xpath-selector-builder/).

![no-code-selector-builder.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/no_code_selector_builder_b90d46b1c1.png)

### Work with different environments

If you are fully committed to testing automation, chances are you will need to run tests on different environments. Each environment can be accessed with different URLs, for example:

-   Production - acme.com
-   Staging - stage.acme-test.com
-   Local - localhost:3000

You should store these URLs in a variable with 3 [profiles](https://docs.bugbug.io/editing-tests/variables#profiles). After you set up more than one profile, a new dropdown menu will appear in each test, that lets you run tests with different environments. Learn more in our [detailed guide](https://docs.bugbug.io/editing-tests/variables#work-with-different-development-environments).

![profiles (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/profiles_1_e2d0e47b36.png)


  

### Automate tests for new user registration

To reliably automate user registration you need to use a combination of several advanced features

-   Third-party inbox that supports [plus-addressing](https://www.msoutlook.info/question/886#:~:text=Exchange%20Online%20(Microsoft%20365)%2C,also%20known%20as%20%E2%80%9Csubaddressing.%E2%80%9D) (for example Outlook.com)
    
-   Built-in “testRunId” variable for generating random e-mail address alias
    

  

Learn more in our separate step-by-step [guide on testing sign-up.](https://docs.bugbug.io/editing-tests/variables#test-user-registration-and-login-using-variables)

### Keep it simple

Don’t try to overcomplicate your test steps. Use components and variables only when it will actually save time for you in case of product changes. Remember that re-recording is a really fast way of fixing existing tests - sometimes it’s just faster to re-record steps in 5 tests than debugging a complex net of variables and dependencies. Keep it simple and if you have any problems feel free to [talk to our friendly customer support](https://bugbug.io/contact).  😊

[Read next: advanced guide with tips on CI/CD integration](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-3/)

medium_illustration - guide level2.png

small_illustration - guide level2.png

thumbnail_illustration - guide level2.png

illustration - guide level2.png

Automation testing guide for startups - Level 2: automate hundreds of regression tests

automation-testing-guide-for-startups-level-2

Tools for automation