Product

Selenium IDE is a tool for recording and launching application tests. It was created by Shinya Kasatani in 2006 (sic!) and is based on „record & playback” idea, which nowadays is no longer a novelty. The problem with most of the tools available on the market (including Selenium IDE) is not the outdated idea, which basics still can be perceived as elegant, but its realisation. Below we listed the biggest accusations that can be directed towards Selenium IDE and solutions to them offered by the tool named BugBug.

## Generation of improper elements’ selectors

While recording the application tests, the newest version of Selenium IDE is able to generate several selectors for the element with which the interaction was made. The problem is that this tool doesn’t give you the possibility to configure which type (or types) of selectors should be set as default ones. It also prefers wrong approach to the generation of selectors, based on the names of the CSS classes. Nowadays, when CSS-in-JS approach is becoming more and more popular (i.a. due to [styled-components](https://styled-components.com/), which generate random class names), it makes the user responsible for modification of all selectors generated during the recording session. The result is that the work that should be automatically done by the tool is done manually and at a much slower pace by the user.
->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Solution to the problem provided by the BugBug

During a recording session, BugBug – just as Selenium IDE – generates many selectors for the elements with which interaction was made. The difference is that in this case the user decides about the hierarchy of the generated selectors. It can be easily modified in the configuration of the project:

![bb-blog-6-1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_1_373223ad89.png)

Thanks to this solution you have complete freedom and control over the generated selectors – you just act accordingly to the project requirements. Nowadays, testing in the projects very often is based on the data-test or data-testid attributes, that are also preferred by the BugBug. However, if you have to deal with the older type of the project, you can always prioritize generation of the selectors based on the CSS classes or elements ID. The key here is that the user independently decides what he needs and changes configuration of the project accordingly.

While recording the application tests we are not always sure if the selectors are correct. That is why the BugBug at the first launch and after each change in the test verifies selectors and corrects them if there is such a need – for example if the selector disappeared for some reason (it could be caused by the random CSS classes) or it points to the improper element. Thanks to implementation of these mechanisms interference of the tester with the process of the selectors’ selection is reduced as much as possible. He also saves his precious time, so far wasted for manual searching and creation of the selectors.
->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Lack of active waiting for the elements (waiting conditions)

Another common problem with the record & playback tools, which concerns also Selenium IDE, is lack of active waiting for the elements with which interaction should be made. In practice it works like that: if the chosen element is embedded on the website (in DOM), Selenium IDE attempts to interact with it, even though it can be invisible, moves, animates or is covered by another element (loading overlay). The result of such scheme of actions are so-called “flaky tests”. This notion defines the tests which once work, and another time they don’t. The reason for such behaviour is very often lack of appropriate waiting conditions for the element with which the interaction is to be made.

## Solution to the problem provided by the BugBug

Creators of the BugBug took into account defects of the tools available on the market and effectively addressed them. On the level of the project configuration you can define waiting conditions that have to be met so that the interaction with the element could proceed:

![bb-blog-6-2.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_2_1eed216be1.png)

The global configuration of the project is inherited within all steps of the test. Of course there is also possibility to overwrite this configuration on the level of the single step. You can also add additional conditions there, such as waiting for the page reload:

![bb-blog-6-3.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_3_ed8486fb8b.png)

Thanks to this solution BugBug emulates the human behaviour, what means that it waits for an appropriate moment to proceed with the particular step of the test. The tool can actively wait when:

- element animates or changes its characteristics (CSS animations and transitions): the tool identifies that the element animates/moves and waits till the end of this process.
- element is invisible: BugBug’s mechanism is able to assess whether the element is in fact visible for the user. If so, it proceeds with the planned steps.
- element is covered by another element.
- element is inactive.
- element changed its position: the tool is able to identify change of the element’s position while it is proceeding with the step. In such a scenario the interaction is repeated.
 competitors.

Implementation of all the mechanisms listed above reduces the risk of „flaky tests” to minimum. Thanks to that BugBug offers much more stable tests than its competitors.
->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Misleading process of replaying the steps (with the use of JavaScript)

Selenium IDE as well as most similar tools available on the market proceeds with the steps of the test that only partially emulate behaviour of the average user. In practice it means that tests recorded this way can behave very unpredictably, leading to serious errors. It would be easier to explain with the use of the specific example:

The step click on any „Log in” button realised within Selenium IDE would trigger the following event in the JavaScript browser:

- click

## Solution to the problem provided by the BugBug

The same step realised in BugBug results in the sequence of the following events:

- mousemove
- mouseover
- mouseenter
- mousedown
- mouseup
- click

The difference described above stems from the constraints of the Selenium IDE architecture, which can emulate the user’s behaviour only with the use of JavaScript. Architecture of the BugBug and implementation of the [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/) gives it full control over the browser. Thanks to it the behaviour of the user can be accurately emulated. It is crucial in testing contemporary, very extensive and complex applications (Single Page Applications).

## Application tests are not launched separately

Majority of the record & playback tools has a problem with performing the tests separately. It results in a situation, when launch of the particular test impacts another one. The problem resides in the fact that cookies or localStorage mechanism in the browser are shared between windows and tabs. When you use Selenium IDE, you have to always delete cookies (or localStorage) and remember about the necessity to clear this data between the tests.

## Solution to the problem provided by the BugBug

Despite the fact that the problem described above is simply burdensome, it also causes flaky tests. To prevent this, application tests in BugBug are by default performed separately with the use of Incognito mode in the browser. Thanks to that QA who builds the tests doesn’t have to remember about cookies, localStorage and other data sharing mechanisms in the browser. Tests are isolated regardless of whether you launch them in the browser or in the cloud.

![bb-blog-6-4.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_4_136c914e62.png)

## Launching tests in the cloud is impossible

Selenium IDE was created many years ago, when cloud and SaaS services weren’t so widespread as they are nowadays. As a result, Selenium IDE functions only as the browser ad-on and doesn’t offer testing in the cloud. Tests created with the use of Selenium IDE can be exported to the chosen programming language and then launched within your own infrastructure (or with the use of some commercial solution).


Solution to the problem provided by the BugBug

BugBug from the very beginning was designed as a tool that aims at supporting the whole testing process (end-to-end). It enables for:

- [recording and easy editing of the tests.](https://docs.bugbug.io/core-features/recording-the-tests)
- launching of the tests locally, within user’s browser, in order to verify the tests.
- planning and launching application tests in the cloud (through www interface as well as CLI tool).

Look in our previous post to see [10 features every test automation tool should have](https://new.bugbug.io/blog/10-features-every-test-automation-tool-should-have/).

Thanks to that you receive one, integrated tool that can be used for creation and launching of the tests as well.

![bb-blog-6-5.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_5_94ab9b819c.png)
->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Lack of support for the Page Object Model (POM) pattern

Page Object Model (POM) is a verified design pattern for end-to-end tests, which enables you to freely re-use steps within the tests, thus simplifying its maintenance. This pattern bases on creating group of steps that represent pages of tested Internet application as so-called Page Object. Thanks to that after you create “Log in” Page Object you can use it multiple times – in each test that requires logging in. If the log in form changes, it suffices that you fix the test in one place – as a consequence all tests based on “Log in” Page Object would be changed accordingly.

Unfortunately, Selenium IDE doesn’t support creation of the tests based on the Page Object Model. Steps in each test are strictly assigned to it. When you have many tests scenarios that use the same steps, if one step stops working, you have to fix it separately in each test.

## Solution to the problem provided by the BugBug

BugBug was designed to support Page Object Model pattern from the very beginning. This tool enables you to freely group the steps in tests and use them as re-usable elements. What is more, a natural consequence of such an approach is the possibility of creating visualisation of the tests in one, coherent graph. Such presentation of all tests gives you clear information about which business paths within the project are covered with end-to-end tests.

![bb-blog-6-6.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_6_bf3d39ac25.png)

## Summary

![bb-blog-6-7-768x480.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_7_768x480_7b04bf2e6c.png)


large_bb_blog_6_23f3987e2a_a8876041ba.png

medium_bb_blog_6_23f3987e2a_a8876041ba.png

small_bb_blog_6_23f3987e2a_a8876041ba.png

thumbnail_bb_blog_6_23f3987e2a_a8876041ba.png

bb_blog_6_23f3987e2a_a8876041ba.png

Application Tests: BugBug vs Selenium IDE

application-tests-bug-bug-vs-selenium-ide

Features

Pricing

Roadmap

Integrations

Case studies

Blog about test automation

Privacy policy

Terms of service

About us

Careers

Book a demo

Contact

Get more tips and product related content. Zero spam.

Don't miss any updates

Automate web app testing easier than ever. Without excessive costs. Faster than coding. Free forever.

End-to-end testing made simple! BugBug is a new, more intuitive tool for test automation of web apps. Get all the benefits of test automation without hiring new people or infrastructure changes.

BugBug.io | Intuitive and reliable test automation for startups

large_opengraph_image.png

medium_opengraph_image.png

small_opengraph_image.png

thumbnail_opengraph_image.png

opengraph_image.png

Software Testing Knowledge

[TABLE_OF_CONTENTS]

Compatibility testing is non-functional testing, which means it focuses on such aspects as the reliability and usability of software applications. Thanks to focusing on the compatibility of the software, you can make sure that your app will work as expected on different hardware and software configurations.

You can say that there’s software compatibility when various modules and components of software work together and exchange information to produce expected results that satisfy the user of the software. In other words, if an app works with other apps, systems, and hardware doing what it was created for, it’s compatible with them.

## Software compatibility testing is important, but what is its purpose?

As a part of non-functional software testing, compatibility testing is a method that allows you to check if an application or software works correctly with different software and on various hardware configurations. The goal of compatibility testing is to ensure that there will be no problems when your app runs on a specific device, platform, or together with other apps. An incompatibility issues will result in bugs and problems that you can detect thanks to testing and fix early in the process.

## Key objectives of compatibility testing process

- The key objectives of performing a successful compatibility testing process are:

- Checking if the software is compatible across various platforms, devices, operating systems, or hardware configurations.

- Finding problems and fixing them before the software is released.

- Improving user experience since more users with different configurations can use an app that has good compatibility.


->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Types of compatibility testing

There are two main kinds of compatibility testing: backward and forward compatibility testing.

### Backward compatibility testing

It's when you check if your app works on older hardware or previous versions of operating systems (the latter is often referred to as operating system compatibility).

### Forward compatibility testing

It’s the process opposite to that of backward compatibility testing as it consists in ensuring that your app works correctly on new platforms and hardware.

Also, we can distinguish more specific subtypes of compatibility testing like the ones discussed below.

**1. Browser compatibility testing**

Making sure that your website or web app works correctly on various browsers is highly important given how many browsers there are. It usually involves checking the layout and features on such popular web browsers as Chrome, Edge, Safari, and Firefox.

**2. Network compatibility testing**

Most apps today use networks and connectivity to function properly. Even if your software doesn’t require an internet connection to run, some features, such as updates may be unavailable if the app isn’t connected to the internet. For this reason, it’s essential to check if your app is compatible with different network configurations and different conditions, for example, low download/upload speeds or latency issues.

**3. Device compatibility testing**

People will use your app on such devices as phones, tablets, and notebooks. Different devices have different operating systems, hardware configurations, and screens. If you offer your app for many types of devices, it’s vital to make sure that you have dedicated versions of your app that have been thoroughly tested on each platform.

**4. Mobile compatibility testing**

This kind of compatibility testing concentrates on mobile devices. As mobility is highly important for modern users, you should make it a priority too.

**5. Software compatibility testing**

Software interactions are not uncommon. For example, two different antivirus applications won’t get along too well. You should check if your software is compatible with other popular apps that people may regularly use.

**6. Version compatibility testing**

Apps get updated frequently these days. It’s important to know that after an update, all the features of your app will still be working. You’ll need to check, for example, libraries, app components, and other elements that might be affected by updates.

**7. Database compatibility testing**

As applications often store their data in databases, you’ll have to make sure that your software works well with various databases. The testing focuses on checking how your app handles queries and how it interacts with different databases.

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Key components of compatibility testing

### Test environment setup

It’s important to make your test environment mimic that of production. To do that, you have to make sure that your hardware, software, and data are identical or at least similar to those used in the production environment.

### Test case design and execution

You’ll need to prepare a set of tests to check the compatibility of your software. First of all, you will need to list elements that you’ll have to test. Secondly, you’ll need to establish what the prerequisites for testing are. In other words, you’ll have to determine such conditions as hardware and software configurations. Finally, you’ll prepare the test cases for each defined testing condition. Once you’ve completed these steps, you’re ready for test execution which involves reporting, tracking, and managing the detected bugs as well as retesting and reporting after you’ve finalized the test execution stage.

### Test coverage considerations

Determining what test coverage should be is vital as it helps to make sure that you can identify all the compatibility problems and find solutions for them. It involves establishing what the target configuration will be (including hardware and software), specifying the requirements that need to be met, and understanding the software architecture and its implementation (because the way software was designed and implemented may lead to compatibility issues).

### Compatibility test automation

You can (and should!) use automation tools that will allow you to create, schedule, and run automated test tasks for you. Such an approach will give you:

- greater speed of testing since you can have tests running on many platforms at the same time.

- higher efficiency and accuracy because taking out the human factor allows for consistent and comprehensive execution of test cases.

- generating savings as not having to rely on manual testers can significantly reduce costs.

## Steps to follow if you want to perform compatibility testing

Once you've decided to perform compatibility testing, you can follow these steps:

1. Determine what the designated platform for your app is. 
2. Get your hands on the required hardware, operating system, and any software related to the designated platform that will be needed for testing your app.
3. Set up your app and configure it.
4. Run it on the designated platform and test how the app works.
5. Find, record, and report any issues with compatibility that you encounter.


## Example of compatibility testing

Imagine that a software company has built a Windows app. They will need to find out how their app behaves on different versions of Windows, for example Windows 11 or Windows 10, as well as on Windows computers with different configurations. That’s why they’ll have to test the app for compatibility. Also, the company will need to test the app under various conditions which includes compatibility testing in multiple network configurations or with different amounts of data that the app will have to handle.

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Benefits of compatibility testing for QA teams

### Ensuring software functionality across multiple platforms and environments

The main goal of compatibility testing is to make sure that your applications works on various platforms, configurations, and under different conditions. The rule here is simple: the better the compatibility of your app, the more users will be using it.

### Mitigating risks associated with incompatibilities

Compatibility testing can help reduce risks of compatibility-related issues. Thanks to such testing, you can identify risks early before they turn into real issues for your users.

### Enhancing user experience and customer satisfaction

High compatibility of your product improves the satisfaction of your customers. Given how many users with different configurations there are, software that has a high compatibility level will satisfy more people and positively affect the image of your software development team and company.

### Supporting product scalability and market reach

Quality Assurance specialists can use compatibility testing to ensure that their software works correctly on various platforms and under different usage. And if the app can handle more data, bigger workloads, and a growing user base, it means that their product shows good scalability. It also results in a better market reach as more compatible products appeal to users of different devices and platforms so the company can reach a wider audience.



## Best practices to follow when compatibility testing is performed

### Comprehensive test planning and strategy development

By focusing on test planning and strategy, QA teams can effectively test their software for various compatibility issues. A good test strategy goes a long way and will help you avoid dealing with frustrated users who experience compatibility problems.

### Testing early and often

The sooner you start testing, the more compatibility issues you’re likely to identify and address. Test early and test often as it increases your chances of finding compatibility issues in the most bizarre configurations and scenarios that your creative users can come up with.

### Prioritizing most critical issues

First things first. Make the compatibility issues that are the most common and affect the greatest number of your users your top priority. You won’t be able to fix every tiny problem, that’s why it’s important to focus on compatibility scenarios that really matter. If you find some time after fixing your top compatibility priorities, you can always move on to other issues.

### Continuous monitoring and reporting of compatibility issues

Be vigilant and be on the lookout for any compatibility problems by establishing a good monitoring process. Monitor and report issues as soon as you spot them. Sometimes such a process will help you catch an issue before it’s noticed during regular testing.

## Tools for compatibility testing

When choosing a good compatibility testing tool, you should focus on what devices and browsers a specific solution supports as well as features that it offers, including testing methods or report generation.

### Overview of popular compatibility testing tools

When it comes to compatibility testing, you have a plenty of great tools at your disposal. Some good examples include:

**BrowserStack** - a cloud-based solution that offers cross-browser testing capabilities.

**LambdaTest** - another cloud-based platform that provides parallel testing and mobile testing features.

**TestComplete** - a commercial solution that you can use to test web apps as well as mobile and desktop applications.

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Conclusion

Testers should dedicate their time to compatibility testing because that way, they can make sure that their software will work seamlessly on multiple different devices and platforms. Early detection of compatibility issues can help fix them before the new release hits the production environment and users start submitting support tickets. In other words, in the long run, compatibility testing can positively affect the company’s image and user experience.

## Key takeaways and recommendations for effective compatibility testing

The most important things that QA specialists and testers should remember about compatibility testing is that testing should be done often and early in the development process. Always test on multiple different configurations and platforms to detect as many compatibility issues as possible. Document and report all problems and keep on monitoring for compatibility problems all the time.

## Stay updated with emerging technologies in the field

New software versions and devices enter the market every day. As a tester, you should keep up with all tech novelties and approach from the compatibility standpoint. It’s important because many of your users may be early adopters of new technologies. They will also be the first ones to point out any compatibility issues, so create a good compatibility testing strategy, test the application that your team's been developing, and stay ahead of any compatibility problems!



small_Test plan template (5).png

thumbnail_Test plan template (5).png

Test plan template (5).png

What is Compatibility Testing?

what-is-compatibility-testing

Software developer currently working as a tech writer and PM.  
[linkedin.com/in/marcin-%C5%82ojewski-a4191518/](https://www.linkedin.com/in/marcin-%C5%82ojewski-a4191518/)

large_DSC04042-Edit-3.jpg

medium_DSC04042-Edit-3.jpg

small_DSC04042-Edit-3.jpg

thumbnail_DSC04042-Edit-3.jpg

DSC04042-Edit-3.jpg

Marcin Łojewski

[TABLE_OF_CONTENTS]

Recently we have covered the test plan and we did quite a good job doing it. The document itself covers the testing process quite thoroughly. We agreed that the core of the test plan are the test cases and test suites. Now let’s shift our focus a bit more on the test cases and test scenarios and as a bonus we will come back to the test suites for a moment or two. 

Both test cases and test scenarios are crucial components of the testing process. Unfortunately it is not uncommon that they are not defined and understood properly.  Also it happens that they are confused with each other. The ability to set them apart is very important for proper test execution. Each tester, QA, test manager should have this knowledge. We think a high-level understanding would be beneficial even for the business people as well. We will try to be professional but not too technical in this article so hop on board.

## Test scenario

Test scenario shows a specific use case of a given functionality. It can be written in a non technical way as it describes an expected (positive and negative) result of a high level action. What is also significant for a test scenario, is that it might touch multiple layers of your software. If we were to create a smooth definition, we would aim for something like the following.

**A test scenario is a described pattern of a positive or negative action performed by the software’s user.**

Remember our article on [test plans](https://bugbug.io/blog/software-testing/how-to-create-a-test-plan/)? Let’s use the same example app to create an example of a test scenario. 

*“Only an authenticated user can see the salary on a job posting subpage”.* 

This is a great example, because one simple sentence describes triggering multiple functionalities. This test scenario contains a basic action that the user may perform on the web application. There are multiple processes and actions involved. 

What does the user see instead of the salary? If the user is not logged in, does he/she see the login button or link? Does the authentication work properly? Finally, when the user does log in after moving from the job posting subpage, is he/she redirected back to the same subpage after authentication? 

We can go on and on about that one, but hopefully you get the point now. 

There is a few additional key information here. 

- Test scenarios can not be automated and are not really reusable. They cover multiple aspects of your software and may include many variables. 

- A test scenario may be prepared by a technical as well as non-technical person. Usually the business people will tell what the app should do, and then the developers will have to know how to achieve it. It is the same in testing.

- Test scenario covers the end to end functionality of the software that needs to be tested.

- Remember that test scenarios may come from outside your testing team, it is common that some scenarios are derived from use cases that happen after the app is complete. The final users are great in finding ways of "breaking" your app. Ways that you would have never thought of. Best practice alert! If it is sound from the business perspective - foresee a beta test period for your app.


Can you perform software testing just based on a test scenario? No, you need to be more thorough. That is why you need something more detailed. You need a...

## Test case

A test case lets the tester know how to test a given feature of your app. We have to emphasise the singular here. The rule of thumb is one feature - one test case. Test cases are derived from test scenarios. It will usually include some input and expected result. Most test cases are written by and for a technical person, so expect more programmers’ vocabulary than it was in the test scenario. Our definition of a test case follows:

**A test case is a set of instructions, inputs and expected outcomes that is needed to validate the proper functionality of one software feature.**

Some technical knowledge and understanding the subject matter of your software are necessary to write test cases. As stated in the definition, we do not only need to describe what needs to be tested. We also need to know what is the data we want to use and what is the expected outcome.

Let’s jump right in and try to produce a test case from the test scenario above.

*Test case #1 - verify if the salary is hidden in a job post*

*Actions:*

*- Go to “/ads/job/2” url*

*- Scroll to div with id = “salary”*

*- Expected outcome is “Please log in to see full details” with “log in” being a link to “/login” url*

*- Failure condition is “$100 per week”*

*- Report any different outcome*

As you can see there are a few different elements that make a test case. The case ID - just for clarity. The case name or title if you will - a brief description of what is being tested. Last, but not least there are the test steps for the test engineer.  

Such test cases are easy to reflect in test automation tools. Some of them, like BugBug, will even enable you to create many test cases with using little to no code with their drag and drop functionalities. 

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## The difference between Test case and Test scenario in software application testing

Trying to compare a test case and a test scenario is like trying to compare a book and a page. One could exist without the other but you have to agree that one page will never give you the whole context, right? It is the same for test scenario vs test case recognition.  

In software testing it is quite tricky to find actual similarities between scenarios and cases. They both serve a testing purpose, have a designed scope and purpose. The key is not to confuse them. A testing scenario describes user behaviour and may (should actually) be transformed into a list of technically defined single test cases.

Let's focus on the key differences to have a better understanding what a test scenario and test case is.

|                                            **Test scenario**                                           |                                    **Test case**                                    |
|:------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------:|
| Everyone related to the project can write test scenarios                                               | Technical personnel is responsible for creating test cases  and performing the test |
| A test scenario covers end-to-end behaviour of the user  what can go across multiple functionalities   | Test case contains one single functionality to be tested                            |
| Is created with no possibility of reuse or automation;  includes many variables and is not a procedure | Is reusable and easily adaptable for test automation                                |
| Informs what needs to be tested                                                                        | Informs how to perform every test                                                   |
| Strategically overviews how the users will be using  the software                                      | Gives some structure to the testing process                                         |



If we were to simplify the matter as much as we can, remember this. A test scenario is basically a set of test cases. There can be one or many test cases for a single test scenario. 

![Test scenario 1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Test_scenario_1_94473c7c69.png)

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## How to write Test cases and Test scenarios?

After the first few paragraphs we are sure that you are a person clearly understanding what a test case and a test scenario is. Creating test scenarios is easy for people with a high level overview of the software whereas test cases are easier for people with more experience in the software testing field. Whichever description suits you more, it is crucial to stick to the how / what division. 

You should not overthink while creating a test scenario. Take the easiest approach possible and become the user for a moment. Think of what you would like to do in your app and then write it down. It should be that easy. Try not to consider (and describe) what different functionalities would your scenario trigger. This is for the test cases to check.

When you are creating test cases for each scenario, your job will be a lot more thorough. We suggest the "from the general to the particular" approach. 

Once you have the test scenario, try thinking about what functionalities it triggers and list them down. Afterwards analyze how those functionalities should and should not behave under the circumstances given in the scenario. Then provide or mock the data that need to be used in the testing effort. Last but not least, define the positive and negative outcome of each test. 

For starters try using our template from this article. We do encourage you to experiment and adapt such templates for your testing environment. Remember that creating test cases is a one-time job, an investment if you will. They are the core of test automation and are easily reusable throughout the testing process. 

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-

## Test case vs Test scenario - summary

At the very beginning we promised adding the test suite into the equation. Thanks for sticking around for that one. In our last article we described the test suite as a set of test cases. Here we do the same for a test scenario. There is a key difference though. 

A test suite is a set of test cases relevant to each functionality and is an artificial entity that helps to manage the testing process. 
Whereas a test scenario can cover multiple functionalities and is an effect of natural user actions performed in the app. 

They both cover multiple test cases, but a test suite can be automated whereas test scenario can not.

We hope we were able to convince you that test scenario and test case are two completely different beings used during the testing of an application. The scenario focuses more on the "what" and the case focuses on the "how". Remember that test cases are derived from test scenarios and not the other way around. From the good practice perspective it is wise to encourage your users to beta test as they have much more creative ideas on how to break your app. 




small_Test plan template (4).png

thumbnail_Test plan template (4).png

Test plan template (4).png

Test Case vs Test Scenario: Compare, Contrast and How to 

test-case-vs-test-scenario

[TABLE_OF_CONTENTS]

We recently went through the whole theory on the purpose of such document. We hopefully proved to each tester that such a document is more than a list of features to be tested. A good plan vastly reduces the risks associated with the whole process. It is time to present you with our sample test plan document. In the following article you can read our test plan example as well as get a sample template so you can write a similar one on your own. 

Before we do that, we need an example app, so welcome do BestAdPortal! (disclaimer - I found no such portal so I'm calling "dibs" on the name). 

From software development standpoint it is a most basic app, that does the following:

- user account functionalities (create, store, password reset, authentication, limiting access to given pages)
- ad related functionalities (CRUD)
- internal message system with email notifications
- billing functionalities 

The list is just an example, we can focus our tests on.




## Sample test plan

### Test scope and objectives

This is the place where you put the comprehensiveness of the testing effort. In our case the test approach might be as follows.


*In a few sentences we were able to describe what we are going to test, and why we limit the testing efforts. On top of that let's move to the identification of the major testing goals.*

*We would consider the test deliverables as complete if all of the below occur:*

*- All CRUD operations on the User can be performed flawlessly*

*- The User authentication process is functional*

*- All user access levels are functional*

*- All CRUD operations on the Ad can be performed flawlessly*

*- The internal message system is functional*

*- The message system cooperates with external email tool*

*- The billing is done properly (i.e. no double billing occurs during the testing effort)*

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-



### Test environment

As you might remember from the last article - [how to create a test plan](https://bugbug.io/blog/software-testing/how-to-create-a-test-plan/), this is the place to describe the resources that will be used to test the designated app. We first describe the general info in bullets and then present the selected tool as well as other information on how and where does the testing activity take place.

*To perform testing activities the requirements are as follows:*

*- A test automation tool*

*- Designated communication channels: email address and Slack channel plus access to Jira and GIT*

*- Implementing the testing tool to our CD/CI pipeline*

*- Separate server space and a test environment setup accordingly to the CD/CI pipeline so the latest released app version is in place*

*- Dummy data creation functionality in place, at least 10 users (2 admins and 8 clients) and at least 50 ads created*

*The decision was made to use a low-code or no-code test automation tool. As we have no tester per say in the team, this is the best decision from the business perspective. After thorough research we decided to choose BugBug as it has low entry requirements, is easily integrated with communication clients as well as CD/CI pipelines.*

Providing the test environment allows a full understanding on what testing resource is needed to perform each part of testing services.  


### Test strategy and approach

We have already described the testing scope, objective and the necessary tools. It is time to let the recipients know what human resources are we going to utilize and what would be the high-level approach to their work.  Remember that it is not yet the time to define the schedule of testing. This chapter is also a good place to describe a high level situation of the app if it was not stated before. You can never foresee the changes in the team and what could be missed in the onboarding process. 

*In order to successfully complete the testing goals listed above, we need to consider our situation first. Given the agile approach to software delivery and MVP status we have decided that the testing efforts should be first and foremost iterative and highly automated. Each test milestone will cover the functionality provided in the latest development sprint.*

*The testing team will consist of the following:*

*- John Doe - test manager - major testing tasks and estimation*

*- Jane Doe - tester 1 - test execution, collecting reports for bugs encountered during the testing process*

*- Jeremy Doe - tester 2 - test execution and updating test scenarios*

*- Julia Doe - communication between the testing and the development team, assistant test project manager.*

*All of the above will be involved in testing for 20 hours per week.*

Remember that the business and finances people also must approve this plan. This is why the example covers also information that helps with the estimation of the time required. 

->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-



### Test cases and test suites

The core of the document. This is where you write the test scripts that will be executed. The software backlog is something that will support the test suites and cases for you. Given that we have chosen the iterative approach it will be relatively easy to pull the functionalities from the milestones identified in the software planning. The following also allows you to define any additional test milestones. 

| ID | Test suite          | Test case                           | Description                                                                                                                                      |
|----|---------------------|-------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|
| 1  | User creation       | Proper data in user creation form   | Insert proper and valid dummy data into the registration form. Check if the user was created correctly.                                          |
| 2  |                     | Invalid data in user creation form  | Insert invalid dummy data into each field of the registration form. Check if the user is prompted about the mistake and if the user was created. |
| 3  |                     | Post creation behaviour             | Check if the user is redirected to the profile page after successful creation                                                                    |
| 4  | User authentication | Proper data in the user login form  | Insert proper and valid dummy data into the login form. Check if the user was authenticated correctly.                                           |
| 5  |                     | Invalid data in the user login form | Insert invalid dummy data into each field of the login form. Check if the user is prompted about the mistake and if the user was created.        |
| 6  |                     | Check credentials for a new user.   | Check if a new user without admin rights can access the admin panel page.                                                                        |




This is just a brief, high level example to make it easier to create a test plan. Test cases and suites might not be put into a table for instance. The crucial part is - it has to reflect your needs and requirements. Maybe you need to place a column for test data, or just an "additional information" column will do just fine? Be creative in using this example. 




### Test execution schedule

This part of the test management has to consider the software project schedule as well. One thing you might reduce your effort with, is supplementing the table above with a "sprint number" column. That way you end up with a highly readable to do list. If you choose the more descriptive approach, try something like the following.

*The testing will be performed on a similar schedule to the one the application is being designed. After each iteration is released to the relevant environment (at the end of each sprint), the test team will test the designated groups of features. As each of the people involved in the test phase is available part time, test items might require increased managerial attention. As of now there is little possibility to estimate the time required for each case to be conducted. This is mostly because the team is not yet onboarded to the chosen tool.*

Remember that a good test plan foresees changes. Also note there is some flexibility in the schedule for each testing task, since you will deliver those on sprint basis. There is no need to plan the actions in high detail.


->[Automate your tests for free](https://app.bugbug.io/sign-up/){.markdown-cta}<-


### Defect reporting and tracking

As we have already planned the whole testing process, the final touch would be to take care of the “what if..” scenarios. Yes, the final purpose of the test plan is to find actual bugs in your software. The final part of the document should then outline the course of action after a bug is found. Remember, that proper setting up automated processes here is an investment that will easily and quickly pay off. Imagine sending out a hundred emails weekly just to initiate the fixing process.

*Thanks to multiple integration opportunities from the chosen tool we are able to notify the key people from both development and testing teams. To reduce unnecessary email communication, the reports on started and passed tests will be sent only via the designated Slack channel.*

*If a bug occurs, the following actions will be automatically undertaken:*

*- A message will be sent via the designated Slack channel*

*- An email will be sent to the Test manager and to the development teams’ PM*

*- A Jira ticket will be created with the description of the test case and error log*

*- A GIT branch will be created. It will be linked in the Jira ticket mentioned above*

*Such a flow will be efficient for the bug fixing process.*

Do note that not every test management tool allows you to easily integrate such functionalities, so choose wisely. In the test plan template you will be able to download, you can also find a flow chart for defect reporting.




## Sample test plan document

Here is one especially for you. After clicking the link below you can download sample test plan. If you are not the test manager, send it to someone responsible for providing the test documentation.  It contains the cases described above but put into one easy to read document. 


[Test Plan Example](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Bug_Bug_TEST_PLAN_example_20b66d9113.pdf)




## Software test plan template 

We are not yet tired of giving away gifts. A sample test plan template will enable you to document the procedures for your app and meet the delivery date. And at the end of the day, what does it do? After reading the last article you should already know. It saves you money! Successful delivery of test items might and will reduce the man hours spent on bug fixing. Our template is universal so it will cover every field of software testing. 

[Test Plan Template](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Bug_Bug_TEST_PLAN_template_3f4dd12d86.docx)




If you feel that this article does not consist enough information on how to create a test plan on your own, you are right. We created a separate article that includes much more knowledge. Check it out [here](https://bugbug.io/blog/software-testing/how-to-create-a-test-plan/).


