Software Testing Knowledge

**It turns out that following the “golden rules” of testing does not always  bring about the desirable effects. Disregarding the rules can be equally disastrous as exemplified by companies and institutions that have become household names. One thing is for certain: software defects are expensive. Fortunately, there are ways to mitigate the damage.**

Imagine that, as a result of an defect in the medical care system,  the patients’ medical records which are the basis for writing out prescriptions, are not updated. As a result, physicians prescribe medications unsuitable for the reported ailments which may pose a threat to some people’s health or even lives. This is not a script of a science fiction film. This situation took place in 2018 in Europe’s second largest economy. What caused it? A bug in the system operating in National Health Service (NHS) in the United Kingdom. Back then, more than 10 thousand people received wrong prescriptions. Fortunately, adverse consequences for the health were not involved. It is difficult to evaluate the losses incurred by this type of defects because, on top of purely economic costs of rectifying these mistakes, the reputation-related consequences may be even more damaging (especially for companies from outside the public sector). **In the United States alone, the damage resulting from launching to the market poor quality  software was estimated in 2018 at $2.8 billion.**

## The results of testing defects

The trade experts are far from disregarding the importance of defects generated in the process of designing software. Following a survey carried out in 2019 by Micro Focus in the US,  it was possible to define how the three areas of software written by means of the Agile/DevOps methodologies: functional, performance- and security-related, affect the specific areas of a company’s operations. It was agreed that defects on all the three levels impact most a company’s revenues, brand reputation and have legal consequences.

![elements-contributing-to-negative-business-impacts.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/elements_contributing_to_negative_business_impacts_9e27ba5281.png)

Allegedly, the situation can be improved by the rapidly growing automatic testing market which (and this comes as a surprise) is also an area which poses considerable challenges to companies, as indicated in the latest “World Quality Report 2019-2020″:

![challenges-currently-faced-in-applying-testing-to-agile-developments.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/challenges_currently_faced_in_applying_testing_to_agile_developments_005725ae28.png)

The most typical challenges indicated in the survey included:

1. Lack of appropriate test environment and data.
2. Inability to apply test automation at appropriate levels.
3. Difficulty in identifying the right areas on which tests should focus.

Interestingly, testing was indicated as the biggest challenge accompanying actions aimed at preventing new defects in software:

![top-challenge-of-mitigating-defects.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/top_challenge_of_mitigating_defects_6c5a6e3a12.png)

Why is testing perceived as such a difficult task? This may stem either from lack of knowledge or inability to apply it. One solution is to audit the activities instigated in the company; the list below may prove useful.

## Poor testing strategies or how extra costs can be generated

**1. Underestimated testing scenarios.**

Testing scenarios should result from consultations held within the testing team working on the scenarios aligned with the previously collected business requirements. At the very early stage of talking with product owners, project managers and developers, detailed product documentation should be compiled. Together with other things, it will provide the basis for creating detailed action scenarios (the testers’ skills and experience play an important role, too). Ironically, a firm plan, implemented since the beginning of work on designing software in accordance with the shift-left testing idea (i.e. starting tests at the earliest stage of development) facilitates flexible adjustments to changes typical of the Agile approach. It is easier for testers to modify an existing plan than to work in chaos aggravated by subsequent sprints. A case in point is a small software company, described in a scientific work entitled  “’Good’ Organisational Reasons for ‘Bad’ Software Testing: An Ethnographic Study of Testing in a Small Software Company” by D. Martin, J. Rooksby, M. Rouncefield and I. Sommerville. For the benefit of the study, a company from the United Kingdom is referred to as W1REsys. It creates development environments which make it possible for customers to design mobile applications. W1REsys employs seven people including four software developers. Due to the lack of a dedicated team of testers, the requirements for new products or functionalities are laid down either with respect to the clients’ needs or ideas for the elements which potential new clients can find attractive. These ideas are provided by representatives of the marketing and sales departments while the scientists surveying the company decided that testing was based on suggestions of what needs to be tested rather than how to do it. Therefore, the automatic tests created in W1REsys are not a part of a structured plan and as such, they allow only selected functionalities to be developed. The ideas for enhancing them are originated also in the process of developers’ work; they result from discussions about the way the system is working. This approach incurs the risk of grave defects in production and, as the survey’s authors admit, these defects do occur  yet they are maintained on a level acceptable by the company.

**2. The testing process is not aligned with the way in which an organization operates.**

The example of W1REsys shows how important it is to adjust the principles of effective testing to an organisation’s capabilities. In larger companies which are in charge of complex projects, following the good practices guarantees success. In smaller companies, oftentimes coping with staff shortages, these rules need to be tailored to the employees’ actual capabilities. Otherwise, they will work under enormous pressure of time and frustration that they cannot accomplish all the goals towards which they have been working. As the Micro Focus research from 2019 suggests, the defects detected at the last stage of designing software (production) tend to be the most costly. This stems from research into both projects managed in Waterfall, now a less popular methodology, and Agile, a methodology enjoying great interest:

![effect-on-cost-by-development-stage-1024x523.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/effect_on_cost_by_development_stage_1024x523_09c6c9266d.png)

[According to Eric Elliot who has written an article on production defects for Medium.com](https://medium.com/javascript-scene/the-outrageous-cost-of-skipping-tdd-code-reviews-57887064c412), the cost of remedying a production defect can be 100 times higher than the cost of correcting it at the design stage, and 15 times higher that the cost of repair at the level of implementation.

Smaller companies, which cannot afford extended testing scenarios, should arrange their tasks from the most to the least important to a project. The mentioned software company W1REsys from the United Kingdom set a goal of testing the functionalities most frequently used by the clients. As a result, it can properly manage defects identified at the production stage. This is a good example of approaching  testing not like a way of excellent testing of the entire code which, in many cases, is simply impossible but rather, as a way of managing risk that can be mitigated when specific testing tasks are given priorities.

**3. Ignoring the importance of tests in the speed and efficiency of  mobile applications.**

As a result of focusing on functionality tests (working/not working), some teams ignore the importance of speed and efficiency tests. A survey conducted by Dimensional Research in 2015 informs about the actual costs of adopting this attitude. Users of mobile applications were asked for their opinions on the response time. As the diagram below shows, their expectations are very high with 2 seconds required for the optimum response time:

![how-fast-should-app-respond.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/how_fast_should_app_respond_0d7c860cce.png)

80% of the respondents admitted that in the case of problems with an application, they give it not more than three chances before they abandon it for the benefit of competitive solutions. Interestingly, in the process of choosing an application, the functionalities promised by the producers are as important as the reviews published by other users:

![which-of-the-following-most-influences-the-app-you-pick.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/which_of_the_following_most_influences_the_app_you_pick_76401d2bb5.png)

In 2017, Pinterest found out about the business benefits of rebuilding an application to make a faster and more responsive one; following its implementation, Pinterest enjoyed bigger revenues and traffic:

![comparing-old-mobile-web-to-new-mobile-web-1024x528.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/comparing_old_mobile_web_to_new_mobile_web_1024x528_6332cda00b.png)

**4. Giving up on Test Driven Development.**

This approach assumes designing automatic testing before works on writing the right code even commence. Of course, at the beginning the tests are failed; over time, as the subsequent lines of the code are written, at a certain stage they all end positive. As a result, the “trouble-free” section of the code grows; you can also think over and plan how the entire application works. Eric Elliot presented the following calculations as evidence of the  effectiveness of the method which daunts many IT managers due to the high costs involved at the initial stage of a project:

![source-medium.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/source_medium_e8b7b31c22.png)

In the fictitious example provided by Elliot, development without TDD was to cost 1,000 hours (the cost of maintaining the system excluded). TDD allowed to save 623 hours which is equivalent to a month of work of a team of four.

**5. Under- or overestimating automatic testing.**

The success of any testing team is based both on qualified employees and skilfully selected tools. Automatic testing significantly speeds up and fosters work thus releasing the testers who can focus on exploratory testing or a product analysis with respect to the users’ requirements or experience. Teams distrustful of automation tools or disappointed with older generation programmes may prefer manual tests. Frequently, this results in overloading the employees with work. They are also frustrated because they need to handle repetitive, mundane tasks. On the other hand, automatic testing cannot replace an employee who may come up with the right testing scenarios aligned with the business guidelines or even supplement them with suggestions of new and interesting functionalities. An optimum testing strategy helps to design the testers’ responsibilities to ensure that they can make the most of their automatic testing tools, at the same time taking advantage of the specific workers’ competence in carrying out manual exploratory testing.





large_zle-strategie-testowania.png

medium_zle-strategie-testowania.png

small_zle-strategie-testowania.png

thumbnail_zle-strategie-testowania.png

zle-strategie-testowania.png

Bad testing strategies or how to learn from other people’s mistakes and to minimise damage

bad-testing-strategies-or-how-to-learn-from-other-people-s-mistakes-and-to-minimise-damage

The work of a manual tester is not always as appreciated as it should be. Often, words like “testing” boil it down to (seemingly) boring activities following a scenario. This is a superficial opinion; check the 10 reasons why this profession can be recognised as surprisingly interesting:

**1. The job of a manual tester=creativity that many marketing guys could dream of.**

As a manual tester you get to know the product and create testing scenarios. This is the part of the work where you can unleash your creativity: you come up with the biggest possible number of ways in which the user can make use of your product and where an error can be detected. Even if you are a natural when it comes to looking for “weak points” in an application, you go beyond it and foresee the user’s behaviour and the operation of the entire software. Looking at the scenarios you have created, many people could say “there is no way this will happen!”. How about you? When your scenario shows an unexpected error, you simply flick an invisible speck of dust off your shoulder.

**2. Manual testing is heaven for an analytical mind.**

Analytical thinking and drawing conclusions will come in handy at every stage of your work: since the moment you get acquainted with the technical specification and the product’s application up to writing the mentioned scenarios. A good tester has the skill of inductive thinking i.e. drawing conclusions on the basis of pieces of information coupled with deductive thinking (from the general to the detailed). A tester’s role is not limited to checking how specific functionalities work; a tester also places them in a broader context – the product and its trade environment. Questions like “Is this the best solution? Does the design meet the standard? Can you simplify it?” are elements of daily work. A manual tester is not a person who follows the instructions blindly but rather, he/she creates them respecting the latest standards and sometimes even challenging them.

**3. A manual tester criticises as much as he/she likes.**

Did you dream as a child of being a literary, theatrical or film critic? Even if you did not, as a manual tester you have an opportunity to show the part of your personality that enjoys pinpointing mistakes and highlighting imperfections. There is more: in this job it will be more than welcome (with reference to the tested application, not your team mates, that is )!

**4. A manual tester stands for a “great communicator”.**

Did you know why Ronald Reagan was dubbed “the great communicator”? For his rhetoric skills in general and one talent in particular. It actually helped him to be elected president. In 1980, during a TV debate with Jimmy Carter, Reagan addressed all Americans directly and during his air time he looked straight into the camera, making an impression that he was speaking to each and every viewer individually. How does this connect with a tester’s responsibilities? As part of your work, you will also have many opportunities to look someone in the eye and say something  harsh, e.g. to show a code error to a developer or to ask a project manager for extra time for testing if the situation requires it. Rest assured that these will not always be nice conversations… It is good to know because many people perceive a tester as an individual who is knowledgeable about logs and only produces reports. However, this job requires a lot of interaction with people at various levels (especially when your company uses the scrum methodology) and here communication skills are much coveted. Assertiveness (ability to present your opinion without hurting other people’s feelings) is worth its weight in gold.

**5. Testing and… empathising.**

I do not mean only empathy with a developer who, following your remarks, needs to repair the bugs ;). As a tester, you empathise mainly with the user who will ultimately make use of the product you are testing. You try to best understand the user’s needs and train of thoughts. Your aim is to make the application as easy to handle as possible. This is yet another soft skill that you can use in your work.

**6. People keen on accuracy make good testers.**

If you are applying for a job of a manual tester, perfectionism will certainly be perceived as your strength during the job interview. Accuracy and care for the detail are prerequisites for being a tester; otherwise you might miss an important aspect of an application’s functioning. As a result of this omission, the company could lose its good reputation and considerable amounts of money. 

**7. As a manual tester you use your organisational skills.**

You do not have unlimited time for testing; in many cases, there is less time than you hope. Your planning skills and effective use of time come in handy. Time pressure is visible not only in Jira; it also affects the people managing a project.

**8. A manual tester’s continued growth.**

As a tester you need to keep up with the changing trends and technologies, also as part of your specialisation. There is no room for stoppages or boredom because every day, new information is disseminated, new courses and ideas originated. When implemented, sometimes they challenge the assumptions about reality. Every tester needs to know the product he/she handles and this is no mean feat: you need to know the rules of operation and find out the solutions offered by the competitors plus analyse in depth the technical documentation, trace the logs etc. A tester’s work requires regular updates.

**9. As a tester you are responsible for the product’s quality.**

By clicking on “Test passed” in Jira, you confirm that a specific functionality and, by extension, the entire product, meets the requirements. You are some sort of a quality controller and this is more than just checking if an application works as planned. As a result of your work, a product wins market trust while its effectiveness improves the corporate owner’s reliability.

**10. The position of a manual tester is the first step; there is always more**

Later on, you can branch out into automatic testing, QA or even programming. The more testing tools, technologies and programming languages you know the better. Consequently,  you strengthen your position on the job market. Your opportunities for growing, learning and enhancing your skills  are limitless.

large_dlaczego-praca-testera-moze-byc-ciekawa.png

medium_dlaczego-praca-testera-moze-byc-ciekawa.png

small_dlaczego-praca-testera-moze-byc-ciekawa.png

thumbnail_dlaczego-praca-testera-moze-byc-ciekawa.png

dlaczego-praca-testera-moze-byc-ciekawa.png

A manual tester: 10 reasons that make this job interesting

a-manual-tester-10-reasons-that-make-this-job-interesting

Analysts agree that in 2021 and the subsequent years, the software testing services market will flourish. How will that affect the entire IT industry? Which business areas will be most impacted by this growth? Will the effects be always positive?

The optimistic forecasts for the software testing services market are coming true as we speak. According to Marketsandmarkets research company which has produced its forecast on the basis of indicators from 2016, in 2017-2022 the value of this specific market will grow to $50.14 billion with the compound annual growth rate (CAGR) amounting to 13.5%.

In 2020, Technavio published its report on “Software testing services market by product, geography and end-user. Forecast and analysis 2020–2024”. According to the report, in the four years the market value of the software testing services market will grow by $ 81.73 million with the CAGR amounting to 12.62%:

![bb-8-1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_1_1929945956.png)

The two above mentioned research companies also agree that North America will be the region with the biggest growth potential on the software testing services market. The related division of power has been illustrated by Marketsandmarkets in the following manner:

![bb-8-2.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_2_4f2997f99d.png)

## The software testing services market

In an article for Medium.com, Astasia Myers published a diagram which shows the major dividing lines on the software testing services market. It also provides a picture of the growth potential of the market’s specific parts:

![bb-8-3.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_3_d3540021a4.png)

The fastest growing areas of the software testing services market include test automation (17%CAGR) and API testing (19%CAGR). Testing services have the smallest annual growth rate (13%CAGR) but enjoys the biggest value ($50 million).

## The possible business outcome of the processes

In order to foresee the outcome, the growing shares of the specific parts of the market need to be viewed from the perspective of the trends in software production. Here, the ever so popular DevOps and Agile methodologies play a significant role. The latter methodology was indicated in the latest report “State of the Agile 2019” as the most frequently applied in unit testing. Both methodologies enforce a quick rate of code development, at the same time placing strong emphasis on its quality and reliability. In order to ensure the best possible processes of monitoring quality and detecting errors in the resulting solutions, more and more often managers decide to automatize the work of testers and QA. This has been confirmed by the results of a survey carried out by Capgemini and published in “World Quality Report 2019-2020”:

![bb-8-4.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_4_80ce264d0a.png)

The report’s authors draw attention to the fact that the need to increase work automation will involve increasingly advanced technologies for creating the right tools. Here, solutions based on AI and machine learning will have an important role to play.

Which solutions are the most popular among managers who want to streamline and accelerate tests in the DevOps and Agile environment? In a questionnaire developed by Capgemini, tools for automated tests were indicated most frequently. They were followed by the BDD (behaviour-driven development) which should not come as a surprise in the light of its benefits in solving complicated business problems and making decisions about the types of tests which are best for a specific situation:

![bb-8-5.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_5_07c6a7ea65.png)

The above chart shows also a third trend which has lately gained in popularity as the Agile – CI/CD methodology is developing. In the testing context it is continuous testing i.e. continued exchange of feedback between teams of developers and testers. It is possible on condition that a majority of the software testing processes are automatized. Only then the code quality can be effectively and sufficiently quickly checked.

These benefits have been appreciated in the IT industry for several years as presented below in a cross-cutting summary of the biggest benefits of automatizing the process of software testing. Every year, their value grows. Interestingly, in 2019 managers indicated better control and transparency of the supervised processes as the most appreciated value added resulting from automation:

![bb-8-6.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_6_8c99e341b6.png)

## The financial outcomes of the changes in the software testing services market

The development of automation tools has also decreased the costs of testing as evidenced by the summary below:

![bb-8-7.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_8_7_cbccf44ab1.png)

How was it possible to substantially decrease testing-related costs? The report provides information about employing:

- solutions for cloud testing which lower the costs, at the same time increasing the effectiveness and flexibility of the teams involved
- virtualization,
- Agile and DevOps methodologies,
- automation,
- shift-right testing i.e. testing is performed shortly before a solution goes live. Of course this does not exclude agile methodologies or code review in sprints, preferred by Agile.



large_bb-8-0.png

medium_bb-8-0.png

small_bb-8-0.png

thumbnail_bb-8-0.png

bb-8-0.png

The software testing services market and the business outcomes of its growth in 2021

the-software-testing-services-market-and-the-business-outcomes-of-its-growth-in-2021

When you introduce automated tests in your organisation you have to keep in mind that it is not only a matter of choosing the best tool. Such a move also entails many changes in the organizational scheme of the team. There are also many other factors that have to be considered before you start introducing automated tests. How to prepare for it so that your “mission” would not be a failure?

According to the World Quality Report 2018-2019 prepared by Capgemini, end-user satisfaction became an important priority for QA and strategic software testing teams. As we can read in the summary of the report, “Low levels of automation and challenges with test data and test environments are holding back QA and testing efficiencies”. At a time when more and more software is developed and there is a permanent lack of people able to test it, appropriately selected and implemented tool for test automatization is worth its weight in gold. The tool itself is not a guarantee of success understood as an increased effectiveness of the work and test coverage if you don’t take into account the following issues:

## Testing strategy in the project

„Automated tests” is a keyword that describes the whole group of solutions on different levels of advancement. On this market there are available simple extensions for recording tests in the browser, created 10 years ago, as well as state-of-the-art tools based on artificial intelligence. Before you make a decision about using free software you may want to ask yourself few questions.

- What kind of bugs do you want to detect the most?
- What kind of tests do you want to automate (mobile application or the website? GUI or compatibility issues? Security or operating fluency?
- What parts of the project in which you would like to implement automated tests are the most prone to bugs?
- Which bugs in the software that you test would be the most expensive taking into account the end-user satisfaction? (or its lack) Which ones would be the most expensive for the sake of the amount of time needed to fix them?
- What are the security requirements for using such a tool in your organisation?
- How the project management methodology (Agile, waterfall) implemented in your company can determine the scheme of testing?

Answers to these questions could help you select the criteria for the automated tests programme that will suit your projects best.

## Who do you want to convince to invest in the automated tests?

Regardless of your choice of the tool, bear in mind that you will not use it alone. Although convincing testers’ team to it would be easy, the managerial team can require hard evidence that this would be a good investment. It seems obvious that before meeting with them it is good to prepare data regarding estimated increase of the testing team effectiveness and enhanced bug detection. The second (and often overseen) factor is the workflow in the project – not only in testers’ team, but also between developers and business unit. If you show how the tool you selected will improve the work (i.e. thanks to the reports that business will surely appreciate, or map of the bugs which will enable for faster submission of amendments to developers) it would be perceived as a big asset.

## Choosing the best tool for your team

List of requirements created by the testers’ team would be a good start; you can also consult it with developers. Apart from precise comparison of the offers from different vendors you can also take into account the following advantages:

- Was this particular tool for automated tests designed taking into account its future users? Is it intuitive?
- Can it – despite its advanced features – be launched from the browser?
- Does it enable you to edit tests and add new parts to them without recording the whole scenarios from the beginning?
- Does it have a built-in modulus enabling for minimisation of so-called flaky tests?
- Does it provide the map of the bugs, thanks to which you can identify the most prone to bugs parts of the code?

Real added value to your research would be recommendations from your colleagues as well as getting acquainted with the demo versions of the tools that you take into account. Read article about features every test automation tool should have.

![bb-7-1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_7_1_ddd248eaf2.png)

## Who would be using this tool? Is he competent to do it?

Members of the testers’ team differ in terms of competences and predispositions . Some of them are beginner manual testers, others – more experienced – prefer exploratory tests. Some of the automated testing tools available on the market require programming skills on different levels. Not every tester can boast of them. And even if he can, he may not have enough time to programme automated tests. Fortunately, modern tools for automatization available on the market don’t require such skills from the tester – it is good to take a look at these offers too.

## Automated tests – do they pay off?
This question concerns not only the subscription fee for the tool. You should also take into account the cost of time that your employees would have to devote to learn how to use this programme. On the bright side, in your calculations you can also include the benefits such as work acceleration and increased effectiveness of your team.

Introduction of the new tool for automated tests in your company would require a lot of time and effort (i.e. devoted to comparing the offers). Although this task (on condition that it would be done properly) can translate into real improvement of testers’ work and comfort, as well as have positive impact on the overall results of the project. It is worth trying.

large_bb-7-0.png

medium_bb-7-0.png

small_bb-7-0.png

thumbnail_bb-7-0.png

bb-7-0.png

Want to introduce automated tests in your organization? Here are 5 challenges that you have to face

want-to-introduce-automated-tests-in-your-organization-here-are-5-challenges-that-you-have-to-face

Product

Selenium IDE is a tool for recording and launching application tests. It was created by Shinya Kasatani in 2006 (sic!) and is based on „record & playback” idea, which nowadays is no longer a novelty. The problem with most of the tools available on the market (including Selenium IDE) is not the outdated idea, which basics still can be perceived as elegant, but its realisation. Below we listed the biggest accusations that can be directed towards Selenium IDE and solutions to them offered by the tool named BugBug.

## Generation of improper elements’ selectors

While recording the application tests, the newest version of Selenium IDE is able to generate several selectors for the element with which the interaction was made. The problem is that this tool doesn’t give you the possibility to configure which type (or types) of selectors should be set as default ones. It also prefers wrong approach to the generation of selectors, based on the names of the CSS classes. Nowadays, when CSS-in-JS approach is becoming more and more popular (i.a. due to [styled-components](https://styled-components.com/), which generate random class names), it makes the user responsible for modification of all selectors generated during the recording session. The result is that the work that should be automatically done by the tool is done manually and at a much slower pace by the user.

## Solution to the problem provided by the BugBug

During a recording session, BugBug – just as Selenium IDE – generates many selectors for the elements with which interaction was made. The difference is that in this case the user decides about the hierarchy of the generated selectors. It can be easily modified in the configuration of the project:

![bb-blog-6-1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_1_373223ad89.png)

Thanks to this solution you have complete freedom and control over the generated selectors – you just act accordingly to the project requirements. Nowadays, testing in the projects very often is based on the data-test or data-testid attributes, that are also preferred by the BugBug. However, if you have to deal with the older type of the project, you can always prioritize generation of the selectors based on the CSS classes or elements ID. The key here is that the user independently decides what he needs and changes configuration of the project accordingly.

While recording the application tests we are not always sure if the selectors are correct. That is why the BugBug at the first launch and after each change in the test verifies selectors and corrects them if there is such a need – for example if the selector disappeared for some reason (it could be caused by the random CSS classes) or it points to the improper element. Thanks to implementation of these mechanisms interference of the tester with the process of the selectors’ selection is reduced as much as possible. He also saves his precious time, so far wasted for manual searching and creation of the selectors.

## Lack of active waiting for the elements (waiting conditions)

Another common problem with the record & playback tools, which concerns also Selenium IDE, is lack of active waiting for the elements with which interaction should be made. In practice it works like that: if the chosen element is embedded on the website (in DOM), Selenium IDE attempts to interact with it, even though it can be invisible, moves, animates or is covered by another element (loading overlay). The result of such scheme of actions are so-called “flaky tests”. This notion defines the tests which once work, and another time they don’t. The reason for such behaviour is very often lack of appropriate waiting conditions for the element with which the interaction is to be made.

## Solution to the problem provided by the BugBug

Creators of the BugBug took into account defects of the tools available on the market and effectively addressed them. On the level of the project configuration you can define waiting conditions that have to be met so that the interaction with the element could proceed:

![bb-blog-6-2.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_2_1eed216be1.png)

The global configuration of the project is inherited within all steps of the test. Of course there is also possibility to overwrite this configuration on the level of the single step. You can also add additional conditions there, such as waiting for the page reload:

![bb-blog-6-3.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_3_ed8486fb8b.png)

Thanks to this solution BugBug emulates the human behaviour, what means that it waits for an appropriate moment to proceed with the particular step of the test. The tool can actively wait when:

- element animates or changes its characteristics (CSS animations and transitions): the tool identifies that the element animates/moves and waits till the end of this process.
- element is invisible: BugBug’s mechanism is able to assess whether the element is in fact visible for the user. If so, it proceeds with the planned steps.
- element is covered by another element.
- element is inactive.
- element changed its position: the tool is able to identify change of the element’s position while it is proceeding with the step. In such a scenario the interaction is repeated.
 competitors.

Implementation of all the mechanisms listed above reduces the risk of „flaky tests” to minimum. Thanks to that BugBug offers much more stable tests than its competitors.

## Misleading process of replaying the steps (with the use of JavaScript)

Selenium IDE as well as most similar tools available on the market proceeds with the steps of the test that only partially emulate behaviour of the average user. In practice it means that tests recorded this way can behave very unpredictably, leading to serious errors. It would be easier to explain with the use of the specific example:

The step click on any „Log in” button realised within Selenium IDE would trigger the following event in the JavaScript browser:

- click

## Solution to the problem provided by the BugBug

The same step realised in BugBug results in the sequence of the following events:

- mousemove
- mouseover
- mouseenter
- mousedown
- mouseup
- click

The difference described above stems from the constraints of the Selenium IDE architecture, which can emulate the user’s behaviour only with the use of JavaScript. Architecture of the BugBug and implementation of the [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/) gives it full control over the browser. Thanks to it the behaviour of the user can be accurately emulated. It is crucial in testing contemporary, very extensive and complex applications (Single Page Applications).

## Application tests are not launched separately

Majority of the record & playback tools has a problem with performing the tests separately. It results in a situation, when launch of the particular test impacts another one. The problem resides in the fact that cookies or localStorage mechanism in the browser are shared between windows and tabs. When you use Selenium IDE, you have to always delete cookies (or localStorage) and remember about the necessity to clear this data between the tests.

## Solution to the problem provided by the BugBug

Despite the fact that the problem described above is simply burdensome, it also causes flaky tests. To prevent this, application tests in BugBug are by default performed separately with the use of Incognito mode in the browser. Thanks to that QA who builds the tests doesn’t have to remember about cookies, localStorage and other data sharing mechanisms in the browser. Tests are isolated regardless of whether you launch them in the browser or in the cloud.

![bb-blog-6-4.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_4_136c914e62.png)

## Launching tests in the cloud is impossible

Selenium IDE was created many years ago, when cloud and SaaS services weren’t so widespread as they are nowadays. As a result, Selenium IDE functions only as the browser ad-on and doesn’t offer testing in the cloud. Tests created with the use of Selenium IDE can be exported to the chosen programming language and then launched within your own infrastructure (or with the use of some commercial solution).

Solution to the problem provided by the BugBug

BugBug from the very beginning was designed as a tool that aims at supporting the whole testing process (end-to-end). It enables for:

- [recording and easy editing of the tests.](https://docs.bugbug.io/core-features/recording-the-tests)
- launching of the tests locally, within user’s browser, in order to verify the tests.
- planning and launching application tests in the cloud (through www interface as well as CLI tool).

Look in our previous post to see [10 features every test automation tool should have](https://new.bugbug.io/blog/10-features-every-test-automation-tool-should-have/).

Thanks to that you receive one, integrated tool that can be used for creation and launching of the tests as well.

![bb-blog-6-5.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_5_94ab9b819c.png)

## Lack of support for the Page Object Model (POM) pattern

Page Object Model (POM) is a verified design pattern for end-to-end tests, which enables you to freely re-use steps within the tests, thus simplifying its maintenance. This pattern bases on creating group of steps that represent pages of tested Internet application as so-called Page Object. Thanks to that after you create “Log in” Page Object you can use it multiple times – in each test that requires logging in. If the log in form changes, it suffices that you fix the test in one place – as a consequence all tests based on “Log in” Page Object would be changed accordingly.

Unfortunately, Selenium IDE doesn’t support creation of the tests based on the Page Object Model. Steps in each test are strictly assigned to it. When you have many tests scenarios that use the same steps, if one step stops working, you have to fix it separately in each test.

## Solution to the problem provided by the BugBug

BugBug was designed to support Page Object Model pattern from the very beginning. This tool enables you to freely group the steps in tests and use them as re-usable elements. What is more, a natural consequence of such an approach is the possibility of creating visualisation of the tests in one, coherent graph. Such presentation of all tests gives you clear information about which business paths within the project are covered with end-to-end tests.

![bb-blog-6-6.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_6_bf3d39ac25.png)

## Summary

![bb-blog-6-7-768x480.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_6_7_768x480_7b04bf2e6c.png)


large_bb-blog-6.png

medium_bb-blog-6.png

small_bb-blog-6.png

thumbnail_bb-blog-6.png

bb-blog-6.png

Application tests, BugBug vs Selenium IDE

application-tests-bug-bug-vs-selenium-ide

Automated testing became an indispensable part of the software development business. It is also given specific business objectives to attain. Which trends will dominate automated testing in 2021 and determine who will get the biggest market share?

The rapid development of the Agile and DevOps methodology integrated automated testing into Agile (which also means accelerated) process of the code generation. One of the direct consequences of such a change was so called shift-left testing, which means moving the testing to the earlier phase of software development – it can be even gathering of the business requirements. Before “tha Agile era” testing could start only when the final effects of the developers’ work were available. It resulted in the higher costs of the maintenance and bug-fixing (because some of them were overseen in the earlier phases of the work). Not to mention huge losses of the company’s image and outflow of the customers disappointed with the product. It therefore comes as no surprise that the executive IT management surveyed in “The World Quality Report 2019-2020” recognised “contribution to business growth and business outcomes” as the most important objective for testing and QA in 2019:

Chart 1. Answers of the executive IT management concerning objectives for testing and QA in 2019:

![bb-blog-5-1.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_5_1_1081b97858.png)
Source: The World Quality Report 2019-2020

However, testing in 2013 (when Agile started to gain in popularity) and now are two different worlds. What will contribute to increasing business results after Agile became an indispensable methodology of work in the software development business?

## Automatization of the new generation

In recent years, the software testing market has been experiencing a real boom and – according to the latest analysis – it is only the beginning. As stated in the Techavio report “Software Testing Services Market by Product, Geography, and End-user – Forecast and Analysis 2020-2024”, it is expected to grow at an even faster pace – with CAGR at the level of 12,62%. Read more about business consequences of this phenomenon. The necessary element of this process is development of the automated tests, which supports changes taking place thanks to the implementation of the Agile and DevOps methodology. Currently, the tools which offer only simple functions such as recording the tests performed on the website or in the app, are insufficient. To keep up with the rapid changes in the development environment there is a need of the software which will enable for automatization of the most actions taken by the testers. Additionally, such a tool should be easy to handle for the person without any programming skills and offer the feature of quick identification of the most bug-prone parts of the code. Check out 10 characteristics that the new generation testing automation tool should offer. Automated testing ceased to be only the tool which supports testers in their work – it became an indispensable element in the end-to-end testing process.

## Artificial intelligence (AI) and machine learning (ML)

Although they became very popular and catchy slogans, the newest Techavio report “Software Testing Services Market by Product, Geography, and End-user – Forecast and Analysis 2020-2024” confirms that the testing sector has just started making use the benefits which stem from this technologies. The results are encouraging, and the forecasts concerning the pace of development of the testing market (according to the TechWiz its value in 2025 could reach 190,6 billion USD) confirm its great potential. As we read in the Techavio report, implementation of the artificial intelligence in the testing process is crucial in striving for the smart testing, which means ensuring maximum effectiveness of the testing from the earliest stages of the code creation. What is important, QA specialists and testers who will be introducing the artificial intelligence in the company should receive appropriate training – it is necessary to fully benefit from the possibilities provided by this technology. Companies which will be first to discover such a need would gain a huge competitive advantage. The experts quoted in “The World Quality Report 2019-2020” indicate the complementary role of machine learning in the process of enhancing quality of the testing with the utilisation of the AI. How could it happen? As stated in the report, “As organizations grow accustomed to mechine learning, they will increasingly understand its distinction from artificial intelligence. They will see that in order for AI testing to be effective, it needs access to the right data – and machine learning can provide this, by processing records in volume, thereby helping to identify and optimize test cases”.

## Big Data technology

It is another shining example how testing is intrinsically linked with the latest trends in the world of the software and new technologies, as well as with the e-commerce market. Big data is used in the development of the programmes for an automated testing (i.e. with the use of the ML described above), although the data sets also require testing and continuous monitoring. Big online shops or the software for health systems are great example here – such a huge data conglomerates require testing on each phase: introduction, processing, migration, performance and speed. Along with the progress of digitisation process, which concerns each line of business, the demand for such services will be growing.

## New challenge – internet of things

“The World Quality Report 2019-2020” mentions quick development of the testing in IoT in the following industries: car, energy and telecommunication. The latter would face a great challenge in the coming years: consequences of the 5G technology implementation. As piszą Darren Coupland (Sogeti) and Mandan Sundaraju (Capgemini) write, “This will create a new quality assurance (QA) dynamics. All these new use cases will need to be validated end to end. How so? Because the increase of internet of things (IoT) developments implicit in 5G has implications not just for IT applications, but also for the physical devices themselves. For example, it’s not just the app managing the traffic light that needs to be tested: it’s also the traffic light sensor itself”. Test coverage of the smart devices interconnected with the applications became a necessity in the times of the rapid expansion of the IoT market. As the Testing Whiz reports, in 2016 its worth was estimated for 6,4 billion USD, in 2020 – for over 20 billion USD.

## Blockchain technology as a new business model

Blockchain solutions are being used increasingly with regard to the processing and protection of the data. The report “5 Blockchain Trends for 2020”, prepared by Deloitte, reflects its growing popularity and scope of use:

Illustration 1. Conclusions for the coming year with regard to use of the blockchain technology in the business.

![bb-blog-5-2.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_5_2_09dd6a8b48.png)

In the coming 12 months implementation of the blockchain technology is planned to be increased in the following sectors:

- technology, media and telecommunication (from 45 to 49 pct.)
- energy (from 37 to 43 pct.)
- industry (from 40 to 43 pct.)
- life sciences and health care (from 36 to 41 pct.)
- finance (from 33 to 38 pct.)
- professional services (from 24 to 33 pct.)
- industrial and construction services (from 32 to 33 pct.).
- 
Testing plays crucial role in the blockchain-based products, because it monitors its security level and effectiveness of data-gathering. In these fields there is no room for error.

## The increased significance of the security testing

The statistics presented in “The World Quality Report 2019-2020” confirm this trend. The survey respondents indicated that matters of security and risks are the most important for the IT strategy adapted by the company. The security testing scale in the cloud computing environment increased from 42 to 57 pct., what reflects development of the securities and the way such tests are performed, as well as adaptation of this line of automated testing to the fast pace of work in Agile methodology, because testing in the cloud enables for quicker implementation of the changes and reaction to them. In order to ensure the maximum coverage of security testing, the authors of “The World Quality Report 2019-2020” recommend introducing it on the earlier phases of the software development and carry it out with the tools for automated testing. What is even more interesting, the level of automation in security testing is surprisingly low – it discharges only 13 pct. from this kind of tests. However, if you look at the weighting attached to the particular aspects of security in the process of the software development, the increased significance of security testing seems inevitable:

Chart 3. Ranking the importance of certain aspects of secure development strategies.

![bb-blog-5-3.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bb_blog_5_3_36f89419ef.png)

## Performance engineering as an essential mind-set and way of working

The time when automated testing and software development consisted only of delivering the specific solutions, and when each team had been working on its task separately from the other, can be described as the programming stone age. Nowadays the measure of success is creation of the product that all teams had been working on together. All people involved in the project (business line, project managers, developers, QA, testers) from the very beginning get acquainted with the business requirements and the needs of the end-user. They ask questions and talk about possible solutions. Thanks to such a workflow it is possible to implement many changes at an early stage, which prevents from the irreversible (or highly costly) damages that can occur after the deployment. In this way the performance engineering perfectly dovetails into the Agile objectives, because it postulates to look at the product from the meta-level and requires all teams to closely cooperate with each other. The effects are astonishing: the products that were created in this approach have a great market value in terms of the software, hardware, security, UX, effectiveness and performance.

large_bb-blog-5.png

medium_bb-blog-5.png

small_bb-blog-5.png

thumbnail_bb-blog-5.png

bb-blog-5.png

7 trends in automated testing that will dominate in 2021

7-trends-in-automated-testing-that-will-dominate-in-2021

Blog about test automation

Features

Pricing

Roadmap

Case studies

Privacy policy

Terms of service

About us

Book a demo

Contact

Get more tips and product related content. Zero spam.

Don't miss any updates

End-to-end testing made simple! BugBug is a new, more intuitive tool for test automation of web apps. Get all the benefits of test automation without hiring new people or infrastructure changes.

BugBug.io | Intuitive and reliable test automation for startups

large_Open Graph 1200x630 3.png

medium_Open Graph 1200x630 3.png

small_Open Graph 1200x630 3.png

thumbnail_Open Graph 1200x630 3.png

Open Graph 1200x630 3.png

Updates

Guides to automation

small_pawel-about-512.png

thumbnail_pawel-about-512.png

pawel-about-512.png

Paweł Bylina

Are you a tech leader, CEO or CTO in a tech startup? You want to implement automation testing for your SaaS product? We've written this guide especially for you. It's packed with actionable tips on how to approach test automation in an agile way.

We're going to tackle the following questions:

-   What is the business goal of automation testing?

-   How to implement automation incrementally?

-   What resources do you need to develop automated tests?

-   What are typical problems and obstacles and how to overcome them?


The strategic approach to automation testing
--------------------------------------------

Automation testing is a very broad term. There is no single goal of automation. Depending on the level of automation you will get different benefits for your organization.

*Figure 1: Strategic matrix of different goals of test automation*
![bugbug-goals-of-automation-testing.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_goals_of_automation_testing_9308b9cb1b.png)

See this strategic matrix above. You can achieve all of these goals, but not all at once. You should pick the right order of goals, starting with the ones that are low-effort and high-value. 

To achieve each of these goals, you need a different combination of resources and skills. We'll guide you through each of the goals, explaining the approach, time & resources and how to achieve it with BugBug.

Step 1: low cost & high impact goals
------------------------------------

### **Goal:**

-   Prevent loss of business due to production downtimes

### **Approach**

-   You don't need to hire new people, you can do it yourself

-   Create 5-10 automated tests and run them every hour to monitor your production, for things like login, landing pages, registration form, contact form, basic app navigation

### **Time & Resources**

-   2-4 hours of your own time

### **BugBug features that you should use**

-   [Recording tests](https://docs.bugbug.io/creating-tests/recording-the-tests)

-   [Cloud runs](https://docs.bugbug.io/running-tests/in-cloud-on-server)

-   [Schedules](https://docs.bugbug.io/running-tests/schedules)

-   [Notifications](https://docs.bugbug.io/collaboration/notifications)

-   [Re-recording test steps](https://docs.bugbug.io/recording-tests-steps/re-recording-steps)

Step 2: low hanging fruit
-------------------------

### **Goal:**

-   Improve team collaboration on quality assurance

### **Approach**

-   Give all the scrum teams access to the testing platform

-   Share direct links to failed tests in your tickets or on a chat

-   Configure who gets notifications

### **Time & Resources**

-   Inviting people only takes 5 - 10 minutes, but you should spend 2-4 hours on communicating its purpose

### **BugBug features that you should use**

-   [Invite users](https://docs.bugbug.io/collaboration/inviting-team-members)

-   [Project settings > Notifications](https://docs.bugbug.io/collaboration/notifications#choose-when-people-get-email-notifications)

Step 3: incremental development of automated tests
--------------------------------------------------

### **Goals**

-   Save time on manual regression testing

-   Release faster

-   Prevent cost of regression bugs

### **Approach**

-   Talk to the whole team about what you want to achieve and what's the incremental plan

-   Choose 1-2 people responsible for automated tests 

-   Create dedicated test user accounts that will be used for automation only

-   Run automation on some testing environment (staging)

-   Create separate test suites for staging and prod (run all tests on staging, but only essential smoke tests on prod)

-   Add more tests incrementally, 2-3 test cases per week (after 3 months you should have about 50 test cases automated)

-   Start adding [data-test-id attributes](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors) in your HTML

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [New test from here](https://docs.bugbug.io/creating-tests/new-test-from-here)

-   [Duplicating tests](https://docs.bugbug.io/creating-tests/duplicating-tests)

-   [Components](https://docs.bugbug.io/editing-tests/wip-make-component)

-   [Variables](https://docs.bugbug.io/editing-tests/variables)

-   [Profiles](https://docs.bugbug.io/editing-tests/variables#profiles) to [test on different environments](https://docs.bugbug.io/editing-tests/variables#work-with-different-development-environments)

-   [Suites](https://docs.bugbug.io/organizing-tests/suites)

-   [Manually editing steps](https://docs.bugbug.io/editing-tests/manually-creating-the-test)

-   [Testing registration](https://docs.bugbug.io/editing-tests/variables#test-user-registration-and-login-using-variables)

-   [Custom javascript](https://docs.bugbug.io/editing-tests/custom-javascript-actions) might be needed

Step 4: integrate automation with CI/CD
---------------------------------------

### **Goal:**

-   Improve software development lifecycle, being sure that no code is merged without tests

### **Approach**

-   Integrate automation with your development pipelines

-   Trigger automated tests with every pull request

-   Show test results directly in your repo tool (Bitbucket, GitHub, GitLab, etc.)

-   Ask developers to run automated tests locally on the localhost environment before they commit code

### **Time & Resources**

-   You will see the benefits after 4-8 weeks of ongoing work

-   You need to engage 1-2 people on the team. Make sure they have at least 2 hours per week to work on automation.

-   You need at least one technical person on the team who knows how selectors work.

### **BugBug features that you should use**

-   [CLI](https://docs.bugbug.io/integrations/api)

-   [Overriding variables](https://docs.bugbug.io/integrations/api)

-   [Running on localhost](https://docs.bugbug.io/running-tests/test-in-your-browser)

Step 5: increase coverage
-------------------------

### **Goal:**

-   Get 100% test coverage, be confident that all the features, all the browsers, all the devices work correctly

### **Approach**

-   Pick your battles, choose what is the most essential to automate

-   We recommend you keep adding 1-2 test cases every week and don't try to achieve the impossible 100% coverage

### **Time & Resources**

-   To automate everything you would need an Infinite amount of time :)

-   To automate 2-3 new test cases per week you still need 2-3 hours weekly plus a bit more experience with tests and javascript.

-   Note that the more test cases you have, the longer the maintenance. The trick is to find the right balance.

### **BugBug features that you should use**

-   Project settings > Browser width (for [testing mobile screens](https://docs.bugbug.io/workflow-tips/mobile-version-testing))

-   Lots of custom Javascript

-   [Javascript variables](https://docs.bugbug.io/editing-tests/variables#javascript-variables)

-   [Javascript assertions](https://docs.bugbug.io/editing-tests/custom-javascript-actions#advanced-javascript-assertions)

-   [Parallel running](https://docs.bugbug.io/running-tests/parallel-mode)

Common problems and solutions in implementing automation testing:
-----------------------------------------------------------------

Here are some typical problems you may encounter when you want to start automating tests for a SaaS product. We've made a short list with possible solutions that worked for other [BugBug clients](https://bugbug.io/case-study/).

### **Captcha**

You need to [disable the captcha only for automation testing](https://docs.bugbug.io/troubleshooting/captcha-in-automation-testing). Unfortunately, it's not trivial and you need to read through captcha docs or ask a developer for help.

### **Date-based features, calendars** 

Is your app searching train timetables for tomorrow? It's difficult to automate because the date and the expected outcome can change every day.

Solution A: In the automated test, pick a date that's far in the future. Then update the tests regularly, changing this date and the expected outcome.

Solution B: create a dedicated environment for automation with a mocked-up hardcoded date on the backend

### **Dynamic lists such as search results, product listings**

It's hard to automate a test if some data is changing all the time. You need to create independent tests that are always sure what to expect, for example, if you're checking a list of promoted products on your homepage, your test should first go to the admin panel and mark such products as promoted.

You can also have a separate dedicated test automation environment where you have a hardcoded set of data.

### **Can't use the same user email twice**

Create a new user with each test - use the [built-it variable "testRunId"](https://docs.bugbug.io/editing-tests/variables#use-built-in-variables-for-dynamic-or-random-values) in combination with a [temporary email catcher](https://www.google.com/search?q=temporary+email&ei=ARVtY_X8IomKrwS_i4k4&ved=0ahUKEwi1gZ2u86P7AhUJxYsKHb9FAgcQ4dUDCA8&uact=5&oq=temporary+email&gs_lcp=Cgxnd3Mtd2l6LXNlcnAQAzIECAAQQzIFCAAQkQIyBQgAEJECMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDoKCAAQRxDWBBCwA0oECE0YAUoECEEYAEoECEYYAFDhAVjhAWC1CmgCcAF4AIABa4gBa5IBAzAuMZgBAKABAcgBCMABAQ&sclient=gws-wiz-serp) always to run tests with a fresh new user. You can also delete these users at the end of the test by recording test steps in an admin panel (if you have one) Read more in how to test user registration.

### **Selectors are not reliable**

The only way to make sure your tests are stable and easy to update is to start using "data-testid" attributes in your HTML structure, [dedicated attributes for automation testing selectors](https://docs.bugbug.io/preventing-failed-tests/selectors#use-custom-attributes-for-selectors). You need to communicate that to the front-end developer and make sure you remember about it during the code review. No worries, adding additional attributes have no risk, it's just additional data for automation with no other function, it will not introduce new bugs.

### **Multilanguage**

Your page can run in different languages and you want to verify all of them. Try to assess the risk of having a bug only in one language without affecting the others. Perhaps it's not likely that such a bug would occur?

If you still need to test different languages, you have two options:

Option A: Record separate tests for each language. Hard to maintain on a bigger scale

Option B: Use variables & profiles to run the same test on different languages

-   Add data-testid to all of your app elements, so that the selectors are independent of the app language and the displayed test 

-   Add a way in your app to change the language via a URL, for example, "myapp.com/?lang=fr"

-   Create a variable "language code"

-   Create different profiles with different language codes

-   Replace the language code in the "Go to URL" step with the variable

-   Create separate suites for all the languages, overriding their profile with a chosen language

**Tests take long time to finish**

[Upgrade your BugBug plan](https://bugbug.io/pricing/) to be capable of [running cloud tests in parallel](https://docs.bugbug.io/running-tests/parallel-mode)

### **Downloading files**

Currently BugBug can't verify if the files have been download. Soon we'll add a new type of assertion for that, [talk to our support](https://bugbug.io/contact/) if you need it ASAP.

### **Drag & drop interactions**

BugBug support for drag&drop is in beta, you can achieve it by combining the "Mouse press", "Hover" and "Release mouse button" [actions](https://docs.bugbug.io/editing-tests/actions). Talk to our support for details.

### **A recorded selector doesn't work**

Sometimes it's not possible to record a reliable selector and you need to use your human intelligence and debug HTML a bit. 

Use our [No-code XPath Selector Builder](https://bugbug.io/xpath-selector-builder/) which makes it easy to craft a good selector manually and can teach you in-depth about [best practices in selectors](https://bugbug.io/xpath-selector-builder/#How-to-create-reliable-XPath-selectors-for-test-automation).

Success in test automation depends on the regularity of your progress
---------------------------------------------------------------------

Implementing automation testing in a SaaS company is a long-term process. You should do it step by step. The biggest challenge for tech leaders is lack of time to dedicate on QA area. The truth is you don't need a lot to push it forward. Better slow than never. [😉](https://emojipedia.org/winking-face/) Block yourself a slot in calendar every week to work on QA automation and keep the momentum going.

medium_bugbug-matrix-illustration.png

small_bugbug-matrix-illustration.png

thumbnail_bugbug-matrix-illustration.png

bugbug-matrix-illustration.png

Tech leader's guide to automation testing

tech-leaders-guide-to-automation-testing

Test Automation Expert and Test Architect. He is a seasoned tester who can’t stay long in one place. He is looking for perfection and a place to challenge his skills.  He gathered experience working for different companies with different working models, From small to big corporations, From Products via In-house development to software house. Thanks to that he has a wide perspective on testing quality and delivering value.\

[linkedin.com/in/wyrodek/](https://linkedin.com/in/wyrodek/){ref="nofollow"}

large_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

medium_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

small_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

thumbnail_https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

https://i0.wp.com/wyrodek.pl/wp-content/uploads/2017/01/T1A0194.jpg

Maciej Wyrodek

Test automation, especially end-to-end, is currently enjoying a revival. A lot of new tools and approaches have emerged over the last few years. Selenium has finally launched version 4, Cypress and Playwright have brought coded automation to a new level and tools like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim.io](https://www.testim.io/), have revitalized the idea of record and play automation. Since a lot is happening, let's take a look at automation and compare the strengths and weaknesses of 3 approaches to Automation: Custom Made Frameworks, Out of the box framework and Record and play. We will look at the pros and cons of all the solutions and when they work best. However, first we need to discuss the reasons for using test automation.

Why we automate tests
-----------------------------------

If you ask typical testers why they automate tests, they can provide many reasons: from shortening the Regression time, improvement of test coverage, to providing faster feedback. Ultimately, if we look hard enough, all goes back to the one and only goal of companies, as stated by [Eliyahu M. Goldratt in The Goal](https://en.wikipedia.org/wiki/The_Goal_(novel)) - Making money. Testing by itself often is considered a cost - as it does not bring more money. But let's not forget that more money can be made not only by selling more - but by increasing the margin - either by making the product worth the price or by reducing the cost. Testing can help to protect the bottom line, by reducing the cost of mistakes and finding them more easily and faster. This also leads to reducing the cost of customer support, refunds, etc. But as [DevOps](https://itrevolution.com/the-devops-handbook/) and [Modern Testing](https://www.moderntesting.org/) state - testing can synergize with a great experimentation culture, leading to possible delivery of a quality product in a shorter time and cost-savings on the development cycles. But this cannot be achieved without robust test automation. And automation needs to follow a few rules. For that, we need to consider the properties of great automation.

Traits of good automation
-------------------------

For automation to be useful in the DevOps world, it needs to be: 

-   Reliable 

-   Fast

-   Easy to write and maintain new test 

-   Easy to run

### Reliability

If test automation is not reliable, we won`t believe its results. As a result, we will still depend heavily on manual rechecking of everything and that is time-consuming. Or even worse, it can lead to wasting time as now no one will even bother to check the results.

### Fast

The development cycle is fast. If developers need to wait for the results of their work more than a coffee break, they will switch to the next task. Now the results will need to wait which leads to the cost of context switching. And longer [lead time of a task](https://kanbantool.com/kanban-library/analytics-and-metrics/kanban-definition-of-lead-time-and-cycle-time).

### Ease of Writing and Maintaining

If automation should be a regular part of a software development lifecycle, their benefit must justify the cost. If writing a test takes 4 days, it'd better be worth it - especially if developing  the tested feature takes less than that. The same holds true for maintenance: we need to be able to easily understand why a test has failed and whether this is a code issue or a test issue and how to fix it.

### Ease of running

This relates to being fast: anybody who needs to use a test to check a code should be able to do it without extra hoops, for example automatically running them locally or via CI/CD.

How to choose the best automation approach
------------------------------------------

There are a lot of ways of choosing tools for a project. When looking for tools that resolve a problem, we need to choose a tool that contributes the coveted value. The list of questions requiring answers is long, but today we will look at only one aspect, namely automation development and maintenance cost. We will look at the 3 most popular approaches, starting from ones requiring most coding and maintenance effort, like "Custom Frameworks", through usage of "Out of the box" Frameworks and finishing with Record and Play tools. We will look at the pros and cons. As it is my belief, all these approaches have made a name for themselves on the market and suit different project needs.

**Pros**

-   Great Customizability

-   Ownership

-   "On-site support"

**Cons**

-   Huge Maintenance Cost

-   Time Consuming

-   Requires lots of skill to Maintain

When I was starting test automation, this was the way to go. Mainly because Selenium Web Driver was the tool of choice back then. And as its description stated, [Selenium automates browsers](https://web.archive.org/web/20120208115222/http://seleniumhq.org/). Selenium was just a tool that did exactly what it said it delivered.

It didn't provide any functions for running test, or making assertions. 

At that time, practically every company had to build its own framework.

Often it was the role of test automation testers. In most cases, they were just learning how to automate. Making lots of mistakes in the process. But some companies managed to make great frameworks and, as a result, decided to share them with others. Some of them are still in use and developed but other have not stood the test of time. [Ocaramba](https://github.com/ObjectivityLtd/Ocaramba),[Selino](https://github.com/TestStack/TestStack.Seleno), [Serenity](https://serenity-bdd.github.io/theserenitybook/latest/index.html),[Selendie](https://selenide.org/), [Protractor](https://www.protractortest.org/#/)

This approach is still used today. Why?

### Pros

The main advantage of this approach is control. It gives a company the greatest level of control in customization. If a specific feature for a project is required, it is relatively easy to find someone who can develop this functionality.

Additionally, it gives a company considerable independence; there is no risk that at one day, the tool support will end or [repo will be deleted](https://nakedsecurity.sophos.com/2022/01/11/javascript-developer-destroys-own-projects-in-supply-chain-lesson/), leading to massive issues.

What is more, there is someone who knows how the tool works on site and can teach new team members.

### Cons

The main drawback of this solution is the cost - maintaining internal test automation framework costs time and money. Many companies underestimate the cost of developing test automation.

Time is of essence here: this automation needs a long time to be developed,  from creating a framework to writing a test. And this is the aspect that can be most damning. As discussed before, for automation to bring value, it needs to provide fast feedback.

Creating a custom framework is as big a project as any and it requires a good dose of skills, ideas, and planning. This means a lot of expertise while typically, companies are not willing to spend precious resources on test automation.

Out of the box frameworks: Pros & Cons
--------------------------------------

**Pros**

-   Shorter time needed to write first test

-   Much less need for framework maintenance

-   Lots of available online trainings and knowledge

**Cons**

-   Often only limited support from community

-   Writing and Test Maintenance takes time

-   Huge dependence on framework developers
As mentioned before, the growth of Selenium was accompanied by a lot of frameworks created on its base. Over time, the issues with the web driver became increasingly prominent, leading to the re-emergence of new automation frameworks not based on Selenium: [Cypress](https://www.cypress.io/), [Puppeter](https://github.com/puppeteer/puppeteer), [Playwright](https://playwright.dev/), to name a few most recent ones.

All these tools had one thing in common - they provided the user with out-of-the-box, ready options to write automation. No matter if this was Ocarmaba based on Selenium or Cypress: the goal of the solution was the minimalism needed for framework maintenance.

These tools usually provide built-in test runners, an extensive assertions library, boiler plates for starting writing project and tests, and - most importantly- a built-in mechanism to deal with test flakiness and general instability.

Additionally, these frameworks usually provide their own philosophy of writing test automation. As long as what we need to do is with accordance to their approach, everything goes well. Problems start when we need to go beyond what is suggested.

Until recently, the best example has been a test covering multiple domains in Cypress, but as of [version 9.6](https://docs.cypress.io/guides/references/changelog#9-6-0), it is no longer such a grievous case.

In another scenario, we need to rely heavily on framework developers, if they introduce breakthrough changes. Usually, we are left with two options: update and fix the code. Alternatively, don't update and hope that there won't be any issues in the framework that need fixing.

### Pros

Compared to the custom frameworks, we can start writing tests almost immediately without the need of spending weeks or months on writing the toolset first. Similarly, we don't need to wait whenever the operating system or browsers have an update. The tool maintainer will update the framework for us. And since most tools in this category are open-source ones, there is lots of information easily available online.

### Cons

As mentioned before, these tools have their own test philosophy, which is much more restrictive. Quite often support is limited - these tools are usually open-source and support in case of issues can be unavailable where we need to resort to forums and chats where we hope someone will answer a question; sometimes we get the answer in minutes but it may never come just as well.

However, the greatest risk of all is that we are at the mercy of framework developers - these tools become a part of our code base and if we want to keep support, sooner all later we will need to update even if we don't want to. A case in point is Cypress 10 - [it introduced breakthrough changes](https://docs.cypress.io/guides/references/changelog#10-0-0). As a result, many plugins and customer functionalities won't work without extensive updating.

Record and Play Testing Tools: Pros & Cons
------------------------------------------

**Pros**

-   Fast Test Writing Time

-   Low Entry Level

-   Professional Customer Support

-   Easy Scalability

**Cons**

-   Usually cloud-based, requires access to test environments

-   Still young, "untested" technology

-   Requires a new mindset

-   Data management can be quite complicated

-   Usually a commercial paid tool


The idea of record and play automation is not new, even Selenium has Selenium IDE which, after a few years on developments lately, has won some users' recognition.

Microsoft had Coded UI Test as part of the Visual Studio from 2010 and only in [2019 announced it deprecation](https://devblogs.microsoft.com/devops/changes-to-coded-ui-test-in-visual-studio-2019/)

Back then this technology was long before its time, the frameworks were very unstable and the maintenance of this test was problematic.

But in the last 4-5 years, a lot has changed. Solutions like [Bugbug.io](https://bugbug.io/), [Mabl](https://www.mabl.com/), [Testim](https://www.testim.io/) have taken over what modern technology has to offer and reinvented the concept.

### Pros

Most coded automation requires great investment of time into writing and maintaining test - usually even a trivial change costs around half an hour. In case of Automation Records and Play, test are usually meant to be disposed of  after they  stop working, as it is much easier to re-record a test instead of fixing it. This is of special importance for startups and projects with a lot of changes in the UI layer. When a team is spread thin, it is hard to find the time to write a test. But usually, you can record a few tests while checking if what has been done even works.

On the other hand,e out-of-the-box coded frameworks offer usually an easier learning curve. Record And Play tools usually do not require coding knowledge - although in more complicated cases some JavaScript knowledge can prove very helpful.

Since most of the record and play tools are paid solutions, it also means that companies have professional Trainers and Customer Support to help when issues arise. Usually, it means that in the case of young projects, we can even influence their feature roadmap.

These modern record and play solutions come with their own cloud runners, making scalability of test automation unbelievingly easy. Typically, paralyzing a test takes a few clicks.

### Cons

Unfortunately, Cloud Running is also a huge Con. First and foremost, we need to give this tool access to our test environments. 

An additional issue related to the Cloud is Term of Service - not all projects and customers can afford cloud services.

Most of these Record & Play tools are relatively young and not battle-tested. For long-term projects, this uncertainty can be very risky.

In my opinion, the biggest issue is Data Management. So far, no Record and Play solution has provided an easy way to manage test data. And this can be a huge blocker. Consequently, we need to change our approach to E2E test automation, with a desire to cover all the important and easy paths as fast as possible, to make this automation possibly most valuable.

Lastly, there is the price - cloud solutions are not cheap and if we need to run a lot of tests, it may be quite expensive. But some tools like [BugBug](https://bugbug.io) are on the other hand very cheap and offer unlimited cloud runs for a fixed mohtly fee. Arguably, it is still less expensive than maintaining coded automation and its infrastructure.

No clear-cut distinction
------------------------

Before the final thought, some clarification is required:

As much as we humans like clear categorizing, the real-word borders are much blurred.

For example: should Playwright be in the Custom frameworks or in the Out-of-the-box category? After all, its strength is the ability to use its own test runners, reporting services and easy building of the new features we need.

Similarly, it has the Record and Play functionally and so did Cypress (at the moment of writing this post, the latest version of Cypress is 10.3 and since 10 Cypress Studio, which allowed test recording, has been deactivated).

There are also tools that I like to refer to as combine-harvesters: basically, they do everything and claim that they can test anything, from mobile via web to desktop and many others -- the examples include TOSCA and Test Complete.

That's why we need to look at the numerous different dimensions of automation - and how much coding is just one of them.

Summary
-------

It is my belief that over time, both no-code development and record and play automation become the dominant players on the market. Yet there is a lot of room for improvement. All the types of approaches to automation, starting from custom frameworks, or using out-of-the-box coded solutions ending on Record and Play tools, have their uses on the market.

They answer to different needs and solve different problems. So, before selecting any tool, it is always good to ask yourselves: "What are we trying to gain?"

large_image (3).jpg

medium_image (3).jpg

small_image (3).jpg

thumbnail_image (3).jpg

image (3).jpg

Record and Play vs Coded Automation testing - how to choose what works for your team

record-and-play-vs-coded-automation-testing

We’ve prepared a short guide with essential recommendations on test automation that we created specifically for startups. We divided the guide into 3 levels:

[★☆☆ Level 1: get the most out of basic automation](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/)

[★★☆ Level 2: automate hundreds of regression tests](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)

[★★★ Level 3: integrate with CI/CD builds](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-3/)
  
Also see our guide dedicated to non-technical people: [beginner's tutorial to automation testing with BugBug](https://docs.bugbug.io/in-depth-guides/beginners-tutorial-to-automation-testing)
 
### What will you achieve on Level 1?
After finishing this level you will run 5-10 automated tests every hour and check if your core features are working properly. You will detect production bugs with big impact on users sooner than before.

### TLDR:
-   Start with short and simple tests that check your core features
-   Remember about recording “hover”
-   Create special test user accounts dedicated to automation
-   Schedule the tests to run every hour
-   Incrementally add more tests

### Focus on the most important features

![80-20 rule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/80_20_rule_1_c2126d9815.png)

Many companies spend endless hours on a neverending pursuit to automate everything. This leads to very costly automation that requires a whole team to create and maintain. Don’t be like these companies and keep it simple. Agile startups should start with automated testing of the most essential functionality. 

> 👍 **Rule of thumb:**
> Usually, it just takes 5 to 10 automated tests to be sure that most users are able to achieve their goals.

What should you test? Think of your features. What percentage of users are using a particular feature? If the answer is higher than 80%, then this is a core feature. The goal of automating tests of such important features is to make sure that you can detect bugs that have a huge impact on your business as soon as possible (new user registration is for sure a core feature, but it’s a little bit more tricky to automate, which we will cover in the next episode)






### Start with short & atomic tests

When you [record your first tests](https://docs.bugbug.io/creating-tests/recording-the-tests) start with something very simple and atomic. Atomic means that the test is small, independent and checks just one thing.

A good idea is to start with testing the basic navigation of your landing page. Click “record” and create a test that just clicks one item of your navigation link in the menu. Add [assertion](https://docs.bugbug.io/recording-tests-steps/recording-assertions) for some text in a heading. Finish the test and run it. Awesome! You’ve created your first automated test that touches the most essential area of your business!

![Screenshot 2022-07-25 at 13.44.12 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_12_1_0ebde0ddbb.png)

Now create a separate test for each navigation link. Here are screenshots of how BugBug team set up their landing page tests.

![Screenshot 2022-07-25 at 13.44.15 (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/Screenshot_2022_07_25_at_13_44_15_1_74fb39e9de.png)


### Don’t forget about recording hovers

If your website navigation appears on hover, you need to tell BugBug that you want to record a mouse moving to a specific element. To do this you need to click “Hover” during the recording and then click an element. When running the test, BugBug will move the mouse cursor to this element.

![bugbug record hovers.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/bugbug_record_hovers_81639fbd30.png)


### Create special test users

Another idea is to test if login works. Create a separate test that only checks that one thing.

To achieve that you need to prepare a “test user” - a user account that is dedicated only to testing if login works properly

-   Manually register a new user with a new unique email, for example with john.smith+testautomation@email.com (you can use [disposable mail](https://www.google.com/search?q=disposable+mail) or [plus aliasing)](https://support.google.com/a/users/answer/9308648?hl=en)
-   Go to BugBug and record a test that clicks the login button, enters the user login and password and submits the form
-   Add an assertion for some text that confirms that the login was successful
-   Finish your test
    

  

You might be tempted to test other features in the same test, but don’t do this now. We want to be sure that the login test is atomic and independent from other features.

  

Remember that you can create as many test users as needed. For example, if your product offers multiple combinations of preferences, you can create a separate user for each combination.

  

> 👍 **Rule of thumb:**
> Always create new test users if this leads to less test steps

  

### Schedule all tests to run every hour

With BugBug you just need a couple of clicks to start running the tests every hour. Just go to “Schedules”, select the “Hourly” one and add “All tests”.

![hourly schedule (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/hourly_schedule_1_445d7fe266.png)


  

That’s it, now you will start getting emails every hour with the results of your test automation. Tip: create a filter on your email account so that you avoid inbox clutter or [configure notifications to only get an email on failures.](https://docs.bugbug.io/running-tests/schedules#configure-email-notifications)

![email notifications (1) (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/email_notifications_1_1_a23f96ba47.png)


  

### Incrementally add more tests

You’ve successfully created test automation that monitors your production! Now keep adding more short tests one by one. Don’t forget to focus on being cost-effective - if you have trouble automating a use case, skip it and go to the next one or find a different way of checking if it works. And remember: you can always [ask BugBug support to help you with any troubles](https://bugbug.io/contact).

Read next: [next level guide on how to easily maintain hundreds of automated tests ](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-2/)
  
  

medium_illustration - guide - level1.png

small_illustration - guide - level1.png

thumbnail_illustration - guide - level1.png

illustration - guide - level1.png

Automation testing guide for startups - Level 1: get the most out of basic automation

automation-testing-guide-for-startups-level-1

In the [first episode](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/) of our 3-part guide, we covered the basics and described how to get the most business value from simple test automation. Now it’s time to learn some more advanced concepts and increase coverage of your automated tests. Here we go!
 
  
### What will you achieve on this level?

Once you finished this level you will:
-   have 100+ nicely structured tests
-   be able to update multiple tests at once when needed
-   run tests on different environments
-   automate testing of new user registration
    

  

### TLDR:

-   Organize your test steps in groups
-   Convert some groups into shared components
-   Duplicate tests efficiently with “new test from here”
-   Set up a URL variable to quickly swap between environments
-   Set up a testing email inbox to automate tests of new user registration
-   Use [built-in variables](https://docs.bugbug.io/editing-tests/variables#use-built-in-variables-for-dynamic-or-random-values) to generate random user email aliases
    
### Scaling the test automation

![illustration-number-of-tests.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/illustration_number_of_tests_7ee4d49d68.png)


In this episode, we will focus on scaling your test automation to a larger number of test cases so that your regression tests also cover corner cases and niche features. This is “Level 2” so things will get more complicated, but BugBug makes everything easy to achieve.

Example: in [the episode 1 of this guide](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-1/) we’ve created just one test that checks if log-in works. Now we want to create several tests that check different outcomes of logging in, for example:

-   Log in - success
-   Log in - incorrect password
-   Log in - account blocked
-   Log in - password reset required
-   Log in - account doesn’t exist
    
You can see that we aim to increase the test coverage dramatically. Instead of 5-10 tests, we’re going to have hundreds of them.

### Keep tests nice and tidy

If you want to have more than 10 tests, you should start organizing the test steps and keep them in order. Why is it important? Imagine that you’ve created 100 tests, but later you changed something in the UI. All of your tests started to fail. If you haven’t organized test steps, you will need to go to each test individually, see what’s wrong and re-record the failed steps 100 times. Fortunately, BugBug allows you to easily avoid such situations.

### Group your test steps

Keep your steps tidy by grouping them. Give a clear name to each group, which will help you to navigate through long test cases. Move your cursor in between the steps to see a plus icon, then click “[split group here](https://docs.bugbug.io/editing-tests/grouping-steps)”.

![split-group-here.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/split_group_here_fce12649d5.png)


  

### Convert groups to shared components

Once you start grouping your test steps, soon you will realize that some groups are exactly the same in multiple tests. This makes them a perfect candidate for [components](https://docs.bugbug.io/editing-tests/wip-make-component). Components are magic groups: if you change it in one test, it will change in all other tests that have it.

![component-made (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/component_made_1_afa56da91d.png)
  

Start with creating a component for the first step that goes to your starting URL. Then go to each test and replace the normal group with the component by using “Insert existing component”.

![insert-existing (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/insert_existing_1_985f99b0c0.png)


  

### Duplicate the tests rather than create new ones

Since you already have components, you may not want to record each test case from scratch. A lot of your tests will start with the same components (for example: go to homepage, log in, enter a password, etc.). You can quickly create a new test “in the middle” of a different test. When you click “[new test from here](https://docs.bugbug.io/creating-tests/new-test-from-here)” you will create a duplicate of the test that keeps all the preceding test steps.
 
![new-test-from-here.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/new_test_from_here_5d79377e02.png)


  

If you use components in a smart way, you will be able to fix hundreds of tests with just one [re-recording](https://docs.bugbug.io/recording-tests-steps/re-recording-steps).

### Use data-testid attributes as selectors

Automated tests are most reliable if your [selectors](https://docs.bugbug.io/preventing-failed-tests/selectors) are clearly identifying an element. The best way is to clearly annotate such elements in your HTML structure. Ask your frontend developer to add `data-testid` HTML attributes to elements that you click in your automated tests. 

Why is it better? Let’s see an example. If you don’t use a dedicated attribute, your login button selector might look like `//SPAN[normalize-space(text())="Log in"]`. This selector points to any element with text "Log in". If you change the text in the button, such a selector will stop working and your test would fail. 

If you used an attribute, for example `[data-testid=”login-submit”]` your tests will be more resilient to changes. Even if you change the text in the button, you will not need to edit your automated test. What’s more, the developers will see this attribute in the code so they will be aware that this specific item is a subject of test automation. Once you have such attributes in your HTML, BugBug will automatically create selectors based on “data-testid” attributes during the recording. 

![data-testid.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/data_testid_669fc25baa.png)

### Build robust selectors without coding

Sometimes it's not possible to create a reliable selector with recording. Dynamic lists, tables and hover actions are notorious for being difficult to record. You just need to use your human intelligence and use a more complex XPath selector. We created a dedicated tool for that with an in-depth guide - see [No-code XPath Builder](https://bugbug.io/xpath-selector-builder/).

![no-code-selector-builder.png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/no_code_selector_builder_b90d46b1c1.png)

### Work with different environments

If you are fully committed to testing automation, chances are you will need to run tests on different environments. Each environment can be accessed with different URLs, for example:

-   Production - acme.com
-   Staging - stage.acme-test.com
-   Local - localhost:3000

You should store these URLs in a variable with 3 [profiles](https://docs.bugbug.io/editing-tests/variables#profiles). After you set up more than one profile, a new dropdown menu will appear in each test, that lets you run tests with different environments. Learn more in our [detailed guide](https://docs.bugbug.io/editing-tests/variables#work-with-different-development-environments).

![profiles (1).png](https://bugbug-homepage.s3.eu-central-1.amazonaws.com/profiles_1_e2d0e47b36.png)


  

### Automate tests for new user registration

To reliably automate user registration you need to use a combination of several advanced features

-   Third-party inbox that supports [plus-addressing](https://www.msoutlook.info/question/886#:~:text=Exchange%20Online%20(Microsoft%20365)%2C,also%20known%20as%20%E2%80%9Csubaddressing.%E2%80%9D) (for example Outlook.com)
    
-   Built-in “testRunId” variable for generating random e-mail address alias
    

  

Learn more in our separate step-by-step [guide on testing sign-up.](https://docs.bugbug.io/editing-tests/variables#test-user-registration-and-login-using-variables)

### Keep it simple

Don’t try to overcomplicate your test steps. Use components and variables only when it will actually save time for you in case of product changes. Remember that re-recording is a really fast way of fixing existing tests - sometimes it’s just faster to re-record steps in 5 tests than debugging a complex net of variables and dependencies. Keep it simple and if you have any problems feel free to [talk to our friendly customer support](https://bugbug.io/contact).  😊

[Read next: advanced guide with tips on CI/CD integration](https://bugbug.io/blog/software-testing/automation-testing-guide-for-startups-level-3/)

medium_illustration - guide level2.png

small_illustration - guide level2.png

thumbnail_illustration - guide level2.png

illustration - guide level2.png

Automation testing guide for startups - Level 2: automate hundreds of regression tests

automation-testing-guide-for-startups-level-2

Tools for automation