Selenium IDE is a tool for recording and launching application tests. It was created by Shinya Kasatani in 2006 (sic!) and is based on „record & playback” idea, which nowadays is no longer a novelty. The problem with most of the tools available on the market (including Selenium IDE) is not the outdated idea, which basics still can be perceived as elegant, but its realisation. Below we listed the biggest accusations that can be directed towards Selenium IDE and solutions to them offered by the tool named BugBug.
Generation of improper elements’ selectors
While recording the application tests, the newest version of Selenium IDE is able to generate several selectors for the element with which the interaction was made. The problem is that this tool doesn’t give you the possibility to configure which type (or types) of selectors should be set as default ones. It also prefers wrong approach to the generation of selectors, based on the names of the CSS classes. Nowadays, when CSS-in-JS approach is becoming more and more popular (i.a. due to styled-components, which generate random class names), it makes the user responsible for modification of all selectors generated during the recording session. The result is that the work that should be automatically done by the tool is done manually and at a much slower pace by the user.
Solution to the problem provided by the BugBug
During a recording session, BugBug – just as Selenium IDE – generates many selectors for the elements with which interaction was made. The difference is that in this case the user decides about the hierarchy of the generated selectors. It can be easily modified in the configuration of the project:
Thanks to this solution you have complete freedom and control over the generated selectors – you just act accordingly to the project requirements. Nowadays, testing in the projects very often is based on the data-test or data-testid attributes, that are also preferred by the BugBug. However, if you have to deal with the older type of the project, you can always prioritize generation of the selectors based on the CSS classes or elements ID. The key here is that the user independently decides what he needs and changes configuration of the project accordingly.
While recording the application tests we are not always sure if the selectors are correct. That is why the BugBug at the first launch and after each change in the test verifies selectors and corrects them if there is such a need – for example if the selector disappeared for some reason (it could be caused by the random CSS classes) or it points to the improper element. Thanks to implementation of these mechanisms interference of the tester with the process of the selectors’ selection is reduced as much as possible. He also saves his precious time, so far wasted for manual searching and creation of the selectors.
Lack of active waiting for the elements (waiting conditions)
Another common problem with the record & playback tools, which concerns also Selenium IDE, is lack of active waiting for the elements with which interaction should be made. In practice it works like that: if the chosen element is embedded on the website (in DOM), Selenium IDE attempts to interact with it, even though it can be invisible, moves, animates or is covered by another element (loading overlay). The result of such scheme of actions are so-called “flaky tests”. This notion defines the tests which once work, and another time they don’t. The reason for such behaviour is very often lack of appropriate waiting conditions for the element with which the interaction is to be made.
Solution to the problem provided by the BugBug
Creators of the BugBug took into account defects of the tools available on the market and effectively addressed them. On the level of the project configuration you can define waiting conditions that have to be met so that the interaction with the element could proceed:
The global configuration of the project is inherited within all steps of the test. Of course there is also possibility to overwrite this configuration on the level of the single step. You can also add additional conditions there, such as waiting for the page reload:
Thanks to this solution BugBug emulates the human behaviour, what means that it waits for an appropriate moment to proceed with the particular step of the test. The tool can actively wait when:
- element animates or changes its characteristics (CSS animations and transitions): the tool identifies that the element animates/moves and waits till the end of this process.
- element is invisible: BugBug’s mechanism is able to assess whether the element is in fact visible for the user. If so, it proceeds with the planned steps.
- element is covered by another element.
- element is inactive.
- element changed its position: the tool is able to identify change of the element’s position while it is proceeding with the step. In such a scenario the interaction is repeated. competitors.
Implementation of all the mechanisms listed above reduces the risk of „flaky tests” to minimum. Thanks to that BugBug offers much more stable tests than its competitors.
Misleading process of replaying the steps (with the use of JavaScript)
Selenium IDE as well as most similar tools available on the market proceeds with the steps of the test that only partially emulate behaviour of the average user. In practice it means that tests recorded this way can behave very unpredictably, leading to serious errors. It would be easier to explain with the use of the specific example:
The step click on any „Log in” button realised within Selenium IDE would trigger the following event in the JavaScript browser:
- click
Solution to the problem provided by the BugBug
The same step realised in BugBug results in the sequence of the following events:
- mousemove
- mouseover
- mouseenter
- mousedown
- mouseup
- click
The difference described above stems from the constraints of the Selenium IDE architecture, which can emulate the user’s behaviour only with the use of JavaScript. Architecture of the BugBug and implementation of the Chrome DevTools Protocol gives it full control over the browser. Thanks to it the behaviour of the user can be accurately emulated. It is crucial in testing contemporary, very extensive and complex applications (Single Page Applications).
Application tests are not launched separately
Majority of the record & playback tools has a problem with performing the tests separately. It results in a situation, when launch of the particular test impacts another one. The problem resides in the fact that cookies or localStorage mechanism in the browser are shared between windows and tabs. When you use Selenium IDE, you have to always delete cookies (or localStorage) and remember about the necessity to clear this data between the tests.
Solution to the problem provided by the BugBug
Despite the fact that the problem described above is simply burdensome, it also causes flaky tests. To prevent this, application tests in BugBug are by default performed separately with the use of Incognito mode in the browser. Thanks to that QA who builds the tests doesn’t have to remember about cookies, localStorage and other data sharing mechanisms in the browser. Tests are isolated regardless of whether you launch them in the browser or in the cloud.
Launching tests in the cloud is impossible
Selenium IDE was created many years ago, when cloud and SaaS services weren’t so widespread as they are nowadays. As a result, Selenium IDE functions only as the browser ad-on and doesn’t offer testing in the cloud. Tests created with the use of Selenium IDE can be exported to the chosen programming language and then launched within your own infrastructure (or with the use of some commercial solution).
Solution to the problem provided by the BugBug
BugBug from the very beginning was designed as a tool that aims at supporting the whole testing process (end-to-end). It enables for:
- recording and easy editing of the tests.
- launching of the tests locally, within user’s browser, in order to verify the tests.
- planning and launching application tests in the cloud (through www interface as well as CLI tool).
Look in our previous post to see 10 features every test automation tool should have.
Thanks to that you receive one, integrated tool that can be used for creation and launching of the tests as well.
Lack of support for the Page Object Model (POM) pattern
Page Object Model (POM) is a verified design pattern for end-to-end tests, which enables you to freely re-use steps within the tests, thus simplifying its maintenance. This pattern bases on creating group of steps that represent pages of tested Internet application as so-called Page Object. Thanks to that after you create “Log in” Page Object you can use it multiple times – in each test that requires logging in. If the log in form changes, it suffices that you fix the test in one place – as a consequence all tests based on “Log in” Page Object would be changed accordingly.
Unfortunately, Selenium IDE doesn’t support creation of the tests based on the Page Object Model. Steps in each test are strictly assigned to it. When you have many tests scenarios that use the same steps, if one step stops working, you have to fix it separately in each test.
Solution to the problem provided by the BugBug
BugBug was designed to support Page Object Model pattern from the very beginning. This tool enables you to freely group the steps in tests and use them as re-usable elements. What is more, a natural consequence of such an approach is the possibility of creating visualisation of the tests in one, coherent graph. Such presentation of all tests gives you clear information about which business paths within the project are covered with end-to-end tests.
Summary
