Search for a command to run...
Kickstart Your Mobile Test Automation Journey in 2025
Software Development Engineer in Tests. Passionate about programming in Java & JavaScript/TypeScript. Experienced, former employee of the insurance industry. Graphic designer by choice. Compulsive learner.
No comments yet. Be the first to comment.
TL;DR This post explains how iOS Universal Links work and what is required to implement them. It covers how to handle incoming links in an iOS app, how to create and configure the Apple App Site Association (AASA) file, and how to correctly host the ...
TL;DR The Page Object Model (POM) is the industry standard, but it is not easily scalable. The Flow Model Pattern improves upon the basic POM, offering a perfectly balanced solution that is not overly complex yet still scalable. The Screenplay Patter...
Preface In a previous article from the start of this year, I mentioned various frameworks that can be used for mobile test automation. The one that came up repeatedly was WebdriverIO, which I have been using actively for over two years now. It is a s...
Let’s be honest - who wakes up and and is excited to read button labels or error messages? But have you ever clicked a vague “Next” button and immediately regretted it because you had no idea where it was taking you? Or stared at an error message lik...
Part 1 - An Introduction to the matter What will this series be about? During this series of blog posts, I will do my best to provide my opinion on specific models, determine which use case each one fits, discuss the pros and cons of using that model...
When it comes to test automation in general, it's easy to say that it's complex. If we were to stick to the classic separation of the Test Pyramid, we'd end up with:
- Unit tests, which can be implemented for both backend and frontend.
- Integration tests, which today can mean anything from API testing, microservices and third-party service integration, user authentication and authorization, to quasi-performance testing.
- End-to-end tests, which for the vast majority of test automation engineers and software developers would mean web application testing (to name a few major frameworks: Playwright, Cypress, Selenium).
There are also tools like Pact for contract testing, Storybook for advanced visual and/or snapshot testing combined with interactive documentation, and probably many more that I don't even know about.
However, these tools are well documented and have huge communities around them, and to be completely honest - I don't think any of these tests are hard (except maybe the contract tests).
When I started working on mobile test automation, I quickly realised that this was not the case. The more I got into mobile software development, the more confusing it became, the mere amount of dependencies and requirements to just get the application running was really bizarre, let alone automating tests on it. The documentation on the various technologies was sometimes not very helpful, and there were no articles on the full spectrum of possible mobile environments.
So let's fix that and talk about mobile test automation, shall we?
First, before we go any further, we need to consider what technologies will be used to build the applications that will be tested with the mobile testing frameworks. The popular frameworks for mobile development are:
| Name | Maintainer | Technology | Supported Platforms | Product Page |
| Flutter | Dart | Android / iOS / Web / Windows / macOS | https://flutter.dev/ | |
| React Native | Meta | JavaScript | Android / iOS / Web | https://reactnative.dev/ |
| Ionic | Ionic | JavaScript | Android / iOS / Web / Windows / macOS | https://ionic.io/ |
| NativeScript | OpenJS Foundation | JavaScript | Android / iOS / Web | https://nativescript.org/ |
| .NET MAUI | Microsoft | C# | Android / iOS / Windows / macOS | https://dotnet.microsoft.com/en-us/apps/maui |
| Cordova | Apache Software Foundation | JavaScript | Android / iOS / Web / Windows / macOS | https://cordova.apache.org/ |
| Xcode | Apple | Swift | iOS | https://developer.apple.com/xcode/ |
| Android Studio | Kotlin | Android | https://developer.android.com/studio |
According to the 2024 Stack Overflow survey, Flutter is the most popular framework for cross-platform development. It's followed by React Native, Electron (not listed in our table because it only supports desktop app development), .NET MAUI (and it's older brother Xamarin, which has been replaced by MAUI), Ionic, and Cordova closing the list. Interestingly, there is no mention of NativeScript at all.
Below is a general overview of the test automation frameworks. There are some other frameworks that I have intentionally left out because they do not seem to be very popular (e.g. EarlGray, which is Google's framework for iOS automation), and they are most likely wrappers of the technologies listed in the table anyway (e.g. Selendroid or any of the cloud mobile test automation providers).
| Name | Maintainer | Main Feature(s) | Supported Native Platforms | Product Page |
| Espresso | Google’s default testing framework | Android | https://developer.android.com/training/testing/espresso | |
| XCUITest | Apple | Apple’s default testing framework | iOS / macOS | https://developer.apple.com/documentation/xctest/user_interface_tests |
| Appium | OpenJS Foundation/ Open-source (Apache-2.0 license). | Supports all of the mainstream platforms | Android, iOS, macOS, Windows, Roku, tvOS, Android TV | https://appium.io/docs/en/latest/ |
| Detox | Wix.com / Open-source (MIT) | Tightly coupled with the React Native architecture | Android, iOS | https://wix.github.io/Detox/ |
| Maestro | Mobile.dev / Open-source (Apache 2.0 License) | Due to implementation of *.yaml files as its test files format, it’s probably the most effortless mobile test automation framework | Android, iOS | https://maestro.mobile.dev/ |
From the above list, Appium can be considered as the most comprehensive and therefore the most complicated framework of them all. In short, it consists of four basic elements: Appium Core, Drivers, Clients and Plugins.
We won't go into the details of what each element stands for, but to give you a better understanding of what technologies are supported by the Appium framework, here is a list of so-called Clients:
| Name | Technology | Product Page |
| WebdriverIO | JavaScript (Node.js) | https://webdriver.io/ |
| Appium Python Client | Python | https://github.com/appium/python-client |
| Appium Java Client | Java | https://github.com/appium/java-client |
| AppiumLib | Ruby | https://github.com/appium/ruby_lib |
| Appium .NET Client | C# | https://github.com/appium/dotnet-client/ |
When you would have to make one of the key decisions in the context of testing strategy, that is, the choice of a framework for automating mobile end-to-end testing the choice is quite complicated, as it depends on many factors.
Let's do a little thought experiment and try to describe some possible scenarios to help you decide.
TL;DR:
When developers are using native development tools and want to maintain grey/white box end-to-end testing and/or you’re building an MVP: Espresso (for Android) / XCUITest (for iOS)
If you're developing an MVP and you only need UI testing (without physical iOS devices and with limited scripting possibilites): Maestro
If you're developing a cross-platform React Native application and want to improve test scripting experience with Large Language Models: Detox
For long-term projects that require stable solutions: Appium-based platforms
The first scenario is an example for likely early stages of development (e.g. seed stage of a startup) when the focus is on only one mobile platform. This means that the natural choice should be either Espresso (for Android) or XCUITest (for iOS).
However, there may be other things to consider, such as:
For example, if the application is being developed on a tight schedule and/or budget, and the developers simply do not have the time to maintain the end-to-end test suites, choosing complex Espresso or XCUITest frameworks can end up being a disaster.
In this type of environment, I would suggest using Maestro as it seems to be the most hassle-free and does not require too much time and effort to maintain test suites and configuration.
If your project's time and budget constraints are not the things that keep you up at night, then you should probably think a little more about the potential development path.
I want to emphasise that we should avoid over-engineering, but on the other hand, it's also important to choose a framework that we can use in a more mature test environment. By a more mature test environment I mean things like automated test data generation and cleanup (which can be achieved with API-based model) or implementation into the existing CI/CD pipelines. And of course we want these things to be relatively easy to implement.
In summary, if you are sure that your project is entirely Android or iOS based, developers are the ones who will write and maintain the test suites, and you do not plan to extend it to the opposite technology, I would suggest using Espresso or XCUITest, as they will give you the most solutions adapted to your existing codebase. Otherwise, I'd lean toward the Appium-based solutions because they're much more extensible and can be used in black-box environments.
Most of the time, companies want to target the broader audience, and since Android has ~72%* of the market share, which means that iOS users have ~28%* of the market share, it means that most companies want to target both platforms.
This scenario, at least from my perspective, is much more complex in terms of things that need to be considered before an informed decision can be made.
Separate development teams for Android and iOS
If your application is developed by separate teams using native development tools (i.e. Android Studio and/or Xcode), you can probably integrate Espresso and XCUITest to perform end-to-end testing. However, I would like to point out that this may be a subpar solution, as other frameworks allow you to test both platforms using the same test scripts.
So, for the situation mentioned in this sub-scenario, I'd suggest using Espresso and XCUITest for smaller scale tests (component testing if you will) that would be maintained by the developers themselves and integrating broader scoped frameworks for end-to-end testing (i.e. Appium or Maestro).
Cross-platform development
As you can see in the table mentioned in the Development Frameworks Overview section, there are many possible frameworks to choose from to develop cross-platform applications. Most frameworks support all platforms (mobile, web, and desktop), but when it comes to mobile development, we focus mostly on Android and iOS.
Unfortunately, in order to figure out which framework we should choose to write and maintain end-to-end mobile tests, we need to proceed with the sub-categorisation of possible needs and settings.
If you really don't have the time to maintain test suites, and therefore need something simple, almost effortless to write and maintain, I would suggest going with the Maestro.
If you are an SDET or an experienced test automation engineer, you may want to lean more towards Appium-based tools, as Appium gives you all the benefits of mobile test automation as well as the freedom to configure the entire test automation framework yourself. I'm not saying that Maestro can't do this, but please take a look at the Maestro Afterthoughts for more insight.
Cross-platform development with React Native
The tool I haven't mentioned yet is Detox. This is because Detox is specifically designed for grey-box end-to-end testing for React Native. I'd recommend giving it a try if you're developing a React Native application, especially given the fact that it's the first known framework to incorporate the Large Language Models to write tests in natural language.
However, if you and your team are developing applications for external customers (so-called enterprise-grade applications) that require stable tools, I'd strongly recommend using Appium-based tools because they are much better documented and have much larger communities built around them.
The last scenario we'll talk about is native mobile development combined with any other version of the app. This is paradoxically the easiest of all.
Appium.
There is simply no other solution that would allow you to test on all of the available platforms.
Of course, if you'd like, you can mix and match Maestro for mobile testing and Playwright for web testing. Or Detox for mobile, Appium for smart TV, and Selenium for web, if that's what you want. It just doesn't seem like a good idea, especially since some of the Appium-based frameworks allow you to create a single test script that would run sequentially on multiple platforms.
For example, WebdriverIO allows you to do something on one mobile device, then switch to a separate web application to change/verify values and perform some other actions on another mobile device.
I felt that Maestro might seem like a great solution in the end, which it can be in some cases, but I also want you to think about possible downsides if you decide to go with it.
Maestro gives you the ability to configure the test automation framework (e.g. run JavaScript code), but since it's written with a specific architecture in mind, it's more of an add-on to the standard YAML flow files than anything else.
Another thing to keep in mind when talking about the Maestro is last year's Cypress intellectual property update case. In a nutshell, the Cypress team removed the ability to use third-party services for dashboards that were previously free, essentially forcing companies to use their own cloud platform.
Not only is that platform not free, but it also prevents Cypress from being used where contracts with customers specifically block the use of cloud platforms due to GDPR or similar laws. I mention this because Maestro has its own cloud platform for running tests called Robin, and it's possible that the similar scenario to the Cypress update could happen to Maestro as well.
The last thing I wanted to mention here is that if you want to write tests for the Apple ecosystem, namely iOS, iPadOS, macOS, or tvOS, you actually need to own a machine based on macOS (either a MacBook, Mac Mini/Studio/Pro, or iMac) because you need to have Xcode installed.
The reason is simple - it contains the Simulator application, which allows you to run your application in the virtual environment and is required to run tests on real devices. You'll also need an Apple Account, and if you want to publish your application on the AppStore, you'll need to sign up for the paid Apple Developer Program.
