Choose the right E2E automation framework for React Native Windows
React Native Windows is an open source project created by Microsoft, which enables React Native developers easily target their application on Windows. Compared to iOS and Android implementation of React Native, React Native Windows is pretty new. How to choose a right E2E automation framework and start your automation testing for React native Windows is still very challenging. I spent about two month to implement E2E for React Native Windows and want to share my thoughts on this topic. Please note these thoughts are my own and do not represent Microsoft.
I started a tweet survey, and three E2E test frameworks are used by developers for their React Native application: Detox, Appium and Cavy. A lot of customer adopted Detox, and then Appium, and finally Cavy.
Cavy is a in application framework, and it can be used in any JavaScript container not just React Native app. I ever created Cavy Extension to extend the functionality of Cavy. Cavy itself is a very simple project, and it’s easy to use. But it also means that it needs native module support if you need functions like Keyboard. Another restriction is that if the native app crashed, there is no easy way to recover since test case is in the same process with the app.
Appium has a very large community, which is widely used out side of React Native. Personally I think if Appium is coming out first, Detox may not be created. Detox is widely used by React Native projects, and it runs fast than Appium. But it’s restricted to React Native apps, also it doesn’t support React Native Windows.
For Windows, To implement Detox on Windows is possible, and I spent two days to prototype it, but I finally dropped it because:
- Microsoft doesn’t have EarlGrey kind of open source implementation on Windows
- Detox is restricted to React Native project, but it requires significant invest on the future maintenance.
- WinAppDriver is the recommended UI automation tool by Microsoft and it’s very mature, also owned by Microsoft.
- WinAppDriver has nice integration with Appium.
Hereafter if I talk about WinAppDriver, I actually means Appium+WinAppDriver.
So If you want a solution on Windows, currently Cavy and WinAppDriver are the only two choices. Since Cavy is very simple, it’s worth you to try Cavy before you make your final decision.
For React Native Windows, I experienced some technical decisions and you may hit too like below:
After WinAppDriver is selected, if your target is only on Windows, and you are familiar with C#, C# is the best choice. It has nice IDE integration and less dependency.
If you want a cross platform solution, it’s good to write your test case on JavaScript. There are three popular webdrivers: wd, selenium-webdriver and webdriverio. wd doesn’t have typescript support, and I suggest you go with selenium-webdriver or webdriverio. For both solution, I successfully integrated them with WinAppDriver, but both requires workaround:
- selenium-webdriver doesn’t support mobile JSON wire protocol yet, so I created selenium-appium project to fill the gap.
- WebDriverIO follows w3c webdriver protocol, but WinAppDriver doesn’t, so I created forked WebDriver to workaround the problem. Expect you can remove this restriction when next WinAppDriver is released.
To ease your setup, I already created some examples for using selenium-webdriver and WebDriverIO.
- E2E test with WebDriverIO for React Native Windows
- Xaml-Controls-Gallery Examples with both selenium-webdriver and webdriverio + Azure Pipeline
- Calulator Example with Jest + selenium-webdriver + WinAppDriver
- Calulator Example with Jasmine + webdriverio + WinAppDriver
selenium-webdriver or webdriverio? It’s hard for me tell you which one you should choose. I created https://github.com/react-native-windows/winappdriver-js-webdriver-example/tree/master/packages and you can see that there is very little difference when you author test case for them. If you are a fan of Jest, webdriverio doesn’t support it. If you are tired of using await in test case, wdio-sync on webdriverio is the life saver.
