Choose the right E2E automation framework for React Native Windows
React Native Windows is an open source project created by Microsoft, which enables React Native developers easily target their application on Windows. Compared to iOS and Android implementation of React Native, React Native Windows is pretty new. How to choose a right E2E automation framework and start your automation testing for React native Windows is still very challenging. I spent about two month to implement E2E for React Native Windows and want to share my thoughts on this topic. Please note these thoughts are my own and do not represent Microsoft.
I started a tweet survey, and three E2E test frameworks are used by developers for their React Native application: Detox, Appium and Cavy. A lot of customer adopted Detox, and then Appium, and finally Cavy.
Appium has a very large community, which is widely used out side of React Native. Personally I think if Appium is coming out first, Detox may not be created. Detox is widely used by React Native projects, and it runs fast than Appium. But it’s restricted to React Native apps, also it doesn’t support React Native Windows.
For Windows, To implement Detox on Windows is possible, and I spent two days to prototype it, but I finally dropped it because:
- Microsoft doesn’t have EarlGrey kind of open source implementation on Windows
- Detox is restricted to React Native project, but it requires significant invest on the future maintenance.
- WinAppDriver is the recommended UI automation tool by Microsoft and it’s very mature, also owned by Microsoft.
- WinAppDriver has nice integration with Appium.
Hereafter if I talk about WinAppDriver, I actually means Appium+WinAppDriver.
So If you want a solution on Windows, currently Cavy and WinAppDriver are the only two choices. Since Cavy is very simple, it’s worth you to try Cavy before you make your final decision.
For React Native Windows, I experienced some technical decisions and you may hit too like below:
After WinAppDriver is selected, if your target is only on Windows, and you are familiar with C#, C# is the best choice. It has nice IDE integration and less dependency.
- selenium-webdriver doesn’t support mobile JSON wire protocol yet, so I created selenium-appium project to fill the gap.
- WebDriverIO follows w3c webdriver protocol, but WinAppDriver doesn’t, so I created forked WebDriver to workaround the problem. Expect you can remove this restriction when next WinAppDriver is released.
To ease your setup, I already created some examples for using selenium-webdriver and WebDriverIO.
- E2E test with WebDriverIO for React Native Windows
- Xaml-Controls-Gallery Examples with both selenium-webdriver and webdriverio + Azure Pipeline
- Calulator Example with Jest + selenium-webdriver + WinAppDriver
- Calulator Example with Jasmine + webdriverio + WinAppDriver
selenium-webdriver or webdriverio? It’s hard for me tell you which one you should choose. I created https://github.com/react-native-windows/winappdriver-js-webdriver-example/tree/master/packages and you can see that there is very little difference when you author test case for them. If you are a fan of Jest, webdriverio doesn’t support it. If you are tired of using await in test case, wdio-sync on webdriverio is the life saver.