C2LC-299: Research End-to-End Browser Testing

Description

Currently we only simulate user interaction using Enzyme and don't actually test things like the WebAudio or WebSpeech APIs directly. There are solutions (like webdriver.io) that can help us test more things in actual browsers.

Comments

• Michelle D'Souza commented 2021-01-25T15:04:01.829-0500

  @@Tony Atkins [RtF] has read up a lot on webdriver.io

• Tony Atkins [RtF] commented 2021-04-15T05:21:54.360-0400

  Just to add a bit more detail, when I first looked into this, I reviewed the (Selenium) Webdriver API reference implementation, Cypress, and webdriver.io.  I gravitated towards webdriver.io because it has react-specific functionality that seems useful.

  My first question at this point is, does it work with GitHub Actions?  Can we test on the platforms we care about (notably Safari)?  Do we need a separate service for that, such as browserstack?

  I also want to understand whether/how this handles coverage data.  Ideally, we could remove most of our istanbul ignore comments and use the browser tests to cover things that only come up in one or more real browsers.

• Tony Atkins [RtF] commented 2021-04-15T05:33:21.904-0400

  Their GitHub Actions docs give an example of using a (presumably linux) image with Headless Chrome.

  It's presumably possible to run this in a container with xvfb, as I've done before for Firefox with webdriver.

  Here's an example of using webdriver with GitHub Actions that includes jobs that run on all browsers and on both Windows and OS X.

• Tony Atkins [RtF] commented 2021-04-15T05:39:47.199-0400

  Just found a good reference for using webdriverio across all platforms, including running mobile Safari tests using the iOS simulator:

  https://github.com/blueimp/wdio

  However the CI workflow for the package only demonstrates using FireFox and Chrome on Ubuntu.