FLUID-6307: Uploader and TTS do not show coverage despite having tests

Description

As the enclosed screenshot shows, the Uploader and TextToSpeech components are showing as mostly without coverage despite having dedicated tests. I can't see a clear reason for this given the coverage configuration.

Attachments

Comments

• Tony Atkins [RtF] commented 2018-07-23T06:38:18.607-0400

  @@Antranig Basman, I'm afraid we have a thornier question here, which is "why isn't the missing coverage reported all the time?" Take a look at this report, generated just now in OS X from a test run against master:

  http://the-t-in-rtf.github.io/coverage-reports/infusion/20180723/

  The components are reported as covered, and the overall coverage is higher than your results.

  =============================== Coverage summary ===============================
  Statements : 93.48% ( 9732/10411 )
  Branches : 84.7% ( 4440/5242 )
  Functions : 94.23% ( 1780/1889 )
  Lines : 93.53% ( 9693/10363 )
  ================================================================================

  I also looked at a recent build on CI:

  https://buildkite.com/fluid-project/infusion/builds/202#9edb5641-219b-4d63-b82b-a04d6111db91

  Although the report itself is not retained, the overall totals are closer to your result than mine.

  =============================== Coverage summary ===============================
  Statements : 92.95% ( 9701/10437 )
  Branches : 84.08% ( 4378/5207 )
  Functions : 93.71% ( 1773/1892 )
  Lines : 92.99% ( 9664/10392 )
  ================================================================================

  I'll try rerunning in a VM to see if I can track this down further.

• Tony Atkins [RtF] commented 2018-07-23T09:39:55.407-0400

  @@Antranig Basman, the results in a clean VM exactly match my results. I will need to fire up a windows VM to see if I can reproduce the discrepancy there.

• Tony Atkins [RtF] commented 2018-07-23T10:45:14.559-0400

  The results from running in a Windows 10 VM are comparable to what I saw in CI (I suspect there are a few windows-unique branches that are hit, the coverage was even better):

  http://the-t-in-rtf.github.io/coverage-reports/infusion/20180723-windows/

  @@Antranig Basman, I'll need to rely on you for input about when you see this, with which combination of browsers, whether it occurs repeatedly, et cetera.

• Justin Obara commented 2018-07-23T11:08:11.450-0400

  @@Tony Atkins [RtF] and @@Antranig Basman there are some TTS related tests that do not run in certain conditions (i.e. the web speech api isn't supported, the pause/resume tests don't run in Chrome, and in some cases the tests don't run when there isn't a proper audio driver which was happening in browerstack but may also be the case for our CI).

• Tony Atkins [RtF] commented 2018-07-24T04:29:15.476-0400

  Thanks, @@Justin Obara, that's an interesting point. The tests run and report coverage in CI. @@Antranig Basman, if you open the TTS fixtures in a browser on the machine where you saw the problem, do they all run?