If you utilize the
pytest-retry package, tests are spit out with the result
Our temporary workaround is to re-run with
--last-failed, which would properly upload failed test runs to the db + thus give us the flakiness stats we wanted. However it shows the build as
failed, which we don’t necessarily want (the build passed as the last try of the test passed).
It’d be great if Test Analytics has built-in support for
pytest-retry to handle this case + show builds passing, but also give us the instability stats for tests that pass with retries.