Add `pytest-retry` support to Test Analytics

If you utilize the pytest-retry package, tests are spit out with the result Unknown

Our temporary workaround is to re-run with --last-failed, which would properly upload failed test runs to the db + thus give us the flakiness stats we wanted. However it shows the build as failed, which we don’t necessarily want (the build passed as the last try of the test passed).

It’d be great if Test Analytics has built-in support for pytest-retry to handle this case + show builds passing, but also give us the instability stats for tests that pass with retries.

Hello, @mliberato! Welcome to the community!
I’m looking into helping you with this issue.
I’ll get back to you when I’ve got more information.


Hello again, @mliberato! Thank you for your feedback!
At the moment, this is the behavior of Test Analytics that isn’t possible to change. Test Analytics keeps growing and developing, so we’ll keep your suggestion in mind.