The flaky flag appears to be being added to tests that fail on non-master branches, which is to be expected during PR development.
Can flaky detection and adding the label be restricted to failures that only happen on master?
This issue is preventing using the slack notification for new flaky issues.
Hi @IanButterworth,
It’s not possible to configure this yet.
Right now a flaky label will be applied to any test which fails then passes on the same commit. If your CI/Test job is configured to retry the test and it fails and passes (even on a non-master branch) it will be labelled flaky.
We are currently working on functionality that allows you to exclude other branches and expect it to be out this quarter.
Great. Thanks!