This change looks good to me. Just a note that marking a test as flaky means that bazel will run it three times: if the test fails all three times then it will be still be marked as failed. So marking it as flaky does not completely prevent from blocking the CI (if your test fails really too often), it's only useful if your tests works often enough :)

This change looks good to me. Just a note that marking a test as flaky means that bazel will run it three times: if the test fails all three times then it will be still be marked as failed. So marking it as flaky does not completely prevent from blocking the CI (if your test fails really too often), it's only useful if your tests works often enough :)

Thanks Amaury - I was not aware of this. Is there a way of not letting CI fail when one of these tests fail and instead just report a warning?

Not at the moment. If there really is a need for it, we could create an attribute to mark some tests as "do_not_fail_ci". This is actually an interesting discussion, to have some blocking and non-blocking tests. Though the downside is that people might ignore non-blocking tests entirely...

Alright, thanks! I've removed the flaky flag from the tests.

Yes, such a "do_not_fail_ci" could be useful. I agree that people might end up just not caring. Maybe it might be possible that a code owner gets notified when such a "do_not_fail_ci" test fails?

@pamaury if this PR is OK for you, could you please approve it?

Removing the flaky flag is fine as the pentests are now stable.

Successfully created backport PR for earlgrey_1.0.0:

• Cherry-pick to earlgrey_1.0.0: [pentest] Add tests to CI #27677