tree 72d400bf364a39f223f29552441f9d80e75bfdb4
parent 4928cc8d516868267da4be9a5b9a5f7856dd0b65
author Daniel Smith <56164590+DanielRyanSmith@users.noreply.github.com> 1641943574 -0800
committer GitHub <noreply@github.com> 1641943574 -0800
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJh3hIWCRBK7hj4Ov3rIwAApgwIAIipnmFaqqEwtuTWvOgbPxXT
 ubacv4g3PxYtwZc3gVff7WZSz79BUr/f5L+8Bd+f2918X7q+UlWY8gbqpsfhs09U
 Vbp815v6xpaUZyTrsO0W+GzzFDeGQwXVQpuS3aOhzybvEr9GJqk7PokHmf+dZxQW
 6OPI3YbNKhweho+DYUY/uYY3ToabkamNoRSDOHFvUU54ix/mqSkHf+9WjgFTPsRD
 MvDSynYQwfBB3+0x6fG3OiPRPJ9mouvbtTVbW61skJpjGgzTaqjPFysPlFbdMbic
 miY4jfX3ReKc3iW+n9aCQqbopNSbcSNyXQ0Ao4eKt8HGp7whBaADDEOLMvaG5tk=
 =hjh8
 -----END PGP SIGNATURE-----
 

7660 Avoid CI stability checks timing out (#32202)

* allow stability checks to avoid TC timeout by checking times in between repeat runs

* fix flake8 issue

* remove empty flag to trigger stability checks

* some commenting explaining how iterations are being tracked

* add --repeat-max-time flag

* better descriptors for announcing results

If the repeat runs stop early to avoid, timeout, a message will be written in the results explaining so.

* cast max_time to timedelta

* correct syntax for kwargs

* specify kwargs source for run_test_iteration

* remove empty css flags tag to trigger stability checks

* Add clause to handle an ineffective number of iterations completing

* replace unassociated change used to trigger stability checks

* Implement changes suggested by @stephenmcgruer

* Add only necessary formatting changes to stability

* wptrunner reformatting changes suggested by @jgraham

* functional changes to stability tests suggested by @jgraham

* flake8 changes "line break before binary operator"

* change run_tests to return counts object

* ensure run_tests return values are properly accessed

Now that wptrunner's run_tests returns more than 1 value, the return type will be a tuple for the older variables that expect only 1 value. We need to ensure that we pull the expected first value (boolean) out of that tuple.

* run_tests has consistent return values even in fail cases

wptrunner's run_tests would return a tuple only if not issues arose while running, and would return only a boolean in the case of some expected issue. Now a tuple is returned in all instances as expected.

* Return full counts in TestStatus class

run_tests will now return a new TestStatus object rather than returning only the number of iterations run. This will allow for more robust statistics to be shown in the future.

* small formatting changes

reducing some comments and logs to taking less vertical space.

* small wording change

TestStatus docstring

* Replace counts with TestStatus object

forego the use of defaultdict counts keeping track of test info/results and instead use the custom class TestStatus.

* convert some log strings to f-strings