Dev: Ensure tests are run in random #9968

PHP Version

8.5.0

CodeIgniter4 Version

4.7.1-dev

Per ChatGPT, tests need to run in random to enjoy the following benefits:

✔️ It reveals hidden dependencies
✔️ Forces true test isolation
✔️ Prevents flaky, misleading test results
✔️ Makes your test suite production-realistic

This issue has been initially tackled by #7717, but due to other complications it didn't fare well. As this is not a fairly easy task, I'm proposing we do it on pieces. Currently, our test suite runs in default order but should be random at some point (i.e., executionOrder="random").

Here's the plan, instead of one big PR changing all tests, we can do it by component (HTTP, Cache, Database, etc.). Then, after all tests run green on randomised testing, we set executionOrder in phpunit.xml to "random".

Please reference this issue for any PRs addressing pieces of this.

• API: refactor: make random-order API tests deterministic #9983
• AutoReview: test: indicate components that already pass random execution tests #10073
• Autoloader: test: indicate components that already pass random execution tests #10073
• Cache
• CLI:
  • refactor: make random-order CLI tests deterministic #9998
  • refactor: fix dependency on test execution order #10014
• Commands
• Config
• Cookie: test: indicate components that already pass random execution tests #10073
• Database
• DataCaster
• DataConverter
• Debug
• Email
• Encryption
• Entity
• Events: test: indicate components that already pass random execution tests #10073
• Files: test: indicate components that already pass random execution tests #10073
• Filters: test: resolve service registry pollution in CSRFTest #10373
• Format: test: indicate components that already pass random execution tests #10073
• Helpers
• Honeypot
• HotReloader: test: indicate components that already pass random execution tests #10073
• HTTP: fix: resolve global state pollution causing random-order test failures in HTTP suite #10372
• I18n
• Images
• Language: test: indicate components that already pass random execution tests #10073
• Log: test: indicate components that already pass random execution tests #10073
• Pager: test: indicate components that already pass random execution tests #10073
• Publisher: test: indicate components that already pass random execution tests #10073
• RESTful: test: indicate components that already pass random execution tests #10073
• Router
  • test: fix determinism of CacheTest #9967
• Security
  • refactor: fix Security test fail by itself #9969
  • test: indicate components that already pass random execution tests #10073
• Session
• Test
• Throttle: test: indicate components that already pass random execution tests #10073
• Typography: test: indicate components that already pass random execution tests #10073
• Validation
• View: test: indicate components that already pass random execution tests #10073