Following up from the declined T225871: Selenium and PHPUnit: Stop execution on failure and tangential to T248531: Abort a Zuul pipeline when one job completed with failures (change zuul scheduler's failure check from areAllJobsComplete to didAnyJobFail):
- have a web app on toolforge, i.e. early-warning-bot.toolforge.org https://wikitech.wikimedia.org/wiki/Tool:Early_warning_bot
- patch Quibble so that when a process ends with exit code other than 0, it POSTs some data about the job (patch details) to the web app
- the web app adds a comment and a Verified -1 to the patch
- the comment should include a link to the Jenkins build URL for the failing job
- Quibble could capture a formatted version of the failing command output, and include this in the comment that the web app leaves in Gerrit. That is related to T209149: Have linters/tests results show up as comments in files on gerrit. We'd need a trimmed version of this for PHPUnit which currently dumps a lot of text. See T331061: Capture output from failed command and transmit to earlywarningbot
Why?
- developers receive pings early via email/IRC that a patch has failures. Yes, you can watch Zuul, but most people don't know to do that, and it doesn't notify you that a job has failed.
- Early feedback on a failed patch allows the developer to 1) fix the patch before switching their context to other things or 2) in a backport situation, know that they need to restart the jobs