Add post-merge analysis CI workflow #138013
Merged
+319
−10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
rustbot
added
A-testsuite
This comment has been minimized.
This comment has been minimized.
|
☔ The latest upstream changes (presumably #138021) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
c40b798 to
6ec42d7
Compare
|
Rebased. @rustbot ready |
This comment has been minimized.
This comment has been minimized.
|
☔ The latest upstream changes (presumably #138058) made this pull request unmergeable. Please resolve the merge conflicts. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
6ec42d7 to
f04a2cc
Compare
|
Rebased. |
Kobzol
force-pushed
the
ci-post-merge-analysis
branch
from
f04a2cc to
d5da6b7
Compare
marcoieni
reviewed
Mar 7, 2025
|
|
||
| for job in &job_db.auto_jobs { | ||
| eprintln!("Downloading metrics of job {}", job.name); | ||
| let metrics_parent = match download_job_metrics(&job.name, parent) { |
There was a problem hiding this comment.
you could simplify this match with something like this:
download_job_metrics(&job.name, parent).inspect_err( eprintln!()).ok()
Comment on lines
+41
to
+43
| r#"Did not find metrics for job `{}` at `{}`: {error:?}. | ||
| Maybe it was newly added?"#, | ||
| job.name, parent |
There was a problem hiding this comment.
Suggested change
| r#"Did not find metrics for job `{}` at `{}`: {error:?}. | |
| Maybe it was newly added?"#, | |
| job.name, parent | |
| r#"Did not find metrics for job `{}` at `{parent}`: {error:?}. | |
| Maybe it was newly added?"#, | |
| job.name |
|
@bors r+ |
bors
added
S-waiting-on-bors
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup of 12 pull requests Successful merges: - rust-lang#137337 (Add verbatim linker to AIXLinker) - rust-lang#137363 (compiler: factor Windows x86-32 ABI impl into its own file) - rust-lang#137537 (Prevent `rmake.rs` from using unstable features, and fix 3 run-make tests that currently do) - rust-lang#137606 (add a "future" edition) - rust-lang#137957 (Remove i586-pc-windows-msvc) - rust-lang#138000 (atomic: clarify that failing conditional RMW operations are not 'writes') - rust-lang#138013 (Add post-merge analysis CI workflow) - rust-lang#138033 (rustdoc: Add attribute-related tests for rustdoc JSON.) - rust-lang#138137 (setTargetTriple now accepts Triple rather than string) - rust-lang#138173 (Delay bug for negative auto trait rather than ICEing) - rust-lang#138184 (Allow anyone to relabel `CI-spurious-*`) - rust-lang#138187 (remove clones) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Mar 8, 2025
Rollup merge of rust-lang#138013 - Kobzol:ci-post-merge-analysis, r=marcoieni Add post-merge analysis CI workflow This PR adds a post-merge analysis workflow, which was discussed [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Reporting.20test.20suite.20statistics.20after.20merge). The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations. It can be tested locally e.g. using this: ``` cargo run --release --manifest-path src/ci/citool/Cargo.toml post-merge-report 3cb0272 fd17dea ``` This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large). CC `@jieyouxu` r? `@marcoieni`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a post-merge analysis workflow, which was discussed here.
The workflow currently analyzes test suite results from bootstrap metrics. It downloads metrics for all known jobs in the parent and current (HEAD) commit, compares them and prints a truncated diff. It then posts this diff to the merged PR as a comment. Later I also want to add other statistics to the analysis, e.g. changes in CI/bootstrap step durations.
It can be tested locally e.g. using this:
This uses a slightly older commit as a parent, to have more results in the diff (normally the diff won't be so large).
CC @jieyouxu
r? @marcoieni