+
Skip to content

Provide GNU test comparison comments for PRs in Github Actions. #400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Jun 23, 2024
Merged

Provide GNU test comparison comments for PRs in Github Actions. #400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fbfc295
Clean up useless steps in actions script.
hanbings Jun 7, 2024
15ef45c
Use CI to show the tests of the current changes in the PR.
hanbings Jun 7, 2024
4825f2a
Modify the comment CI trigger.
hanbings Jun 7, 2024
adcf9b0
Modify the comment CI trigger.
hanbings Jun 7, 2024
5d87b00
Revert the formatted release.yml file.
hanbings Jun 7, 2024
2449782
Merge branch 'main' into improve-ci
hanbings Jun 15, 2024
b129636
Fix event name checking.
hanbings Jun 15, 2024
c18b1d3
Replace the function that gets the workflow id.
hanbings Jun 15, 2024
e434615
Add workflow permissions.
hanbings Jun 15, 2024
bc71cdc
Replace the function that gets the workflow id.
hanbings Jun 15, 2024
3e55c4a
Replace the function that gets the workflow id.
hanbings Jun 15, 2024
1ea7285
Add list runs logic.
hanbings Jun 15, 2024
2352552
Add list annotations logic.
hanbings Jun 15, 2024
f6befde
Add list annotations logic.
hanbings Jun 15, 2024
d973ae9
Convert to await code.
hanbings Jun 15, 2024
b9cfef2
Eliminate non-top-level await.
hanbings Jun 15, 2024
87324f9
Change the commit id used by ref.
hanbings Jun 15, 2024
a639db0
Split out synchronous code.
hanbings Jun 15, 2024
d61698e
Split out synchronous code.
hanbings Jun 15, 2024
6c052fe
Fix incorrect references.
hanbings Jun 15, 2024
e22759f
Split the two CI files.
hanbings Jun 23, 2024
27c22d7
Merge branch 'main' into improve-ci
hanbings Jun 23, 2024
3ca10e0
Merge branch 'uutils:main' into refactor-comment-ci
hanbings Jun 23, 2024
7274cf4
Changed comment.yml trigger.
hanbings Jun 23, 2024
418d87f
Changed comment.yml trigger.
hanbings Jun 23, 2024
db1adb2
Merge pull request #2 from hanbings/refactor-comment-ci
hanbings Jun 23, 2024
1cd319d
Merge branch 'main' into improve-ci
hanbings Jun 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,8 @@ jobs:
cargo test --all --no-fail-fast ${{ matrix.cargo_flags }}
env:
CARGO_INCREMENTAL: "0"
RUSTFLAGS: '-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort -Cdebug-assertions=off'
RUSTDOCFLAGS: '-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort -Cdebug-assertions=off'
RUSTFLAGS: "-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort -Cdebug-assertions=off"
RUSTDOCFLAGS: "-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort -Cdebug-assertions=off"

- name: Generate coverage data
id: grcov
Expand Down
70 changes: 70 additions & 0 deletions .github/workflows/comment.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
on:
workflow_run:
workflows: [External testsuites]
types: [completed]

name: Comment Test results on the PR

jobs:
upload-pr-comment:
name: Upload PR comment
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
permissions:
actions: read
pull-requests: write

steps:
- name: List Annotations
uses: actions/github-script@v7
with:
script: |
let runs = await github.rest.checks.listForRef({
owner: context.repo.owner,
repo: context.repo.repo,
ref: '${{ github.event.pull_request.head.sha }}'
});

let names = ['Run GNU findutils tests', 'Run BFS tests'];
let results = [];
runs.data.check_runs.filter(check => names.includes(check.name)).forEach(run => results.push(run));

let annotations = { data: []};
for (let result of results) {
let run = await github.rest.checks.listAnnotations({
owner: context.repo.owner,
repo: context.repo.repo,
check_run_id: result.id
});

run.data.forEach(data => {
annotations.data.push({
run: result.name,
annotation: data
});
});
}

let fs = require('fs');
fs.writeFileSync('${{ github.workspace }}/annotations.json', JSON.stringify(annotations));
- name: Comment on PR
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
let fs = require('fs');
let annotations = JSON.parse(fs.readFileSync('${{ github.workspace }}/annotations.json', 'utf8'));

let annotationContent = annotations
.data
.map(annotation => `${annotation.run}: ${annotation.annotation.message}`)
.join('\n');

console.log(annotationContent);

github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ github.event.number }},
body: 'GNU testsuite comparison:\n```\n' + annotationContent + '```'
});
270 changes: 135 additions & 135 deletions .github/workflows/compat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,143 +7,143 @@ jobs:
name: Run GNU findutils tests
runs-on: ubuntu-latest
steps:
- name: Checkout findutils
uses: actions/checkout@v4
with:
path: findutils
- name: Checkout GNU findutils
uses: actions/checkout@v4
with:
repository: gnu-mirror-unofficial/findutils
path: findutils.gnu
ref: 5768a03ddfb5e18b1682e339d6cdd24ff721c510
submodules: true
- name: Install `rust` toolchain
run: |
## Install `rust` toolchain
rustup toolchain install stable --no-self-update -c rustfmt --profile minimal
rustup default stable
- name: Install dependencies
shell: bash
run: |
# Enable sources & install dependencies
sudo find /etc/apt/sources.list* -type f -exec sed -i 'p; s/^deb /deb-src /' '{}' +
sudo apt-get update
sudo apt-get build-dep findutils
- name: Run GNU tests
shell: bash
run: |
cd findutils
bash util/build-gnu.sh ||:
- name: Extract testing info
shell: bash
run: |
- name: Upload gnu-test-report
uses: actions/upload-artifact@v4
with:
name: gnu-test-report
path: |
findutils.gnu/find/testsuite/*.log
findutils.gnu/xargs/testsuite/*.log
findutils.gnu/tests/**/*.log
- name: Upload gnu-result
uses: actions/upload-artifact@v4
with:
name: gnu-result
path: gnu-result.json
- name: Download the result
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: gnu-result
repo: uutils/findutils
branch: main
path: dl
- name: Download the log
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: gnu-test-report
repo: uutils/findutils
branch: main
path: dl
- name: Compare failing tests against master
shell: bash
run: |
./findutils/util/diff-gnu.sh ./dl ./findutils.gnu
- name: Compare against main results
shell: bash
run: |
mv dl/gnu-result.json latest-gnu-result.json
python findutils/util/compare_gnu_result.py
- name: Checkout findutils
uses: actions/checkout@v4
with:
path: findutils
- name: Checkout GNU findutils
uses: actions/checkout@v4
with:
repository: gnu-mirror-unofficial/findutils
path: findutils.gnu
ref: 5768a03ddfb5e18b1682e339d6cdd24ff721c510
submodules: true
- name: Install `rust` toolchain
run: |
## Install `rust` toolchain
rustup toolchain install stable --no-self-update -c rustfmt --profile minimal
rustup default stable
- name: Install dependencies
shell: bash
run: |
# Enable sources & install dependencies
sudo find /etc/apt/sources.list* -type f -exec sed -i 'p; s/^deb /deb-src /' '{}' +
sudo apt-get update
sudo apt-get build-dep findutils
- name: Run GNU tests
shell: bash
run: |
cd findutils
bash util/build-gnu.sh ||:
- name: Extract testing info
shell: bash
run: |
- name: Upload gnu-test-report
uses: actions/upload-artifact@v4
with:
name: gnu-test-report
path: |
findutils.gnu/find/testsuite/*.log
findutils.gnu/xargs/testsuite/*.log
findutils.gnu/tests/**/*.log
- name: Upload gnu-result
uses: actions/upload-artifact@v4
with:
name: gnu-result
path: gnu-result.json
- name: Download the result
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: gnu-result
repo: uutils/findutils
branch: main
path: dl
- name: Download the log
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: gnu-test-report
repo: uutils/findutils
branch: main
path: dl
- name: Compare failing tests against master
shell: bash
run: |
./findutils/util/diff-gnu.sh ./dl ./findutils.gnu
- name: Compare against main results
shell: bash
run: |
mv dl/gnu-result.json latest-gnu-result.json
python findutils/util/compare_gnu_result.py

bfs-tests:
name: Run BFS tests
runs-on: ubuntu-latest
steps:
- name: Checkout findutils
uses: actions/checkout@v4
with:
path: findutils
- name: Checkout BFS
uses: actions/checkout@v4
with:
repository: tavianator/bfs
path: bfs
ref: '3.1.3'
- name: Install `rust` toolchain
run: |
## Install `rust` toolchain
rustup toolchain install stable --no-self-update -c rustfmt --profile minimal
rustup default stable
- name: Install dependencies
shell: bash
run: |
# Enable sources & install dependencies
sudo find /etc/apt/sources.list* -type f -exec sed -i 'p; s/^deb /deb-src /' '{}' +
sudo apt-get update
sudo apt-get build-dep bfs
- name: Run BFS tests
shell: bash
run: |
cd findutils
bash util/build-bfs.sh ||:
- name: Upload bfs-test-report
uses: actions/upload-artifact@v4
with:
name: bfs-test-report
path: bfs/tests.log
- name: Upload bfs-result
uses: actions/upload-artifact@v4
with:
name: bfs-result
path: bfs-result.json
- name: Download the result
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: bfs-result
repo: uutils/findutils
branch: main
path: dl
- name: Download the log
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: bfs-test-report
repo: uutils/findutils
branch: main
path: dl
- name: Compare failing tests against main
shell: bash
run: |
./findutils/util/diff-bfs.sh dl/tests.log bfs/tests.log
- name: Compare against main results
shell: bash
run: |
mv dl/bfs-result.json latest-bfs-result.json
python findutils/util/compare_bfs_result.py
- name: Checkout findutils
uses: actions/checkout@v4
with:
path: findutils
- name: Checkout BFS
uses: actions/checkout@v4
with:
repository: tavianator/bfs
path: bfs
ref: "3.1.3"
- name: Install `rust` toolchain
run: |
## Install `rust` toolchain
rustup toolchain install stable --no-self-update -c rustfmt --profile minimal
rustup default stable
- name: Install dependencies
shell: bash
run: |
# Enable sources & install dependencies
sudo find /etc/apt/sources.list* -type f -exec sed -i 'p; s/^deb /deb-src /' '{}' +
sudo apt-get update
sudo apt-get build-dep bfs
- name: Run BFS tests
shell: bash
run: |
cd findutils
bash util/build-bfs.sh ||:
- name: Upload bfs-test-report
uses: actions/upload-artifact@v4
with:
name: bfs-test-report
path: bfs/tests.log
- name: Upload bfs-result
uses: actions/upload-artifact@v4
with:
name: bfs-result
path: bfs-result.json
- name: Download the result
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: bfs-result
repo: uutils/findutils
branch: main
path: dl
- name: Download the log
uses: dawidd6/action-download-artifact@v6
with:
workflow: compat.yml
workflow_conclusion: completed
name: bfs-test-report
repo: uutils/findutils
branch: main
path: dl
- name: Compare failing tests against main
shell: bash
run: |
./findutils/util/diff-bfs.sh dl/tests.log bfs/tests.log
- name: Compare against main results
shell: bash
run: |
mv dl/bfs-result.json latest-bfs-result.json
python findutils/util/compare_bfs_result.py
Loading
点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载