Meta: add infrastructure to generate summary.json #208
Conversation
The setup is as follows: 1. We collect all the issue data from the repository through the GitHub API and store it in a JSON file. Storing the input allows us to monitor changes in it for bugs. We'll likely want to change some issues so they are more suitable for consumption by the script. 2. The script takes this data and produces a JSON file with a summarized view of each issue. The bulk of the logic is in processing the body of the issue (the top comment) which is split between before and after the YAML change. The idea is that the JSON file with the summarized view can be consumed and eventually displayed on webkit.org once we're happy with it. It currently has some flaws due to the raw data, but it seems worthwhile to check this in and then improve the raw data to validate the entire process around updating the issue data.
| @@ -0,0 +1,145 @@ | |||
| #!/usr/bin/env python | |||
There was a problem hiding this comment.
python or python3? I'd hope the latter!
There was a problem hiding this comment.
Is it really best practice to put the version number there? It works fine either way.
There was a problem hiding this comment.
Let's assume this is python 3, and we can set a requirements after.
| page += 1 | ||
| continue | ||
| break | ||
| write_json("summary-data.json", data) |
There was a problem hiding this comment.
Do we want to be using summary-data.json in the current working directory, or summary-data.json adjacent to the script?
Do we even want to add an option to change what file it uses? Maybe?
There was a problem hiding this comment.
I don't think we need to account for different files. Supporting invocation from different directories seems like something we could support as a follow-up, though I doubt we'll need it.
There was a problem hiding this comment.
Currently the code resides in the root directory.
It would probably be better in a separate directory.
Specifically when adding a couple of things for managing the tests, etc.
Could we move all of that in a tooling or something.
Also should the edits on summary.py be separated from summary.json and summary-data.json
I would propose we do.
.github/
data/
summary-data.json
summary.json
tooling/
summary.py
tests/
I will be working on testing later. Once the first PR is merged.
It could even be part of a GitHub actions. So that we have a clear separation in between editing the python factory and the data generations. |
One could reasonably imagine the |
|
I like the idea of more automation, but changing Perhaps enforcing |
At least would it be possible to do different PRs for python code and data file generation. 🙏 |
|
I doubt we're going to change the code a whole lot after this lands and a reviewer probably wants to see the changes doing the right thing? Edit: clarified with Karl that all is in order. |
The setup is as follows:
The idea is that the JSON file with the summarized view can be consumed and eventually displayed on webkit.org once we're happy with it.
It currently has some flaws due to the raw data, but it seems worthwhile to check this in and then improve the raw data to validate the entire process around updating the issue data.