Add the dependency matrix #55
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Every linux configuration takes ~30 minutes, which is ~1.5% of OpenDP's 2000 monthly free minutes. Running a mac combination takes ~30*10 (minute multiplier) = 200 minutes, so 15% of OpenDP's free minutes.
I would like to run mac combinations, since we already know we missed a dependency problem in the past by not running them, but I think we can't afford any. Instead, we should probably just try to make sure I (or someone else with a mac) runs the dependency matrix locally before any release.
I had originally proposed running every python version we support, but if we assume 5 python versions that means every dependency matrix run will use 15% of OpenDP's monthly CI time. Sticking to just the newest and oldest python versions will keep us to 6%, which seems less likely to get us into trouble. That still means that every new python version we support will be in the dependency matrix for some amount of time (just not the entire time it's supported), which I think should catch most issues that are likely to arise.