chore(scripts/run-docker): implement dry run simulation to skip launching container if no packages would have been built #24168

robertkirkman · 2025-04-06T11:51:06Z

Fixes [CI]: worker should perform package skip BEFORE docker image is being pulled. #24116
adds a new script to scripts/bin, build-package-dry-run-simulation.sh, which is used to test whether the entire build-package.sh command would have both proceeded beyond this point,

termux-packages/scripts/build/termux_step_start_build.sh

Lines 13 to 16 in 040710f

    
           if [ -n "${TERMUX_PKG_EXCLUDED_ARCHES:=""}" ] && [ "$TERMUX_PKG_EXCLUDED_ARCHES" != "${TERMUX_PKG_EXCLUDED_ARCHES/$TERMUX_ARCH/}" ]; then 
        
           	echo "Skipping building $TERMUX_PKG_NAME for arch $TERMUX_ARCH" 
        
           	exit 0 
        
           fi

and also proceeded beyond this point,

termux-packages/scripts/build/termux_step_start_build.sh

Lines 42 to 47 in 040710f

    
           if [ "$TERMUX_PKG_HAS_DEBUG" = "true" ]; then 
        
           	DEBUG="-dbg" 
        
           else 
        
           	echo "Skipping building debug build for $TERMUX_PKG_NAME" 
        
           	exit 0 
        
           fi

or if it would have skipped all packages in the list of packages to build without building anything.
if the build would have continued or an unknown situation occurs, zero is returned, but if all packages in the list of packages to build would have been skipped, 85 (C_EX__NOOP) is returned.
if an error handled by this script, such as a syntax error in the -a argument or the package name argument list occurs, a similar error message to what build-package.sh would have printed is shown, and 1 is returned.
in run-docker.sh, the command passed to run-docker.sh is run in build-package-dry-run-simulation.sh before doing anything else only if the command passed has build-package.sh in its first argument, and only continues if a zero value is returned by build-package-dry-run-simulation.sh, which indicates either that at least one package will be built, or that the passed command is empty or not build-package.sh.

robertkirkman · 2025-04-06T11:57:35Z

I am not sure whether this is fully compatible with the case of running the scripts/run-docker.sh command on a MacOS or BSD computer that has a VM-based docker command installed. This scripts/run-docker.sh has been tested on a GNU/Linux computer with a traditional native-container docker command installed. If someone who has MacOS or BSD regularly uses scripts/run-docker.sh locally on their computer, it might be convenient if they could help me test this to implement it fully for their device.

twaik

LGTM.

scripts/run-docker.sh

agnostic-apollo · 2025-04-06T13:50:49Z

And you likely meant --dry-run instead of --dummy.

robertkirkman · 2025-04-07T00:24:57Z

And you likely meant --dry-run instead of --dummy.

I do like that name better for this also. I will change it to that.

scripts/build-package-dry-run-simulation.sh

scripts/run-docker.sh

robertkirkman · 2025-07-02T12:41:45Z

Ok, now I've made sort of another version that's based on agnostic-apollo's request to try using grep instead of sourcing build.sh files.

I believe I have incorporated most of the requests into this version, and while it probably needs more changes, I would be interested in knowing whether this version is changing in the right direction.

scripts/run-docker.sh

scripts/bin/build-package-dry-run-simulation.sh

robertkirkman · 2025-07-05T01:13:57Z

.github/workflows/packages.yml

      if: ${{ steps.build-info.outputs.skip-building != 'true' }}
      env:
        DOCKER_BUILD: ${{ steps.build-info.outputs.docker-build }}
+        TERMUX_DOCKER__CONTAINER_EXEC_COMMAND__PRE_CHECK_IF_WILL_BUILD_PACKAGES: "true"


I assume that this would be the appropriate place to enable this in CI.

I have tested this on my own GitHub Actions in this run, on the ecl package, which has TERMUX_PKG_EXCLUDED_ARCHES="i686, x86_64":

https://github.com/robertkirkman/termux-packages/actions/runs/16082984132/job/45390315328

It is working. What happens in practice is, for the i686 and x86_64 jobs, the "build" workflow step very quickly completes without building anything, but showing a green success mark, not red failed or gray skipped, and an artifact does upload. However, what is in the artifact is just nothing except a .placeholder file, which is empty, and a built_termux-main_packages.txt file, which contains the list of packages that would have been built if the build were not skipped, in this test "ecl".

@twaik do you think this is proper behavior to happen, or should it be adjusted to be different in any way?

scripts/bin/build-package-dry-run-simulation.sh

agnostic-apollo · 2025-07-05T02:27:16Z

Looks fine from my side otherwise, I'll leave any testing to you.

robertkirkman · 2025-07-05T14:38:48Z

since twaik opened the original issue, I would be interested in review from twaik about the current version before merging, because the version originally approved has changed so completely that it's not recognizable anymore

…hing container if no packages would have been built - Fixes termux#24116 - adds a new script to `scripts/bin`, `build-package-dry-run-simulation.sh`, which is used to test whether the entire `build-package.sh` command would have both proceeded beyond this point, https://github.com/termux/termux-packages/blob/040710f4927fc42cb74a85911e42da0a3f2e1f39/scripts/build/termux_step_start_build.sh#L13-L16 - and also proceeded beyond this point, https://github.com/termux/termux-packages/blob/040710f4927fc42cb74a85911e42da0a3f2e1f39/scripts/build/termux_step_start_build.sh#L42-L47 - or if it would have skipped all packages in the list of packages to build without building anything. - if the build would have continued or an unknown situation occurs, zero is returned, but if all packages in the list of packages to build would have been skipped, 85 (`C_EX__NOOP`) is returned. - if an error handled by this script, such as a syntax error in the `-a` argument or the package name argument list occurs, a similar error message to what `build-package.sh` would have printed is shown, and 1 is returned. - in `run-docker.sh`, the command passed to `run-docker.sh` is run in `build-package-dry-run-simulation.sh` before doing anything else only if the command passed has `build-package.sh` in its first argument, and only continues if a zero value is returned by `build-package-dry-run-simulation.sh`, which indicates either that at least one package will be built, or that the passed command is empty or not `build-package.sh`.

robertkirkman · 2025-07-17T11:07:43Z

It seems like twaik might be taking a break, and I believe this is ready, so I will merge this in 24 hours if no problems are found.

robertkirkman requested a review from Grimler91 as a code owner April 6, 2025 11:51

twaik approved these changes Apr 6, 2025

View reviewed changes

agnostic-apollo reviewed Apr 6, 2025

View reviewed changes

scripts/run-docker.sh Outdated Show resolved Hide resolved

robertkirkman marked this pull request as draft April 7, 2025 00:25

robertkirkman force-pushed the dummy-build branch from 95c0d08 to e804ff8 Compare April 7, 2025 02:45

robertkirkman changed the title ~~chore(scripts/run-docker): implement dummy build to skip launching container if no packages would have been built~~ chore(scripts/run-docker): implement dry run to skip launching container if no packages would have been built Apr 7, 2025

robertkirkman force-pushed the dummy-build branch 4 times, most recently from 867df14 to 8b4db25 Compare April 9, 2025 13:42

robertkirkman changed the title ~~chore(scripts/run-docker): implement dry run to skip launching container if no packages would have been built~~ chore(scripts/run-docker): implement dry run simulation to skip launching container if no packages would have been built Apr 9, 2025

robertkirkman force-pushed the dummy-build branch from 8b4db25 to de575be Compare April 10, 2025 01:47

robertkirkman marked this pull request as ready for review April 10, 2025 02:03