这是indexloc提供的服务,不要输入任何密码
Skip to content

chore: optimize termux_step_get_dependencies #24567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 7, 2025

Conversation

twaik
Copy link
Member

@twaik twaik commented May 1, 2025

Removed external commands invocations where possible.
United some conditions under single test invocation
Refactored termux_extract_dep_info to avoid re-sourcing build.sh.
Modified package extraction sequence to avoid storage hit and avoid spawning two more commands for checking if in-tar files are prefixed with ./ or not.

With these changes termux_step_dependencies execution time drops from 12s to 7s during kf6-breeze-icons rebuilding.

Also it reduces ~0.1s-~2.0s per package during archive extracting step (depends on package size), which is good for packages with lots of dependencies (like kf6-breeze-icons).

@twaik twaik requested review from robertkirkman and TomJo2000 May 1, 2025 05:38
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 45445be to fd00a88 Compare May 1, 2025 05:42
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from fd00a88 to 3f47560 Compare May 1, 2025 06:14
Copy link
Contributor

@robertkirkman robertkirkman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it, I can confirm this is faster. I thought that termux_step_get_dependencies() was pretty fast already, but this change can add up to a significant benefit since it runs a lot.

@twaik
Copy link
Member Author

twaik commented May 1, 2025

This change makes it faster mostly for package rebuilding since I modified only this codepath.
But probably we can make regular building with dependency downloading much faster by parallelizing package downloads. I can take a look at this later. But after merging this PR.

@twaik
Copy link
Member Author

twaik commented May 1, 2025

Well, seems like [[ is faster than [. Pure miracle.
... But not much.

$ time for ((i=0; i<1000000; i++)); do :; done
real    0m1.017s
user    0m1.017s
sys     0m0.000s

$ time for ((i=0; i<1000000; i++)); do [ -f /dev/null ]; done
real    0m1.949s
user    0m1.629s
sys     0m0.319s

$ time for ((i=0; i<1000000; i++)); do [[ -f /dev/null ]]; done
real    0m1.389s
user    0m1.075s
sys     0m0.314s

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 3f47560 to f1d793a Compare May 1, 2025 08:46
@twaik
Copy link
Member Author

twaik commented May 1, 2025

Also I placed post-timestamp-creation sleep with check if it is needed or 1 second already passed. Should reduce building of each package by 1 second.

I checked all packages overriding termux_step_get_source and no one of them installs files directly to $TERMUX_PREFIX so it should be fine.

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 2 times, most recently from 3032780 to 3b2e89a Compare May 6, 2025 03:56
@twaik
Copy link
Member Author

twaik commented May 6, 2025

I will merge this in 24 hours if nobody minds or after it gets 2+ more approves.

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 5 times, most recently from 78b1da8 to 6bb8318 Compare May 6, 2025 08:23
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 6bb8318 to acc1065 Compare May 6, 2025 10:19
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 2 times, most recently from da397e2 to c2c9a0d Compare May 6, 2025 19:13
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from c2c9a0d to 4067602 Compare May 6, 2025 19:41
@twaik
Copy link
Member Author

twaik commented May 6, 2025

Ok, I'll merge it in 20 hours if nobody minds or after it gets 2+ more approves.

Copy link
Member

@TomJo2000 TomJo2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to approve this earlier.
Go ahead.

@Maxython
Copy link
Member

Maxython commented May 7, 2025

Ok, I'll merge it in 20 hours if nobody minds or after it gets 2+ more approves.

Wait, let me test how this all works with termux-pacman repos.

…imestamp_file

Removed external commands invocations where possible.
United some conditions under single test invocation.
Refactored termux_extract_dep_info to avoid re-sourcing build.sh.
Modified package extraction sequence to avoid storage hit and avoid spawning
two more commands for checking if in-tar files are prefixed with `./` or not.

Modern `tar` compares timestamps with microsecond precision so we
do not need to sleep for a whole second to create a correct timestamp, 100 ms should be enough
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 4067602 to a5df859 Compare May 7, 2025 11:56
@twaik twaik merged commit 9ad706e into master May 7, 2025
@twaik twaik deleted the improve-step-get-dependencies-speed branch May 7, 2025 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants