这是indexloc提供的服务，不要输入任何密码
Skip to content

chore: optimize termux_step_get_dependencies #24567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 7, 2025
Merged

chore: optimize termux_step_get_dependencies #24567

merged 1 commit into from
May 7, 2025

Conversation

@twaik
Copy link
Member

@twaik twaik commented May 1, 2025 •
edited
Loading

Removed external commands invocations where possible.
United some conditions under single test invocation
Refactored termux_extract_dep_info to avoid re-sourcing build.sh.
Modified package extraction sequence to avoid storage hit and avoid spawning two more commands for checking if in-tar files are prefixed with ./ or not.

With these changes termux_step_dependencies execution time drops from 12s to 7s during kf6-breeze-icons rebuilding.

Also it reduces ~0.1s-~2.0s per package during archive extracting step (depends on package size), which is good for packages with lots of dependencies (like kf6-breeze-icons).

@twaik twaik requested review from TomJo2000 and robertkirkman May 1, 2025 05:38
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 45445be to fd00a88 Compare May 1, 2025 05:42
TomJo2000
TomJo2000 reviewed May 1, 2025
View reviewed changes
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from fd00a88 to 3f47560 Compare May 1, 2025 06:14
robertkirkman
robertkirkman approved these changes May 1, 2025
View reviewed changes
Copy link
Member

@robertkirkman robertkirkman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it, I can confirm this is faster. I thought that termux_step_get_dependencies() was pretty fast already, but this change can add up to a significant benefit since it runs a lot.

@twaik
Copy link
Member Author

twaik commented May 1, 2025 •
edited
Loading

This change makes it faster mostly for package rebuilding since I modified only this codepath.
But probably we can make regular building with dependency downloading much faster by parallelizing package downloads. I can take a look at this later. But after merging this PR.

@twaik
Copy link
Member Author

twaik commented May 1, 2025

Well, seems like [[ is faster than [. Pure miracle.
... But not much.

$ time for ((i=0; i<1000000; i++)); do :; done
real    0m1.017s
user    0m1.017s
sys     0m0.000s

$ time for ((i=0; i<1000000; i++)); do [ -f /dev/null ]; done
real    0m1.949s
user    0m1.629s
sys     0m0.319s

$ time for ((i=0; i<1000000; i++)); do [[ -f /dev/null ]]; done
real    0m1.389s
user    0m1.075s
sys     0m0.314s

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 3f47560 to f1d793a Compare May 1, 2025 08:46
@twaik
Copy link
Member Author

twaik commented May 1, 2025 •
edited
Loading

Also I placed post-timestamp-creation sleep with check if it is needed or 1 second already passed. Should reduce building of each package by 1 second.

I checked all packages overriding termux_step_get_source and no one of them installs files directly to $TERMUX_PREFIX so it should be fine.

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 2 times, most recently from 3032780 to 3b2e89a Compare May 6, 2025 03:56
@twaik
Copy link
Member Author

twaik commented May 6, 2025

I will merge this in 24 hours if nobody minds or after it gets 2+ more approves.

@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 5 times, most recently from 78b1da8 to 6bb8318 Compare May 6, 2025 08:23
robertkirkman
robertkirkman reviewed May 6, 2025
View reviewed changes
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 6bb8318 to acc1065 Compare May 6, 2025 10:19
TomJo2000
TomJo2000 reviewed May 6, 2025
View reviewed changes
Maxython
Maxython reviewed May 6, 2025
View reviewed changes
robertkirkman
robertkirkman reviewed May 6, 2025
View reviewed changes
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch 2 times, most recently from da397e2 to c2c9a0d Compare May 6, 2025 19:13
robertkirkman
robertkirkman reviewed May 6, 2025
View reviewed changes
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from c2c9a0d to 4067602 Compare May 6, 2025 19:41
@twaik
Copy link
Member Author

twaik commented May 6, 2025

Ok, I'll merge it in 20 hours if nobody minds or after it gets 2+ more approves.

TomJo2000
TomJo2000 approved these changes May 7, 2025
View reviewed changes
Copy link
Member

@TomJo2000 TomJo2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to approve this earlier.
Go ahead.

@Maxython
Copy link
Member

Maxython commented May 7, 2025

Ok, I'll merge it in 20 hours if nobody minds or after it gets 2+ more approves.

Wait, let me test how this all works with termux-pacman repos.

Maxython
Maxython requested changes May 7, 2025
View reviewed changes
@twaik
chore: optimize termux_step_get_dependencies and termux_step_create_t…
a5df859 
…imestamp_file

Removed external commands invocations where possible.
United some conditions under single test invocation.
Refactored termux_extract_dep_info to avoid re-sourcing build.sh.
Modified package extraction sequence to avoid storage hit and avoid spawning
two more commands for checking if in-tar files are prefixed with `./` or not.

Modern `tar` compares timestamps with microsecond precision so we
do not need to sleep for a whole second to create a correct timestamp, 100 ms should be enough
@twaik twaik force-pushed the improve-step-get-dependencies-speed branch from 4067602 to a5df859 Compare May 7, 2025 11:56
@twaik twaik merged commit 9ad706e into master May 7, 2025
@twaik twaik deleted the improve-step-get-dependencies-speed branch May 7, 2025 15:15
@robertkirkman robertkirkman mentioned this pull request May 13, 2025
Merged
@robertkirkman robertkirkman mentioned this pull request Aug 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robertkirkman robertkirkman robertkirkman approved these changes

@TomJo2000 TomJo2000 TomJo2000 approved these changes

@Maxython Maxython Maxython approved these changes

@thunder-coding thunder-coding thunder-coding approved these changes

@Grimler91 Grimler91 Awaiting requested review from Grimler91 Grimler91 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@twaik @Maxython @robertkirkman @TomJo2000 @thunder-coding