Libxml2 rebuilds: Part 2 - Small packages #24062

TomJo2000 · 2025-04-01T02:36:19Z

robertkirkman · 2025-04-01T04:06:24Z

Downloading https://www.aleksey.com/xmlsec/download/xmlsec1-1.3.7.tar.gz curl: (22) The requested URL returned error: 403

It seems like it was a temporary server failure, since that file can be downloaded by my connection at the moment.

robertkirkman · 2025-04-02T04:46:28Z

All packages in this branch are now building successfully.

rdrview: calls the function xmlShellPrintNode() multiple times, which has been removed in libxml2 2.14. I added a basic reimplementation of it in a patch as a polyfill by copying and pasting some of its definition from libxml2 2.13, when I test it (at runtime) it seems to work.
xmlstarlet: libxml2 now defines ATTRIBUTE_UNUSED in some sort of a different way, which conflicts with the definition xmlstarlet expects. Redefining ATTRIBUTE_UNUSED back to what it probably was before in xmlstarlet's header works to fix the build error.

TomJo2000 · 2025-04-02T05:37:01Z

Downloading https://www.aleksey.com/xmlsec/download/xmlsec1-1.3.7.tar.gz curl: (22) The requested URL returned error: 403

It seems like it was a temporary server failure, since that file can be downloaded by my connection at the moment.

Seems to be the case, but since it's still failing on the CI I'll pull xmlsec out of this batch with the next rebase of this branch.

robertkirkman · 2025-04-02T05:38:29Z

It seems like https://www.aleksey.com/xmlsec/download/xmlsec1-1.3.7.tar.gz has a problem with intermittent service interruptions, since it keeps having error 403 only sometimes. That means that there is a risk of it having error 403 during the post-merge build and failing to upload to the package host. I am not sure what to do in this situation.

TomJo2000 · 2025-04-02T05:42:41Z

It seems like https://www.aleksey.com/xmlsec/download/xmlsec1-1.3.7.tar.gz has a problem with intermittent service interruptions, since it keeps having error 403 only sometimes. That means that there is a risk of it having error 403 during the post-merge build and failing to upload to the package host. I am not sure what to do in this situation.

That's why I'd like to pull it out into a separate PR and merge it last.

robertkirkman · 2025-04-09T00:16:03Z

error: invalid command 'bdist_wheel'

This did not happen to me when I tested building it on april 1,
I will locally pull changes from master and build them again to try to reproduce it.

TomJo2000 · 2025-04-09T00:20:15Z

This did not happen to me when I tested building it on april 1, I will locally pull changes from master and build them again to try to reproduce it.

Just saw this now, was busy porting patches for Apt 3.0.0 ([DO NOT MERGE] [TRACKER] Apt 3.1.x #24212)
Thanks for looking into it.

robertkirkman · 2025-04-09T02:08:21Z

I do not know why the error happened yet, it is very strange because, this commit built successfully on April 5 in CI,

94e1ca3

but when I use git to check out that commit and then build python-lxml in a clean docker container, the error still happens. This indicates that the change which causes the error is either something that is not tracked by git in this repository, or is something that only happens when building locally, not in CI.

I can see that one difference is that the version of the python wheel package now being used by the termux-packages build system is now version 0.46+, while last week it was 0.45.1,

and in the changelog for wheel on PyPi version 0.46, there is this message:

https://github.com/pypa/wheel/releases/tag/0.46.0

"Removed the bdist_wheel setuptools command implementation and entry point. The wheel.bdist_wheel module is now just an alias to setuptools.command.bdist_wheel, emitting a deprecation warning on import."

This is pretty confusing since this is something newly happening and maybe not very many people are using wheel version 0.46+ yet, but it looks like this update of wheel is probably related to this error.

I will try to figure out if there is any way to resolve this error while using the new release of wheel and if so, then what the best way to change the termux-packages build system or python-lxml to be compatible with it might be.

robertkirkman · 2025-04-09T02:17:01Z

According to the discussion of others who suddenly also have the same error in this issue,

v0.46.0 Breaking CI build pipelines pypa/wheel#660

it looks like we might need to bump our setuptools explicit version here,

termux-packages/scripts/build/setup/termux_setup_python_pip.sh

Lines 63 to 64 in 1217e5f

    
           build-pip install 'setuptools==67.6.1' wheel 
        
           cross-pip install 'setuptools==67.6.1' wheel

to a version that is at least 70 or newer to be updated to have compatibility with wheel version 0.46 or newer.

This would also be a good opportunity, in my opinion, to apply 763ef01 to TERMUX_ON_DEVICE_BUILD=true mode while working on this, since it was never applied to TERMUX_ON_DEVICE_BUILD=true mode, which would cause any modifications to termux_setup_python_pip() that do not correctly update the TERMUX_ON_DEVICE_BUILD=true codepath to continue to fail tests in that mode. I had been planning to do that for a long time but I did not get to it yet.

TomJo2000 · 2025-04-09T02:36:52Z

Sounds good.
I'll leave that to you.
I can pull python-lxml out of this batch so this can move ahead in the meantime tomorrow.

- `wheel` version 0.46+ now requires `setuptools` version 70 or newer - discussed upstream here: pypa/wheel#660 - discovered in termux-packages here: termux#24062 (comment) - I have confirmed that this change works to fix the build of `python-lxml` **both** - **in `TERMUX_ON_DEVICE_BUILD=false` mode (broken for 1 week)**, - fixes `error: invalid command 'bdist_wheel'` - **and also in `TERMUX_ON_DEVICE_BUILD=true` mode (broken for 6 months)**. - fixes `ModuleNotFoundError: No module named 'setuptools'` - It seems appropriate to remove the `if [ "${TERMUX_PYTHON_VERSION#*.}" -lt "12" ]` block because there is no longer any `python3.11` so it does not seem like that code is reachable anymore.

- `wheel` version 0.46+ now requires `setuptools` version 70 or newer - discussed upstream here: pypa/wheel#660 - discovered in termux-packages here: termux#24062 (comment) - I have confirmed that this change works to fix the build of `python-lxml` **both** - **in `TERMUX_ON_DEVICE_BUILD=false` mode (broken for 1 week)**, - fixes `error: invalid command 'bdist_wheel'` - **and also in `TERMUX_ON_DEVICE_BUILD=true` mode (broken for 6 months)**. - fixes `ModuleNotFoundError: No module named 'setuptools'` - It seems appropriate to remove the `if [ "${TERMUX_PYTHON_VERSION#*.}" -lt "12" ]` block because there is no longer any `python3.11` so it does not seem like that code is reachable anymore. - `python-torchvision` needed ajustment of its custom `CFLAGS` to be `CXXFLAGS` instead, because the newer `setuptools` is now compiling it using `aarch64-linux-android-clang++` instead of `aarch64-linux-android-clang`. - `python-pip` and `python-pillow` both had locked `setuptools` in their `TERMUX_PKG_PYTHON_COMMON_DEPS` at versions slightly newer than the old global version, now that the global version is newer than either of those, they do not seem to need that anymore.

robertkirkman · 2025-04-12T00:33:47Z

It will fail again I believe because this isn't merged yet,

bump(scripts/build/setup/termux_setup_python_pip): setuptools 78.1.0, wheel 0.46.1 #24213

sorry about that, if you want I can merge it now so you can build this.

TomJo2000 · 2025-04-12T00:52:51Z

It will fail again I believe because this isn't merged yet,

bump(scripts/build/setup/termux_setup_python_pip): setuptools 78.1.0 #24213

sorry about that, if you want I can merge it now so you can build this.

If #24213 is ready for merge I'd like to get the libxml2 rebuilds wrapped up.

robertkirkman · 2025-04-12T01:42:48Z

What's weird now is that it stopped failing without the PR being merged,

I think this might have happened because wheel 0.46.1 has now been "yanked" on PyPi, which has externally manipulated the termux-packages termux_setup_python_pip() function such that it now only installs wheel 0.45.1 again instead of wheel 0.46.1 like it did a few days ago.

https://pypi.org/project/wheel/#history

The reason given in the description there is "Causes CI failures where setuptools is pinned to an old version".

Because of that, I guess the PR I opened doesn't necessarily need to be merged yet, so you can continue forward with this without worrying about that, and I can wait a bit longer before merging the PR.

TomJo2000 · 2025-04-12T02:46:10Z

What's weird now is that it stopped failing without the PR being merged,

I think this might have happened because wheel 0.46.1 has now been "yanked" on PyPi, which has externally manipulated the termux-packages termux_setup_python_pip() function such that it now only installs wheel 0.45.1 again instead of wheel 0.46.1 like it did a few days ago.

https://pypi.org/project/wheel/#history

The reason given in the description there is "Causes CI failures where setuptools is pinned to an old version".

Because of that, I guess the PR I opened doesn't necessarily need to be merged yet, so you can continue forward with this without worrying about that, and I can wait a bit longer before merging the PR.

I mean, it's good that they retracted it since it has CI issues.
But I don't like the idea that our termux_setup_python_pip function can be changed by an external change.

- `wheel` version 0.46+ now requires `setuptools` version 70 or newer - discussed upstream here: pypa/wheel#660 - discovered in termux-packages here: termux#24062 (comment) - Lock `wheel` version at 0.46.1 so that if a future `wheel` update causes a similar problem in the future, it doesn't immediately propagate into termux-packages without us explicitly bumping it. Recommended by Tomjo2000 here: termux#24062 (comment) - I have confirmed that this change works to fix the build of `python-lxml` **both** - **in `TERMUX_ON_DEVICE_BUILD=false` mode (broken temporarily for 1 week)**, - fixes `error: invalid command 'bdist_wheel'` - **and also in `TERMUX_ON_DEVICE_BUILD=true` mode (broken for 6 months)**. - fixes `ModuleNotFoundError: No module named 'setuptools'` - It seems appropriate to remove the `if [ "${TERMUX_PYTHON_VERSION#*.}" -lt "12" ]` block because there is no longer any `python3.11` so it does not seem like that code is reachable anymore. - `python-torchvision` needed ajustment of its custom `CFLAGS` to be `CXXFLAGS` instead, because the newer `setuptools` is now compiling it using `aarch64-linux-android-clang++` instead of `aarch64-linux-android-clang`. - `python-pip` and `python-pillow` both had locked `setuptools` in their `TERMUX_PKG_PYTHON_COMMON_DEPS` at versions slightly newer than the old global version, now that the global version is newer than either of those, they do not seem to need that anymore.

… wheel 0.46.1 - `wheel` version 0.46+ now requires `setuptools` version 70 or newer - discussed upstream here: pypa/wheel#660 - discovered in termux-packages here: termux#24062 (comment) - Lock `wheel` version at 0.46.1 so that if a future `wheel` update causes a similar problem in the future, it doesn't immediately propagate into termux-packages without us explicitly bumping it. Recommended by Tomjo2000 here: termux#24062 (comment) - I have confirmed that this change works to fix the build of `python-lxml` **both** - **in `TERMUX_ON_DEVICE_BUILD=false` mode (broken temporarily for 1 week)**, - fixes `error: invalid command 'bdist_wheel'` - **and also in `TERMUX_ON_DEVICE_BUILD=true` mode (broken for 6 months)**. - fixes `ModuleNotFoundError: No module named 'setuptools'` - It seems appropriate to remove the `if [ "${TERMUX_PYTHON_VERSION#*.}" -lt "12" ]` block because there is no longer any `python3.11` so it does not seem like that code is reachable anymore. - `python-torchvision` needed ajustment of its custom `CFLAGS` to be `CXXFLAGS` instead, because the newer `setuptools` is now compiling it using `aarch64-linux-android-clang++` instead of `aarch64-linux-android-clang`. - `python-pip` and `python-pillow` both had locked `setuptools` in their `TERMUX_PKG_PYTHON_COMMON_DEPS` at versions slightly newer than the old global version, now that the global version is newer than either of those, they do not seem to need that anymore.

… wheel 0.46.1 (#24213) - `wheel` version 0.46+ now requires `setuptools` version 70 or newer - discussed upstream here: pypa/wheel#660 - discovered in termux-packages here: #24062 (comment) - Lock `wheel` version at 0.46.1 so that if a future `wheel` update causes a similar problem in the future, it doesn't immediately propagate into termux-packages without us explicitly bumping it. Recommended by Tomjo2000 here: #24062 (comment) - I have confirmed that this change works to fix the build of `python-lxml` **both** - **in `TERMUX_ON_DEVICE_BUILD=false` mode (broken temporarily for 1 week)**, - fixes `error: invalid command 'bdist_wheel'` - **and also in `TERMUX_ON_DEVICE_BUILD=true` mode (broken for 6 months)**. - fixes `ModuleNotFoundError: No module named 'setuptools'` - It seems appropriate to remove the `if [ "${TERMUX_PYTHON_VERSION#*.}" -lt "12" ]` block because there is no longer any `python3.11` so it does not seem like that code is reachable anymore. - `python-torchvision` needed ajustment of its custom `CFLAGS` to be `CXXFLAGS` instead, because the newer `setuptools` is now compiling it using `aarch64-linux-android-clang++` instead of `aarch64-linux-android-clang`. - `python-pip` and `python-pillow` both had locked `setuptools` in their `TERMUX_PKG_PYTHON_COMMON_DEPS` at versions slightly newer than the old global version, now that the global version is newer than either of those, they do not seem to need that anymore.

Add polyfill implementation of `xmlShellPrintNode()` `libxml2` 2.14 has removed the function `xmlShellPrintNode()`, so this places a basic polyfill rewrite of it into the source code of `rdrview` 0.1.3 using code copied and pasted from the code that was removed from `libxml2`. tested with the command `rdrview -B firefox https://repology.org/maintainers/`, it appears to be working the same as it was with `libxml2` 2.13. Co-authored-by: Robert Kirkman <rkirkman@termux.dev>

fix build with `libxml2` 2.14 Fixes this error when building with libxml2 2.14 src/xml_pyx.c:203:36: error: expected ')' 203 | pyxExternalSubsetHandler(void *ctx ATTRIBUTE_UNUSED, const xmlChar *name, | ^ Probably has something to do with this commit in libxml2 GNOME/libxml2@208f27f Co-authored-by: Robert Kirkman <rkirkman@termux.dev>

- Apparently missed during termux#24062

- Apparently missed during #24062

TomJo2000 requested a review from Grimler91 as a code owner April 1, 2025 02:36

TomJo2000 force-pushed the libxml2-rebuilds2 branch from dd48bc3 to f40161d Compare April 1, 2025 02:38

robertkirkman force-pushed the libxml2-rebuilds2 branch from 90d7c93 to f3ed117 Compare April 2, 2025 04:53

TomJo2000 mentioned this pull request Apr 2, 2025

bump(main/libxml2): 2.14.1 #24059

Merged

TomJo2000 force-pushed the libxml2-rebuilds2 branch 2 times, most recently from cdb535e to fd47441 Compare April 8, 2025 22:21

robertkirkman mentioned this pull request Apr 9, 2025

bump(scripts/build/setup/termux_setup_python_pip): setuptools 78.1.0, wheel 0.46.1 #24213

Merged

TomJo2000 force-pushed the libxml2-rebuilds2 branch from fd47441 to e12de9a Compare April 12, 2025 00:26

TomJo2000 force-pushed the libxml2-rebuilds2 branch 2 times, most recently from 9257061 to ace7673 Compare April 13, 2025 21:55

TomJo2000 and others added 21 commits April 14, 2025 01:20

revbump(main/lv2): after libxml2 update

84b5b47

revbump(main/mapserver): after libxml2 update

eefe804

revbump(main/monetdb): after libxml2 update

41efbd9

revbump(main/netpbm): after libxml2 update

53bf20d

revbump(main/newsboat): after libxml2 update

7aa1eb3

revbump(main/nzbget): after libxml2 update

df64d94

revbump(main/oathtool): after libxml2 update

ca02292

revbump(main/openbabel): after libxml2 update

ae43ee2

revbump(main/python-lxml): after libxml2 update

6d60dd3

revbump(main/qalc): after libxml2 update

a800285

revbump(main/recoll): after libxml2 update

296347d

revbump(main/spatialite-tools): after libxml2 update

f9abbf5

revbump(main/squid): after libxml2 update

53da3ac

revbump(main/stoken): after libxml2 update

1ca2c57

revbump(main/totem-pl-parser): after libxml2 update

2da5ad5

revbump(main/uwsgi): after libxml2 update

8bdd116

revbump(main/yuma123): after libxml2 update

882dea6

revbump(root/nfs-utils): after libxml2 update

92cf14e

revbump(root/tshark): after libxml2 update

af9cf05

TomJo2000 force-pushed the libxml2-rebuilds2 branch from ace7673 to af9cf05 Compare April 13, 2025 23:20

TomJo2000 merged commit 24a89a0 into termux:master Apr 13, 2025
4 of 9 checks passed

robertkirkman mentioned this pull request May 1, 2025

[Bug]: msgfmt CANNOT LINK EXECUTABLE libattr.so #24569

Closed

TomJo2000 mentioned this pull request May 5, 2025

[Bug]: sc-im can't find libxml2.so.2 #24629

Closed

TomJo2000 added a commit to TomJo2000/termux-packages-prs that referenced this pull request May 5, 2025

revbump(main/sc-im): after libxml2 update

7d2893d

- Apparently missed during termux#24062

TomJo2000 mentioned this pull request May 5, 2025

revbump(main/sc-im): after libxml2 update #24630

Merged

TomJo2000 added a commit that referenced this pull request May 5, 2025

revbump(main/sc-im): after libxml2 update

cc2c5c4

- Apparently missed during #24062

intr-cx mentioned this pull request May 7, 2025

libxml2 2.14 recent updates and legacy - problems in Archlinux eafer/rdrview#45

Closed

TomJo2000 deleted the libxml2-rebuilds2 branch June 8, 2025 02:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Libxml2 rebuilds: Part 2 - Small packages #24062

Libxml2 rebuilds: Part 2 - Small packages #24062

Uh oh!

TomJo2000 commented Apr 1, 2025 •

edited

Loading

Uh oh!

robertkirkman commented Apr 1, 2025 •

edited

Loading

Uh oh!

robertkirkman commented Apr 2, 2025 •

edited

Loading

Uh oh!

TomJo2000 commented Apr 2, 2025

Uh oh!

robertkirkman commented Apr 2, 2025

Uh oh!

TomJo2000 commented Apr 2, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

TomJo2000 commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

TomJo2000 commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 12, 2025

Uh oh!

TomJo2000 commented Apr 12, 2025

Uh oh!

robertkirkman commented Apr 12, 2025

Uh oh!

TomJo2000 commented Apr 12, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Libxml2 rebuilds: Part 2 - Small packages #24062

Libxml2 rebuilds: Part 2 - Small packages #24062

Uh oh!

Conversation

TomJo2000 commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robertkirkman commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

robertkirkman commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TomJo2000 commented Apr 2, 2025

Uh oh!

robertkirkman commented Apr 2, 2025

Uh oh!

TomJo2000 commented Apr 2, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

TomJo2000 commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 9, 2025

Uh oh!

TomJo2000 commented Apr 9, 2025

Uh oh!

robertkirkman commented Apr 12, 2025

Uh oh!

TomJo2000 commented Apr 12, 2025

Uh oh!

robertkirkman commented Apr 12, 2025

Uh oh!

TomJo2000 commented Apr 12, 2025

Uh oh!

Uh oh!

Uh oh!

TomJo2000 commented Apr 1, 2025 •

edited

Loading

robertkirkman commented Apr 1, 2025 •

edited

Loading

robertkirkman commented Apr 2, 2025 •

edited

Loading