WIP: MDEV-20544: Update bundled Groonga and Mroonga #4185

otegami · 2025-07-11T02:56:10Z

This is still in work in progress. I opened this PR just for using CI. I'll update the PR description after this is completed.

The Jira issue number for this PR is: MDEV-20544

Description

TODO: fill description here

Release Notes

TODO: What should the release notes say about this change?
Include any changed system variables, status variables or behaviour. Optionally list any https://mariadb.com/kb/ pages that need changing.

How can this PR be tested?

TODO: modify the automated test suite to verify that the PR causes MariaDB to behave as intended.
Consult the documentation on "Writing good test cases".

If the changes are not amenable to automated testing, please explain why not and carefully describe how to test manually.

Basing the PR against the correct MariaDB version

This is a new feature or a refactoring, and the PR is based against the main branch.
This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

grooverdan · 2025-07-11T07:49:35Z

so list of all CI elements for this PR is https://buildbot.mariadb.org/#/grid?branch=refs%2Fpull%2F4185%2Fhead

note Fedora 40 and Ubuntu 20.04 are getting moved as EOL.

deb-autobake requires debian/* changes for install files.

windows - stricter compiler.

appears to be some dependencies - can you provide a list.

otegami · 2025-07-11T08:27:57Z

Thanks for the pointers! We’ll tackle these one by one.

appears to be some dependencies - can you provide a list.

In the meantime, here are the packages I believe we’ll need:

Groonga (Ubuntu Build-Depends)

libedit-dev
libevent-dev
liblz4-dev
libmecab-dev
libmsgpack-dev
libsimdjson-dev
libssl-dev
libstemmer-dev
libthrift-dev
libxxhash-dev
libzmq3-dev
libzstd-dev
rapidjson-dev
zlib1g-dev

ref: https://github.com/groonga/groonga/blob/7b50b0cd0ca6815fb430a63e945e7220693dd3d7/packages/debian/control#L9-L26
ref: https://github.com/groonga/groonga/blob/main/packages/apt/debian-bookworm/Dockerfile#L24-L47

Mroonga (Build-Depends with MariaDB 11.4)

groonga-devel
groonga-normalizer-mysql-devel
MariaDB-client(RPM)
MariaDB-common(RPM)
MariaDB-server(RPM)
MariaDB-shared(RPM)

ref: https://github.com/mroonga/mroonga/blob/main/packages/mariadb-11.4-mroonga/yum/mariadb-11.4-mroonga.spec.in#L32-L37

CLAassistant · 2025-07-14T00:01:56Z

All committers have signed the CLA.

ParadoxV5 · 2025-07-14T05:55:33Z

🆙 for submodules! 🫠

After updating bundled Groonga and Mroonga, the Debian CI build was failing with dh_missing errors. The upstream CMake files now install additional files that weren't accounted for in the Debian packaging. - Update debian/mariadb-plugin-mroonga.install to include new files - Add distribution-specific scripts to debian/not-installed to exclude them from packaging since Debian has its own packaging mechanisms This aligns the Debian packaging with the upstream installation targets while maintaining proper Debian packaging practices.

Fix the following errors: ``` h_missing --fail-missing dh_missing: warning: usr/share/doc/groonga-normalizer-mysql/groonga-normalizer-mysql/README.md exists in debian/tmp but is not installed to anywhere (related file: "debian/tmp/usr/share/groonga/README.md") dh_missing: warning: usr/share/doc/groonga-normalizer-mysql/groonga-normalizer-mysql/lgpl-2.0.txt exists in debian/tmp but is not installed to anywhere (related file: "debian/tmp/usr/share/groonga-normalizer-mysql/lgpl-2.0.txt") dh_missing: warning: usr/share/groonga/onigmo/AUTHORS exists in debian/tmp but is not installed to anywhere (related file: "debian/tmp/usr/share/mariadb/mroonga/AUTHORS") dh_missing: warning: usr/share/groonga/onigmo/COPYING exists in debian/tmp but is not installed to anywhere (related file: "debian/tmp/usr/share/groonga/COPYING") dh_missing: warning: usr/share/groonga/onigmo/HISTORY exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/groonga/onigmo/README exists in debian/tmp but is not installed to anywhere (related file: "debian/tmp/usr/share/mariadb/policy/selinux/README") dh_missing: warning: usr/share/groonga/rapidjson/CHANGELOG.md exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/groonga/rapidjson/license.txt exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/groonga/rapidjson/readme.md exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/mariadb/mroonga/deb/postinst.sh exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/mariadb/mroonga/deb/prerm.sh exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/mariadb/mroonga/rpm/post.sh exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/mariadb/mroonga/rpm/preun.sh exists in debian/tmp but is not installed to anywhere dh_missing: warning: usr/share/mariadb/mroonga/update.sql exists in debian/tmp but is not installed to anywhere dh_missing: error: missing files, aborting While detecting missing files, dh_missing noted some files with a similar name to those that were missing. This error /might/ be resolved by replacing references to the missing files with the similarly named ones that dh_missing found - assuming the content is identical. As an example, you might want to replace: * debian/tmp/usr/share/groonga/README.md with: * usr/share/doc/groonga-normalizer-mysql/groonga-normalizer-mysql/README.md in a file in debian/ or as argument to one of the dh_* tools called from debian/rules. (Note it is possible the paths are not used verbatim but instead directories containing or globs matching them are used instead) Alternatively, add the missing file to debian/not-installed if it cannot and should not be used. The following debhelper tools have reported what they installed (with files per package) * dh_install: libmariadb-dev (154), libmariadb-dev-compat (4), libmariadb3 (6), libmariadb3-compat (2), libmariadbclient18 (1), libmariadbd-dev (5), libmariadbd19 (1), libmysqlclient18 (1), mariadb-backup (6), mariadb-client (51), mariadb-client-compat (38), mariadb-client-core (6), mariadb-common (1), mariadb-plugin-columnstore (121), mariadb-plugin-connect (2), mariadb-plugin-connect-jdbc (1), mariadb-plugin-cracklib-password-check (2), mariadb-plugin-gssapi-client (1), mariadb-plugin-gssapi-server (2), mariadb-plugin-hashicorp-key-management (3), mariadb-plugin-mroonga (5), mariadb-plugin-oqgraph (2), mariadb-plugin-provider-bzip2 (2), mariadb-plugin-provider-lz4 (2), mariadb-plugin-provider-lzma (2), mariadb-plugin-provider-lzo (2), mariadb-plugin-provider-snappy (2), mariadb-plugin-rocksdb (8), mariadb-plugin-s3 (4), mariadb-plugin-spider (2), mariadb-server (83), mariadb-server-compat (12), mariadb-server-core (47), mariadb-test (54), mariadb-test-data (6), mysql-common (0) If the missing files are installed by another tool, please file a bug against it. When filing the report, if the tool is not part of debhelper itself, please reference the "Logging helpers and dh_missing" section from the "PROGRAMMING" guide for debhelper (10.6.3+). (in the debhelper package: /usr/share/doc/debhelper/PROGRAMMING.gz) Be sure to test with dpkg-buildpackage -A/-B as the results may vary when only a subset is built If the omission is intentional or no other helper can take care of this consider adding the paths to debian/not-installed. dh_install: error: dh_missing --fail-missing returned exit code 25 make: *** [debian/rules:200: binary] Error 25 dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2 ```

otegami · 2025-07-16T07:48:22Z

@grooverdan

About mroonga/mroonga#943 (comment)

Thanks again for the guidance. The Mroonga team has been working hard on this,
but unfortunately we’re unlikely to finish everything by July 24 due to two major blockers at the moment.
We’ll keep pushing on and will update you as soon as we have progress.

Windows CI (buildbot/amd64-windows)

The Windows builder is failing with platform‑specific errors.
We’re also treating all warnings (e.g. C4244) as errors, and there are too many to resolve before the release date.
For example, ii.cpp in Groonga alone generated enormous errors, and we estimate around a thousand such locations across the codebase.

ref: https://buildbot.mariadb.org/#/builders/234/builds/40629
ref: https://github.com/abetomo/groonga/actions/runs/16311415283/job/46067698311#step:13:2714

Compile‑only CI (buildbot/amd64-compile-only-nopart-debug)

We’re encountering ISO C90 "mixed declarations and code" errors in dependencies like H3 and llama.cpp's ggml.
H3's CMakeLists forces C99 features (see https://github.com/uber/h3/blob/2257ace292aeaa1d21ac06b157ca916c012f5add/CMakeLists.txt#L362),
and llama.cpp errors appear as follows.

error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement]
    const uint32_t bits = fp32_to_bits(base);

While disabling these modules would clear these errors because they aren't really necessary for Mroonga functions, there are likely many more hidden issues in other modules, and we currently do not have enough time to fix them all by July 24.

ref: https://buildbot.mariadb.org/#/builders/904/builds/111

kou · 2025-07-17T00:16:56Z

Windows CI (buildbot/amd64-windows)

We can disable some warning compile options in the first step. We can work on suppressing warnings later.

Compile‑only CI (buildbot/amd64-compile-only-nopart-debug)

We can disable the declaration-after-statement warning by adding -Wno-declaration-after-statement.

kou · 2025-07-17T00:18:59Z

diff --git a/CMakeLists.txt b/CMakeLists.txt
index a23f57b83..fe0ba122d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -370,7 +370,7 @@ if(GRN_C_COMPILER_GNU_LIKE)
   check_build_flag("-Wwrite-strings")
   check_build_flag("-Wsign-compare")
   check_build_flag("-Wmissing-field-initializers")
-  check_cflag("-Wno-declaration-after-statement")
+  check_build_flag("-Wno-declaration-after-statement")
   check_cxxflag("-fexceptions")
   check_cxxflag("-fimplicit-templates")
   check_build_flag("-Wno-implicit-fallthrough")

kou · 2025-07-17T00:21:15Z

Or:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index a23f57b83..e54b9bc2e 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1573,6 +1573,8 @@ if(NOT "${GRN_WITH_H3}" STREQUAL "no")
         # https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-2-c4307
         string(APPEND CMAKE_C_FLAGS " /wd4307")
         string(APPEND CMAKE_CXX_FLAGS " /wd4307")
+      elseif(GRN_C_COMPILER_GNU_LIKE)
+        string(APPEND CMAKE_C_FLAGS " -Wno-declaration-after-statement")
       endif()
       set(BUILD_BENCHMARKS OFF)
       set(BUILD_FILTERS OFF)
@@ -1675,6 +1677,8 @@ if(NOT "${GRN_WITH_LLAMA_CPP}" STREQUAL "no")
         # https://learn.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4996
         string(APPEND CMAKE_C_FLAGS " /wd4996")
         string(APPEND CMAKE_CXX_FLAGS " /wd4996")
+      elseif(GRN_C_COMPILER_GNU_LIKE)
+        string(APPEND CMAKE_C_FLAGS " -Wno-declaration-after-statement")
       endif()
       set(BUILD_SHARED_LIBS ON)
       set(BUILD_STATIC_LIBS OFF)

ParadoxV5 · 2025-07-17T01:05:30Z

We’re encountering ISO C90 "mixed declarations and code" errors

Does Mroonga/Groonga still use C90?

kou · 2025-07-17T01:09:33Z

No.

MariaDB adds -Wdeclaration-after-statement:

server/cmake/maintainer.cmake

Line 25 in ce7ab46

-Wdeclaration-after-statement

It's related.

(I think that MariaDB can remove the warning.)

ParadoxV5 · 2025-07-17T01:15:48Z

(I think that MariaDB can remove the warning.)

Maybe MariaDB had it there to enforce a coding style?

kou · 2025-07-17T01:39:27Z

Oh, I missed it.

It was added on 2023-02-14: f74bb51

It'll be still valid now.

otegami marked this pull request as draft July 11, 2025 02:57

otegami mentioned this pull request Jul 11, 2025

WIP: MDEV-20544: Update bundled Groonga and Mroonga #3864

Closed

5 tasks

otegami changed the base branch from main to 11.4 July 11, 2025 06:13

otegami force-pushed the mroonga-update branch from 788f73c to b7a912c Compare July 11, 2025 06:27

svoj added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Jul 11, 2025

abetomo mentioned this pull request Jul 14, 2025

ci: mariadb: Run Configure MariaDB in compile-only-minimal as well mroonga/mroonga#948

Merged

kou and others added 4 commits July 16, 2025 13:58

Update bundled Groonga and Mroonga

fad49a6

update submodules

53ee5b9

otegami force-pushed the mroonga-update branch from 8af82bc to 5b2a10f Compare July 16, 2025 04:59

ParadoxV5 mentioned this pull request Jul 22, 2025

MDEV-34388 debug clang #4145

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

WIP: MDEV-20544: Update bundled Groonga and Mroonga #4185

WIP: MDEV-20544: Update bundled Groonga and Mroonga #4185

Uh oh!

otegami commented Jul 11, 2025

Uh oh!

grooverdan commented Jul 11, 2025

Uh oh!

otegami commented Jul 11, 2025 •

edited

Loading

Uh oh!

CLAassistant commented Jul 14, 2025 •

edited

Loading

Uh oh!

ParadoxV5 commented Jul 14, 2025

Uh oh!

otegami commented Jul 16, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

ParadoxV5 commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

ParadoxV5 commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

Uh oh!

Uh oh!

WIP: MDEV-20544: Update bundled Groonga and Mroonga #4185

Are you sure you want to change the base?

WIP: MDEV-20544: Update bundled Groonga and Mroonga #4185

Uh oh!

Conversation

otegami commented Jul 11, 2025

Description

Release Notes

How can this PR be tested?

Basing the PR against the correct MariaDB version

PR quality check

Uh oh!

grooverdan commented Jul 11, 2025

Uh oh!

otegami commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Groonga (Ubuntu Build-Depends)

Mroonga (Build-Depends with MariaDB 11.4)

Uh oh!

CLAassistant commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ParadoxV5 commented Jul 14, 2025

Uh oh!

otegami commented Jul 16, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

ParadoxV5 commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

ParadoxV5 commented Jul 17, 2025

Uh oh!

kou commented Jul 17, 2025

Uh oh!

Uh oh!

otegami commented Jul 11, 2025 •

edited

Loading

CLAassistant commented Jul 14, 2025 •

edited

Loading