Correct Bundler 2.x Gem path on war/jar load #575

chadlwilson · 2025-10-09T17:36:00Z

This PR attempts a proper solution after digging into JRuby / Rubygems and bundler changes and reviewing the many attempts to address this over the years.

This fixes the integration tests when run via #574 but submitting these PRs independently. I'll rebase one on the other.

re-attempts Added Gem.clear_paths to bundler.erb #437 (which was never merged, but I believe reflected a correct fix to the issue)
obsoletes Try to fix Gem.paths #564
- a side effect of this approach was calling Gem.clear_paths, but not in the ideal location
obsoletes Fix gem path resolution for JRuby 9.4+ and Rails 7+ #572
- a side effect of this approach was calling Gem.clear_paths, but not in the ideal location
obsoletes Support bundler 2 #494
- cherry picks Bundler 2 clean-ups from this branch which reduce the # of combinations to worry about here
obsoletes bundler trait infinite loop fix #556
- removes the old Bundler hack that caused this issue

Observed problem

warbler tries to update GEM_HOME and GEM_PATH but the Gem.paths are not updated
gems can then not be loaded in subsequent bundler usage as the GEM_PATH is not as intended.
- sometimes it may work correctly if GEM_HOME and GEM_PATH are in a particular state from your wider environment, regardless of the value of config.override_gem_home.

Why is this happening?

Gem.paths are cached on usage. If you change ENV afterwards you need to Gem.clear_paths or directly update via Gem.paths
The current warbler code makes various changes to GEM_HOME and GEM_PATH (especially if using the default override_gem_home = true but does not clear any paths already cached by Gem. It seems to assume that no Rubygems will have been used or needed prior to this point.

Thus the current warbler code is EXTREMELY sensitive to any Ruby code run before the boot hooks like this: (and similar in jar.erb)

warbler/lib/warbler/templates/war.erb

Lines 12 to 15 in 523237b

    
             ENV['GEM_HOME'] <%= config.override_gem_home ? '=' : '||=' %> $servlet_context.getRealPath('<%= config.gem_path %>') 
        
           <% if config.override_gem_home -%> 
        
             ENV['GEM_PATH'] = nil 
        
           <% end -%>

What else happens before this code?

Jruby-rack does all sorts of stuff before calling warbler's init.rb hook.

it's not just plain Java code.
executes all sorts of scriptlets
Sequence goes like
1. Create JRuby runtime
2. RackLibrary.load(...)
3. runtime.evalScriptlet("require 'rack/handler/servlet'"); <-- PROBLEM
  1. require 'jruby/rack <-- PROBLEM
    1. require all sorts of initial utility jruby-rack stuff
    2. require 'jruby/rack/core_ext' <-- PROBLEM
      1. require 'jruby/rack/capture' <-- PROBLEM
        
        require 'stringio' <-- ROOT CAUSE of Gem.path initialisation
        
        https://github.com/jruby/jruby-rack/blob/2b54b77f54e9878860db7f90381a6b47b57437e8/src/main/ruby/jruby/rack/capture.rb#L8
        
        there is a special jruby require 'stringio.jar' in here.
4. checkAndSetRackVersion to decide how to boot rack (bundler? rubygems? vendored?)
  - this does some requires and could cause a bundler load here too, but not generally in warbler case
5. runtime.evalScriptlet("load 'jruby/rack/boot/rack.rb'");
  1. require 'jruby/rack/booter'
  2. Booter.boot rack
    1. adjust_gem_paths
    2. other minor stuff
    3. run META-INF/init.rb script <-- this is where warbler's script runs
  3. Booter.bootrails <-- (if using rails, usually things fail here from user perspective)

Why did it start happening in JRuby 9.4?

In Jruby 9.3 StringIO is entirely a JRuby extension. No RubyGems involved:

https://github.com/jruby/jruby/blob/jruby-9.3/lib/ruby/stdlib/stringio.rb

In JRuby 9.4 StringIO extension has been moved to the default gem (although baked into stdlib on jruby?)

I don't really understand why it's different, but this change seems to have changed the load mechanism and cause Gem.paths to be initialized as soon as 'stringio.jar' has been required, which means it is initiailized as soon as the jruby-rack servlet handling is initialized.

$ mise x ruby@jruby-9.3 -- ruby -e 'require "stringio"; p "Gem.path inited? #{!Gem.instance_variable_get(:@paths).nil?}"'
"Gem.path inited? false"

$ mise x ruby@jruby-9.4 -- ruby -e 'require "stringio"; p "Gem.path inited? #{!Gem.instance_variable_get(:@paths).nil?}"'
"Gem.path inited? true"

Why I think we should address it as in this PR

The change I propose here was already proposed in #437 for a different issue much earlier.

this was never merged due to @kares concern at Can't find bundler/shared_helpers after upgrade to jruby 9.2.0.0 #432 (comment) although it was a different issue being addressed at the time
despite these objections, given the caching, it seems wrong to interfere with GEM_HOME and GEM_PATH and assume they have not been "used" earlier and already cached.
- See caching at https://github.com/rubygems/rubygems/blob/8ed1452b6c5dd4c74251989d8ebcc447f026c177/lib/rubygems.rb#L382-L384
- it is worth noting that jruby-rack also can mangle gem_path in some circumstances, prior to running warbler's boot hooks, which will probably cause it to be cached.

Without clearing Gem.paths whenever we mutate GEM_HOME, GEM_PATH etc, we can't be confident they are at least accurate for future calls - as far as I can tell?

Additional changes to possibly make

Move the require 'stringio' from jruby-rack to be done later in process (?)
- It used to be done later in jruby-rack < 1.1.4
  jruby/jruby-rack@86a5b6f
- This may just move the issue somewhere else prior to the warbler boot hook being called.
- edit: Subsequently done in 1.2.x: Ensure that Rubygems is not unnecessarily initialized during boot and Gem.paths are left in consistent state jruby-rack#358 for jruby-rack 1.2.6
Clear gem paths after they are mangled in jruby as well?
- edit: Subsequently done in 1.2.x: Ensure that Rubygems is not unnecessarily initialized during boot and Gem.paths are left in consistent state jruby-rack#358 for jruby-rack 1.2.6

Alternatives

Completely rethink the entire setup of playing with paths in warbler, since its boot hook does not run sufficiently early enough to do so safely. Things may have been loaded already.
- One would have to do it in jruby-rack, and perhaps set things in init parameters that tell jruby-rack what to do. (like config.webxml.gem.path and similar...).
Update Gem.paths directly rather than using ENV, similar to https://github.com/jruby/warbler/pull/572/files. I believe that might lead to inconsistency though, seems better to update both.

Currently warbler is unusable from jruby 9.2 during to this require failing (not sure to which extent though, but at least one spec is failing and there's an open issue about this). If we stop monkeypatching bundler, we don't need to require it, and thus the error disappears. Since doing that doesn't make any tests fail, I will assume the problems caused by not monkeypatching bundler are less important than the problems caused by doing it. So, I'm removing the code to fix the issue and get specs green.

kares · 2025-10-10T10:50:11Z

🎉 LGTM, haven't investigated in detail
but the approach here given the explanation makes sense to me (opposed to earlier attempts).

Move the require 'stringio' from jruby-rack to be done later in process (?)
It used to be done later in jruby-rack < 1.1.4 jruby/jruby-rack@86a5b6f

if that turns out problematic, we could just refactor the require 'jruby/rack/capture' to happen later since it's not really needed under normal circumstances. should just be used when application loading fails.

actually thought it was refactored at some point already,
since the monkey-patching of core types (such as Exception) shouldn't be really needed for the "capture" and is likely not desired, from a user perspective.

chadlwilson · 2025-10-10T14:52:01Z

if that turns out problematic, we could just refactor the require 'jruby/rack/capture' to happen later since it's not really needed under normal circumstances. should just be used when application loading fails.

Good idea, I'll take a look at both.

(edit: done at jruby/jruby-rack#358 however couldn't find a place to move the capture stuff later, as it is at the last possible moment before initializing rack and calling the init method above).

…nt state As noted at jruby/warbler#575 for warbler, it is similarly possible for jruby-rack to leave ENV['GEM_PATH'] and Gem.path in an inconsistent state after it updates them. This would make any later Gem.clear_paths or Bundler usage (driven from the ENV vars) unpredictable. This happens because jruby-rack alters `ENV['GEM_PATH']``, in some cases but does not force Rubygems to recalculate Gem.paths. Normally this is fine, however if Gem.paths had already been used and cached prior to this mangling `Gem.path` will contain a "stale" value with respect to ENV. (e.g possibly missing the `GEM_HOME`/`Gem.default_dir` value, or retaining values from an old `GEM_PATH`. Clearing the paths is the only way to ensure things are consistent for later usage. In particular this currently can happen on JRuby 9.4+ because often `Gem.path` is already initialized by the time we boot, due to the use of default gems such as `stringio`. Rather than assume a given state, it would be better to ensure it is consistent by clearing paths whenever we touch `ENV['GEM_PATH']`.

…lse init parameter option Changes the `false` value to behave the same way as empty, and not touch ENV['GEM_PATH'] or Gem.path at all. Previously this allowed you to alter Gem.path without touching the ENV['GEM_PATH']'. This was unsafe with Bundler 1.6+ (so for a long time), because many things may cause Gem.path to be recalculated from env, including bundler initialisations, restarts and later changes within an application's boot hooks. Furthermore the behaviour varied wildly depending on whether Rubygems was initialised prior to the Rack booter, or afterwards, even without bundler causing indeterminism and issues similar to jruby/warbler#575 I cannot find any reference to use of this value on GitHub, or discussion, and it would not work at all with modern JRuby/bundler so removing it, because it helps avoid further issues. Note that it was added in jruby@2fd826b and then the default changed back in jruby@d3ee8f0 shortly after when issues were discovered, so given it's undocumented/unsupported, it is unlikely to be relied upon by anyone.

On JRuby 9.4 this causes Rubygems initialisation which means realisation of init values. This screws with jruby-rack's own guarantees about env implied by its config values and can cause issues in boot hooks such as jruby/warbler#575 due to Rubygems being in a different state to what they expect.