Bundler works when .rb file called directly, fails when called via exec from other ruby script

Question

I need to call a ruby script2.rb using Bundler + Gemfile from another ruby script1.rb using Bundler + Gemfile.

I noticed that the Gems available to script2.rb are only those of script1.rb. The gems unique to script2.rb are not available when script2.rb is called by script1.rb. When script2.rb is called directly from the bash shell, everything is available.

I made sure it's not an ENV problem, I compared them using diff with helper code like so in both files and made a couple modifications to make them match.

File.open("script2_env.txt", 'wb') {|f| f.write(JSON.pretty_generate(ENV.to_h))}

To make sure it was not a $LOAD_PATH problem, I also made sure they match.

In script2.rb, which is called by script1.rb, I added the following line to match the $LOAD_PATH of script1 :

$:.unshift "/usr/local/Cellar/rbenv/1.1.2/rbenv.d/exec/gem-rehash"

My understanding of the problem is that somehow Bundler is not initialized properly when script2.rb is called from script1.rb, maybe because there is no

eval "$(rbenv init -)"

as there is in my bash_profile

script1/script1.rb :

#!/usr/bin/env ruby
cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir)
require 'bundler' ;  Bundler.setup
require "awesome_print"
ap "in script1, we have awesome_print in our GemFile"
exec("/Users/charbon/wip/script2/script2.rb")

script1/Gemfile

source 'https://rubygems.org'
gem 'awesome_print'

script2.rb :

#!/usr/bin/env ruby
puts "we are now in script2.rb"
$:.unshift "/usr/local/Cellar/rbenv/1.1.2/rbenv.d/exec/gem-rehash"
cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir)

#make ENV match to script1 ENV
ENV.delete('BUNDLER_ORIG_GEM_PATH') 
ENV['BUNDLE_GEMFILE']=ourDir+"/Gemfile"
ENV['RBENV_DIR']=ourDir

require 'bundler' ;
Bundler.setup
require 'awesome_print'

ap "in script2, we also have awesome_print in our GemFile"
puts "but we also have colored, which is not available, this throws an erro"
require "colored"

script2/Gemfile

source 'https://rubygems.org'
gem 'awesome_print'
gem 'colored'

The result is

 /Users/charbon/wip/script1/script1.rb 
"in script1, we have awesome_print in our GemFile"
we are now in script2.rb
ourDir is /Users/charbon/wip/script2
"in script2, we also have awesome_print in our GemFile"
but we also have colored, which is not available, this throws an error
/Users/charbon/wip/script2/script2.rb:19:in `require': cannot load such file -- colored (LoadError)
    from /Users/charbon/wip/script2/script2.rb:19:in `<main>'

Are you able to use an [inline Gemfile](https://bundler.io/v2.0/guides/bundler_in_a_single_file_ruby_script.html) in your second script? — Jay Dorsey, Jul 02 '19 at 01:22
Jay ; it sure works also, thanks. If you make it an answer, I'll up vote it! — MichaelC, Jul 03 '19 at 05:47

score 4 · Accepted Answer · answered Jul 01 '19 at 18:34

script1.rb

#!/usr/bin/env ruby
cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir)
require 'bundler' ;  Bundler.setup
require "awesome_print"
ap "in script1, we have awesome_print in our GemFile"

Bundler.with_clean_env do
  Dir.chdir('/Users/charbon/wip/script2/script2.rb') do
    exec("./script2.rb")
  end 
end

script2.rb

#!/usr/bin/env ruby
puts "we are now in script2.rb"
cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir)


require 'bundler' ;
Bundler.setup
require 'awesome_print'

ap "in script2, we also have awesome_print in our GemFile"
puts "but we also have colored, which is not available, this throws an erro"
require "colored"

thanks, you put me on the right track. The only change required is ```Bundler.with_clean_env { exec("/path/to/script2.rb") }``` (no need for Dir.chdir before) — MichaelC, Jul 03 '19 at 05:48

MichaelC · Answer 2 · 2019-07-03T18:34:52.880

To sum up and to complete Marat Amerov's answer:

Bundler has this very well documented:

Any Ruby code that opens a subshell (like system, backticks, or %x{}) will automatically use the current Bundler environment. If you need to shell out to a Ruby command that is not part of your current bundle, use the with_clean_env method with a block

Second solution is as commented by Jay Dorsey : use the bundler/inline method

I have tested both methods and they work fine.

score 4 · Answer 3 · answered Jul 03 '19 at 14:42

From my comment on the original question:

Bundler supports inline Gemfiles which allow you to specify your required gems (and sources!) directly in your ruby script. As long as you have bundler installed on the machine it should work, allowing you to keep your script dependencies isolated.

Bundler will handle installing & requiring the required gems, allowing you to run your script

score 2 · Answer 4 · answered Jun 30 '19 at 06:45

2

It seems that you need to make Dir.chdir "/Users/charbon/wip/script2/" before calling script2.rb. You are calling script2.rb from a directory, where there is no Gemfile for it. The script will look like this:

#!/usr/bin/env ruby
cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir)
require 'bundler' ;  Bundler.setup
require "awesome_print"
ap "in script1, we have awesome_print in our GemFile"
Dir.chdir "/Users/charbon/wip/script2/"
exec("script2.rb")

answered Jun 30 '19 at 06:45

theendcomplete

69
10

no, cwd=Dir.pwd ; ourDir=File.dirname(__FILE__) ; Dir.chdir(ourDir) ; #takes care of dir Changing, – MichaelC Jun 30 '19 at 06:54
`cwd=Dir.pwd ; ourDir=File.dirname(FILE) ; Dir.chdir(ourDir) ;` is located inside script2.rb, so the script2 itself is called without Gemfile. The script2.rb must be called from a directory where it's Gemfile is located. – theendcomplete Jun 30 '19 at 07:09
I don't think Bundler work that way. But anyway, the solution is something I can't imagine yet, so I did try ```Dir.chdir('/Users/charbon/wip/script2') ; puts "we are now in #{cwd} ; ls gives" ; puts `ls`; exec("/Users/charbon/wip/script2/script2.rb")``` and the result is the same. Bundler just needs to be able to find the Gemfile when it's called via require 'bundler'. – MichaelC Jun 30 '19 at 20:22
it seems that Dir.chdir do not change context to a new directory, shell script works fine: ```#bin/sh ./script1.rb cd ../script2 ls ./script2.rb``` there is a message in https://stackoverflow.com/questions/3410310/exec-the-cd-command-in-a-ruby-script/11954975 : **You can't change the working directory (or the environment variables for that matter) of the shell that invoked you in a ruby script (or any other kind of application). The only commands that can change the pwd of the current shell are shell built-ins.** Sorry for formatting. – theendcomplete Jul 01 '19 at 07:34

Bundler works when .rb file called directly, fails when called via exec from other ruby script

4 Answers4