I'm logged in to my production server to try to manually dig into a problem I'm having with my Capistrano deployments.
Here's the line that Capistrano is running.
deploy@borges:/domains/myapp.com/releases/20140119013611$ env RBENV_ROOT="/home/deploy/.rbenv" PATH="/home/deploy/.rbenv/shims:/home/deploy/.rbenv/bin:$PATH" RBENV_VERSION="2.0.0-p353" /home/deploy/.rbenv/bin/rbenv exec bundle install --gemfile /domains/myapp.com/releases/20140119013611/Gemfile --path /domains/myapp.com/shared/bundle --deployment --without development test
This seems to do the trick:
Using rake (10.1.1)
Using i18n (0.6.9)
Using multi_json (1.8.4)
Using activesupport (3.2.16)
Using builder (3.0.4)
Using activemodel (3.2.16)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.5)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.3)
Using tilt (1.4.1)
Using sprockets (2.2.2)
Using actionpack (3.2.16)
Using mime-types (1.25.1)
Using polyglot (0.3.3)
Using treetop (1.4.15)
Using mail (2.5.4)
Using actionmailer (3.2.16)
Using arel (3.0.3)
Using tzinfo (0.3.38)
Using activerecord (3.2.16)
Using activeresource (3.2.16)
Using highline (1.6.20)
Using net-ssh (2.7.0)
Using net-scp (1.1.2)
Using net-sftp (2.1.2)
Using net-ssh-gateway (1.2.0)
Using capistrano (2.15.5)
Using capistrano-platform-resources (0.1.3)
Using capistrano-rbenv (1.0.5)
Using json (1.8.1)
Using carrierwave (0.9.0)
Using chronic (0.10.2)
Using coffee-script-source (1.6.3)
Using execjs (2.0.2)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using rdoc (3.12.2)
Using thor (0.18.1)
Using railties (3.2.16)
Using coffee-rails (3.2.1)
Using daemons (1.1.9)
Using delayed_job (4.0.0)
Using delayed_job_active_record (4.0.0)
Using haml (4.0.5)
Using rack-protection (1.5.1)
Using sinatra (1.4.4)
Using delayed_job_web (1.2.1)
Using dynamic_form (1.1.4)
Using exception_notification (4.0.1)
Using jquery-rails (3.0.4)
Using mysql2 (0.3.14)
Using newrelic_rpm (3.7.1.182)
Using bundler (1.5.2)
Using rails (3.2.16)
Using riddle (1.5.10)
Using rmagick (2.13.2)
Using rubypants (0.2.0)
Using rvm-capistrano (1.5.1)
Using sass (3.2.13)
Using sass-rails (3.2.6)
Using thinking-sphinx (2.0.14)
Using uglifier (2.4.0)
Using whenever (0.9.0)
Using will_paginate (3.0.5)
Your bundle is complete!
Gems in the groups development and test were not installed.
It was installed into /domains/myapp.com/shared/bundle
Alternately, I can just run bundle install
from that release directory and I get the same output. (Curiously, I get Your bundle is complete!
in green when I just run bundle update
but not when I run the longer line, though the text of the output seems identical.)
However, something is still not quite right about my bundle, because I can't get rails to work without invoking Bundler:
# rails c
The program 'rails' is currently not installed. To run 'rails' please ask your administrator to install the package 'rails'
And when I try to run rails with Bundler, it complains that it can't locate MySQL:
# be rails c
/domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2.rb:8:in `require': libruby.so.2.0: cannot open shared object file: No such file or directory - /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2/mysql2.so (LoadError)
from /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2.rb:8:in `<top (required)>'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:76:in `require'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:76:in `block (2 levels) in require'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:72:in `each'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:72:in `block in require'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:61:in `each'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler/runtime.rb:61:in `require'
from /home/deploy/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/bundler-1.5.2/lib/bundler.rb:131:in `require'
from /domains/myapp.com/releases/20140119013611/config/application.rb:7:in `<top (required)>'
from /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/railties-3.2.16/lib/rails/commands.rb:39:in `require'
from /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/railties-3.2.16/lib/rails/commands.rb:39:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
The even stranger thing about the No such file or directory - /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2/mysql2.so
error is that I can tab-complete my way down that exact path to that exact file. So it's definitely not "missing."
Any ideas what's going on here? What am I missing?
Update: I have noticed that bundle install --deployment
seems to do something else altogether, with Installing rake
etc., rather than Using rake
. Now I can run be rails c
and be rake db:migrate
and they all work. So that's great. But still my cap deploy
is failing, because it's using the first command listed above. How can I address the root cause?
Update 2: Attempting to use bundle exec cap deploy
from my local machine results in similar output. Here's the full output:
be cap deploy
triggering load callbacks
* 2014-01-19 16:46:53 executing `rbenv:setup_default_environment'
* executing "echo \"$HOME/.rbenv\""
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 487ms
* 2014-01-19 16:46:55 executing `deploy'
* 2014-01-19 16:46:55 executing `deploy:update'
** transaction: start
* 2014-01-19 16:46:55 executing `deploy:update_code'
executing locally: "git ls-remote git@bitbucket.org:username/myapp.git HEAD"
command finished in 1983ms
* executing "git clone -q git@bitbucket.org:username/myapp.git /domains/myapp.com/releases/20140120004657 && cd /domains/myapp.com/releases/20140120004657 && git checkout -q -b deploy 850a0e297ad0fd9699174d581fc96f4bafd73f07 && (echo 850a0e297ad0fd9699174d581fc96f4bafd73f07 > /domains/myapp.com/releases/20140120004657/REVISION)"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 10210ms
* 2014-01-19 16:47:08 executing `deploy:finalize_update'
triggering before callbacks for `deploy:finalize_update'
* 2014-01-19 16:47:08 executing `bundle:install'
* executing "cd /domains/myapp.com/releases/20140120004657 && env RBENV_ROOT=\"/home/deploy/.rbenv\" PATH=\"/home/deploy/.rbenv/shims:/home/deploy/.rbenv/bin:$PATH\" RBENV_VERSION=\"2.0.0-p353\" /home/deploy/.rbenv/bin/rbenv exec bundle install --gemfile /domains/myapp.com/releases/20140120004657/Gemfile --path /domains/myapp.com/shared/bundle --deployment --quiet --without development test"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 1641ms
* 2014-01-19 16:47:09 executing `whenever:update_crontab'
* executing "cd /domains/myapp.com/releases/20140120004657 && bundle exec whenever --update-crontab Ink Node --set environment=production --roles db"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
** [out :: 66.228.43.102] [write] crontab file updated
command finished in 2250ms
* executing "chmod -R -- g+w /domains/myapp.com/releases/20140120004657 && rm -rf -- /domains/myapp.com/releases/20140120004657/public/system && mkdir -p -- /domains/myapp.com/releases/20140120004657/public/ && ln -s -- /domains/myapp.com/shared/system /domains/myapp.com/releases/20140120004657/public/system && rm -rf -- /domains/myapp.com/releases/20140120004657/log && ln -s -- /domains/myapp.com/shared/log /domains/myapp.com/releases/20140120004657/log && rm -rf -- /domains/myapp.com/releases/20140120004657/tmp/pids && mkdir -p -- /domains/myapp.com/releases/20140120004657/tmp/ && ln -s -- /domains/myapp.com/shared/pids /domains/myapp.com/releases/20140120004657/tmp/pids"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 477ms
* executing "find /domains/myapp.com/releases/20140120004657/public/images /domains/myapp.com/releases/20140120004657/public/stylesheets /domains/myapp.com/releases/20140120004657/public/javascripts -exec touch -t 201401200047.12 -- {} ';'; true"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 750ms
triggering after callbacks for `deploy:update_code'
* 2014-01-19 16:47:13 executing `deploy:symlink_shared'
* executing "ln -nfs /domains/myapp.com/shared/config/database.yml /domains/myapp.com/releases/20140120004657/config/database.yml"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 464ms
* executing "ln -nfs /domains/myapp.com/shared/assets /domains/myapp.com/releases/20140120004657/public/assets"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 455ms
* executing "ln -nfs /domains/myapp.com/shared/uploads /domains/myapp.com/releases/20140120004657/public/uploads"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 440ms
* 2014-01-19 16:47:14 executing `deploy:create_symlink'
* executing "rm -f /domains/myapp.com/current && ln -s /domains/myapp.com/releases/20140120004657 /domains/myapp.com/current"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 442ms
** transaction: commit
* 2014-01-19 16:47:14 executing `deploy:restart'
* executing "touch /domains/myapp.com/current/tmp/restart.txt"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
command finished in 463ms
triggering after callbacks for `deploy'
* 2014-01-19 16:47:15 executing `deploy:migrate'
* executing "cd /domains/myapp.com/releases/20140120004657 && env RBENV_ROOT=\"/home/deploy/.rbenv\" PATH=\"/home/deploy/.rbenv/shims:/home/deploy/.rbenv/bin:$PATH\" RBENV_VERSION=\"2.0.0-p353\" /home/deploy/.rbenv/bin/rbenv exec bundle exec rake RAILS_ENV=production db:migrate"
servers: ["66.228.43.102"]
[66.228.43.102] executing command
** [out :: 66.228.43.102] rake aborted!
** [out :: 66.228.43.102] libruby.so.2.0: cannot open shared object file: No such file or directory - /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2/mysql2.so
** [out :: 66.228.43.102] /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2.rb:8:in `require'
** [out :: 66.228.43.102] /domains/myapp.com/shared/bundle/ruby/2.0.0/gems/mysql2-0.3.14/lib/mysql2.rb:8:in `<top (required)>'
** [out :: 66.228.43.102] /domains/myapp.com/releases/20140120004657/config/application.rb:7:in `<top (required)>'
** [out :: 66.228.43.102] /domains/myapp.com/releases/20140120004657/Rakefile:4:in `require'
** [out :: 66.228.43.102] /domains/myapp.com/releases/20140120004657/Rakefile:4:in `<top (required)>'
** [out :: 66.228.43.102] (See full trace by running task with --trace)
command finished in 2648ms
failed: "env PATH=/home/deploy/.rbenv/shims:/home/deploy/.rbenv/bin:$PATH RBENV_ROOT=/home/deploy/.rbenv sh -c 'cd /domains/myapp.com/releases/20140120004657 && env RBENV_ROOT=\"/home/deploy/.rbenv\" PATH=\"/home/deploy/.rbenv/shims:/home/deploy/.rbenv/bin:$PATH\" RBENV_VERSION=\"2.0.0-p353\" /home/deploy/.rbenv/bin/rbenv exec bundle exec rake RAILS_ENV=production db:migrate'" on 66.228.43.102