Having written only in general terms about how to fix the memory leak in Psych yesterday, I’ll go into a little bit more detail today.
See in Rails it really isn’t all that simple to fix it. The most obvious way to start is to add
gem 'psych' to the Gemfile. But Bundler itself requires Psych and YAML. So when the Rails boot cycle gets to executing the Gemfile, it will already be too late.
Instead it’s necessary to insert
gem 'psych' into one of the files that gets required before Bundler is required. It could be
config/boot.rb - either one of them will do.
That’s easy enough to figure out.
The real catch
If you’re using Phusion Passenger for your Rails app, the solution above will unfortunately make your app blow up when trying to dump YAML.
The reason is that psych somehow gets required before the
gem 'psych' gets run. Psych will because of this be in an inconsistent state. When starting Passenger and requesting the first page, you’ll get the following two lines in your log:
/Users/rasmus/.rvm/gems/ruby-1.9.2-p180/gems/psych-1.2.0/lib/psych.rb:93: warning: already initialized constant VERSION /Users/rasmus/.rvm/gems/ruby-1.9.2-p180/gems/psych-1.2.0/lib/psych.rb:96: warning: already initialized constant LIBYAML_VERSION
And when trying to dump some yaml, this is what you’ll see:
ArgumentError: wrong number of arguments (2 for 1) from /Users/rasmus/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/psych/nodes/node.rb:33:in `to_yaml' from /Users/rasmus/.rvm/gems/ruby-1.9.2-p180/gems/psych-1.2.0/lib/psych.rb:190:in `dump' ...
Clearly not good!
And there really is no hint in the stack trace as to where the
require 'psych' happens.
To find out where Psych was required, I opened up the psych.rb that’s bundled with Ruby and raised as the first thing in that file - et voila!
The culprit was
passenger/lib/phusion_passenger/utils.rb: 325 (I’ve added a couple of extra lines):
elsif File.exist?('Gemfile') # In case of Rails 3, config/boot.rb already calls Bundler.setup. # However older versions of Rails may not so loading boot.rb might # not be the correct thing to do. To be on the safe side we # call Bundler.setup ourselves; calling Bundler.setup twice is # harmless. If this isn't the correct thing to do after all then # there's always the load_path_setup_file option and # setup_load_paths.rb. require 'rubygems' require 'bundler' Bundler.setup end
Now how do I get around this? Don’t bother looking in the documentation for Passenger - it doesn’t say anything about this.
But fortunately there is plenty of comments in the stills.rb file. Just scroll a bit further up.
There are 3 solutions:
config/setup_load_paths.rb or a file pointed to in an option called
load_path_setup_file. However how this option is passed to the
prepare_app_process method isn’t documented and I haven’t digged around in the Passenger code long enough to know this.
The solution I like the best is anyways using the
config/setup_load_paths.rb file - this can be added to version control instead of using some Bundler dependent file and some configuration option.
If you have added
gem 'psych' to one of the files loaded in the Rails boot cycle, you don’t have to put anything in the
config/setup_load_paths.rb file. It just has to be present to stop Passenger from requiring Bundler.
Ps.: Phusion, why is
config/setup_load_paths.rb et al an undocumented feature?! I can’t be the only one who’s needed to be able to control what is loaded before Passenger starts loading Rails app stuff?
Update: I’ve also written about what to do when running tests.