niczsoft

Security is hard, it requires knowing your system is vulnerable, you should assume it is, but many Ruby on Rails developers seem to forget about it.

I’m writing this because a security vulnerability is promoted and it has to be stopped. The problem is adding bin, ./bin or $pwd/bin to $PATH. This is extended version of the problem with . in $PATH described here http://www.tldp.org/HOWTO/Path-12.html.

When bin or it’s variation is in $PATH, attacker can place there executable that will be executed instead of system files and gives attacker possibility to run code on your system easily. This is described better here (for the . case): http://www.dankalia.com/tutor/01005/0100501004.htm.

So why would anyone advice us to jeopardize our system? The answer is bundle exec, it is so long to write and is required to run proper versions of gems placed in Gemfile. To avoid calling it the bundler gem introduces binstubs, many developers advised adding the bin variation to $PATH to simplify calling bin/binary to just binary.

I guess you are thinking now “how is it relevant, I do review my code before running any commands”, the question is “are you?”. There is a lot of tools and extensions to shell that run commands for you, the simplest would be using PS1 to display git status, something like \u@\h:\w $(git branch) > it will execute a git command when displaying the prompt. So when someone writes bin/git into the repository – it will be executed before you can review code after git pull. The prompt and pre command hooks are very popular now and give a lot of options for attackers when bin is in $PATH.

“What should I do then?”

First, you can stop being lazy and type bin/rake or bundle exec rake this way Bundler will be loaded without the possibility to add extra code to your $PATH.

Another solution that helps to fix this problem – I wrote a gem rubygems-bundler which automates calling bundle exec, it does check if the executed binary is part of Gemfile and automatically calls Bundler.setup when needed, this eliminates the need to use binstubs wrappers.

Two weeks ago I was asked to review book “Instant Sinatra Starter”, usually I get bored by technical books so I try to avoid them, this time I thought I can give it a try.

I need to point out on beginning that my review might be biased as I am maintainer of RVM, which got very low coverage in the book. It would cut few pages of unnecessary instructions that are system specific if RVM was used in the book.

Although I do not agree with few technical explanations and encouraged practices I found the book quite good and it guided me nice through Sinatra. The book goes into deep details about explaining every aspect of working with Sinatra, starting with very simple examples and extending progressively to the point where user can build full web app based on database. With more advanced topics the instructions get more tight missing some interesting details.

The book does not only focus on building the application but also on online hosting sources and application itself so it can be shared with others – the opensource model. Unfortunately the book in some categories goes into deep detail describing something like in case of templates, I think some diversity would help a bit, like beginning with basics of ERB templates and then going into detail of Slim templates, where now it starts and ends with Slim.

I would recommend this book for people wanting to start with ruby, it gives an easy start with great effects and multiple possibilities to play around and share results with friends. As good the book is for beginners I do not think more advanced users would consider it much useful, the book does end up to early not showing a bit more advanced topics like testing or modularization. Finally as tempting it is to use very small framework any app with few database tables, authentication and authorization will require reimplementing rails – so keep it small with Sinatra, for any bigger projects pick Rails.

There is so much confusion around deploying applications using RVM, I went down and verified. The process was quite straight forward, but I found few points that could be improved.

So all my experiments were done using fresh rails app with one model, you can find sources here: https://github.com/mpapis/ad

My goal was to be able to install rails app on server without complicated recipes and doing much on server.

So the boring part first, setup server:

1. I’m assuming your domain (DNS) is configured and points to the server
2. I’m assuming you have nginx up and running, most distributions have own instructions for it, but the process should be rather easy like apt-get install nginx
3. To keep good security levels we assume one user is created per application, I used this command useradd --comment "AD - Example Rails App" -s /bin/bash -m -U ad but you might need other command depending on the system, do not forget to generate and exchange SSH keys
4. Then I had to create nginx configuration for my site /etc/nginx/sites-available/ad.niczsoft.com`` the code bellow
5. And finally link the configuration ln -s /etc/nginx/sites-available/ad.niczsoft.com /etc/nginx/sites-enabled/ad.niczsoft.com, do not forget to restart nginx

server {
  listen 80;
  server_name ad.niczsoft.com;
  access_log /var/log/nginx/ad.niczsoft.com.access.log;
  error_log /var/log/nginx/ad.niczsoft.com.error.log;
  location / {
    proxy_pass http://unix:/home/ad/app/shared/pids/unicorn.socket;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $http_host;  # this is required for HTTPS (another server section):  # proxy_set_header X-Forwarded-Proto https;  }
}

Server is at this point ready and trying to proxy incoming connections to our application. As mentioned above I have minimal rails application, so locally I have installed capistrano and generated basic setup with capify . and replaced config/deploy.rb with this https://github.com/mpapis/ad/blob/master/config/deploy.rb

We need also few gems to be added to Gemfile and then bundle installed:

group :development do
  gem 'capistrano'
  gem 'capistrano-unicorn'
  gem 'capistrano-file_db'
end

but feel free to check the whole file: https://github.com/mpapis/ad/blob/master/Gemfile

On the list is one new gem I have created to simplify deployments capistrano-file_db. It’s minimalistic gem to support deployment of file based databases (defaults to sqlite3).

I have reused also existing gem capistrano-unicorn – this one was not so straight and needed minimal hacking, but I was able to get it running in few minutes, I will be fixing the problems I found shortly, hopefully my changes get merged to master soon.

Lets go through the config/deploy.rb, the first section is pretty standard and reflects most of the configuration we did for server:

set :application, "ad"

set :deploy_to, "/home/#{application}/app"
set :user, "#{application}"
set :use_sudo, false
set :scm, :git
set :repository, "git://github.com/mpapis/ad.git"

The fun starts later, first we set ruby version to our current local selected ruby.

set :rvm_ruby_string, ENV['GEM_HOME'].gsub(/.*\//,"")

Next we tell rvm to check for remote installation in user home – as by default it checks system installation.

set :rvm_type, :user

Here we instruct rvm to install head version remotely, this line can get removed after 1.11.0 gets out, it will be defaulting back to stable.

set :rvm_install_type, :head # before rvm 1.11.0 gets released

We tell bundler to omit development dependencies, which of course are our capistrano libraries.

set :bundle_without, [:development]

And finally we instruct unicorn plugin about the proper pid path, unfortunately it was not the only place we needed to setup it, check https://github.com/mpapis/ad/blob/master/config/unicorn/production.rb for more details:

set :unicorn_pid, "#{shared_path}/pids/unicorn.pid"

That’s almost it, now lets tell capistrano where the deploy, and to always do migrations:

server "niczsoft.com", :app, :web, :db, :primary => true

before 'deploy:restart', 'deploy:migrate'

This two steps are important, they tell capistrano to install rvm and ruby on server, it will also create application gemset if used, it will use rvm_ruby_string, so make sure to set it to something useful like `rvm current`:

before 'deploy:setup', 'rvm:install_rvm'
before 'deploy:setup', 'rvm:install_ruby'

And on the end all the capistrano plugins we want to use:

require "rvm/capistrano"
require "bundler/capistrano"
require "capistrano-unicorn"
require "capistrano-file_db"
load 'deploy/assets'

For the deploy/assets to work we have gem 'therubyracer' in Gemfile.

That’s all, let deploy our application, first prepare server

cap deploy:setup

cap deploy:cold

this will take some time so grab a tee/coffee.

After server is prepared we can deploy application after every commit to repository:

cap deploy

And it’s the end of the journey, it takes a lot longer to write this tutorial then to just do the deployment, doing it second time try to fit in 10 minutes !

You can get more details on RVM / Capistrano integration in the freshly updated documentation page: http://beginrescueend.com/integration/capistrano/ also worth reading is the new project files not requiring trusting: http://beginrescueend.com/workflow/projects/#ruby-versions

As a proof of concept I created an super action that allows to process more than one action at once.
Probably most of you will ask a question ‘why’ if there is an posibility to use nested resources. Yes you can, but what if there are more different actions that do not have common root?

Code bellow is only proof of concept, it was tested only using integration tests, if you ever try it more please let me know.

First Step is to create middleware scaffold ‘lib/super_action.rb’:

class SuperAction
  include Rack::Utils

  def initialize(app)
    @app = app
  end

  def call(env)
    @app.call(env)
  end
end

Next we have to connect it, we can not do it as advised in environment file, we also do not want to change the whole stack, the way to go is to hook into application loading process soewhere after it initilaizes all automatic middlewares. Add the following code (module) into ‘config/environment.rb’:

...
require File.join(File.dirname(__FILE__), 'boot')

module Rails
  class Initializer
    alias :old_load_application_classes :load_application_classes
    def load_application_classes
      configuration.middleware.delete ActiveRecord::QueryCache
      configuration.middleware.delete ActiveRecord::ConnectionAdapters::ConnectionManagement
      configuration.middleware.insert_before ActionController::ParamsParser, 'SuperAction'
      configuration.middleware.insert_before 'SuperAction', ActiveRecord::ConnectionAdapters::ConnectionManagement
      configuration.middleware.insert_before 'SuperAction', ActiveRecord::QueryCache
      old_load_application_classes
    end
  end
end

Rails::Initializer.run do |config|
...

We can not do the middleware changes directly in initializer using config.middleware because then it raises exception NameError on ‘ActiveRecord::QueryCache’ – it is loaded in the initialization stack after environment. So this is the current stack looks now for me:

~/projects/super_action> rake middleware
use Rack::Lock
use ActionController::Failsafe
use ActionController::Session::CookieStore, #
use ActiveRecord::ConnectionAdapters::ConnectionManagement
use ActiveRecord::QueryCache
use SuperAction
use ActionController::ParamsParser
use Rack::MethodOverride
use Rack::Head
use ActionController::StringCoercion
run ActionController::Dispatcher.new

Important here was to be before ActionController::ParamsParser, because it is responsible for providing correct params to action. Putting ConnectionManagement and QueryCache before super action should be considered as an optimization.

When we have correct order of middlewares now the super action functionality, back to ‘lib/super_action.rb’:

class SuperAction
  include Rack::Utils

  def initialize(app)
    @app = app
  end

  def call(env)
    uri  = env['REQUEST_URI'] || env['PATH_INFO']
    wrapper = env['rack.input']
    body = wrapper.read
    wrapper.rewind

    unless body.blank?
      params, format = if uri == '/super_action.xml'
        [Hash.from_xml(body), :xml]
      end
    end

    first_response = nil

    if params
      response_array = params['requests'].map do |req|
        req_env = env.clone
        req_env['PATH_INFO']      = req['url']
        req_env['REQUEST_URI']    = req['url']
        req_env['REQUEST_METHOD'] = req['method']
        req_env['CONTENT_TYPE']   = req['content_type']
        req_env['rack.input']     = Rack::Lint::InputWrapper.new StringIO.new( req['body'] || '' )
        status, headers, req_response = @app.call(req_env)
        first_response ||= req_response
{
  :url => req['url'],
  :method => req['method'],
  :status => status,
  :body => req_response.body,
  :id => req['id']
}
      end

      response_body = case format
      when :xml then 
        xml = Builder::XmlMarkup.new
        xml.responses 'type' => 'array' do
          response_array.each do |resp|
            xml.response do
              xml.url resp[:url]
              xml.method resp[:method]
              xml.id resp[:id], 'type' => 'integer'
              xml.body do
                xml.cdata! resp[:body]
              end
            end
          end
        end
      end

      header = HeaderHash.new
      header['Content-Type'] = 'application/xml; charset=utf-8'
      header['Content-Length'] = response_body.size.to_s
      header['Cache-Control'] = 'private, max-age=0, must-revalidate'

      first_response.instance_variable_set(:@body, response_body)
      [ 200, header, response_body]
    else
      @app.call(env)
    end
  end
end

I know the code looks tricky, it realy is, but finally it allowed me to run the following test:

  test "update many books" do
    Book.delete_all

    start = Time.now

    xml = Builder::XmlMarkup.new
    xml.requests 'type' => 'array' do
      xml.request do
        xml.url '/books.xml'
        xml.method 'POST'
        xml.id 0
        xml.content_type 'application/xml'
        xml.body do
          xml.cdata! '<title>book2</title>'
        end
      end
      xml.request do
        xml.url '/books.xml'
        xml.method 'POST'
        xml.id 1
        xml.content_type 'application/xml'
        xml.body do
          xml.cdata! '<title>book3</title>'
        end
      end
      xml.request do
        xml.url '/books.xml'
        xml.method 'GET'
        xml.id 2
      end
    end

    assert_difference "Book.count", 2 do
      post '/super_action.xml', xml.target!
    end
    responses = Hash.from_xml( response.body )['responses']

    stop = Time.now
    puts "separate:#{stop-start}:"

y [ xml.target!, response.body ]

    assert_response :success
    assert_equal 3, responses.size
    responses.sort! { |a,b| a['id']  b['id'] }

    assert_equal 0, responses[0]['id']
    assert_equal 'book2', Hash.from_xml(responses[0]['body'])['book']['title']

    assert_equal 1, responses[1]['id']
    assert_equal 'book3', Hash.from_xml(responses[1]['body'])['book']['title']

    assert_equal 2, responses[2]['id']
    assert_equal 'book2', Hash.from_xml(responses[2]['body'])['books'][0]['title']
    assert_equal 'book3', Hash.from_xml(responses[2]['body'])['books'][1]['title']
  end

Speed up in integration tests is over 2x on three actions, this makes this something considerable … only if the code would be not so tricky.

Some time ago I wrote about complex associations, now time to add another method and corrections.

First the finder_by_sql, in that particular case It was necessary to add

:readonly => true

So the code looks now like this:

  has_many :roles, 
    :readonly => true, 
    :finder_sql => '
SELECT roles.name FROM roles
INNER JOIN responsibilities ON roles.id = responsibilities.role_id
INNER JOIN assigments ON responsibilities.group_id = assigments.group_id
WHERE assigments.user_id = #{id}
GROUP BY roles.id
  '

There is one realy big downside of using finder_sql – it does not work with find_by_… or named scopes, so this forced me to continue searching and this is the result:

  def roles
    Role.scoped(
     {
       :joins => { :responsibilities => { :group => { :assigments => :user } } },
       :conditions => {"users.id" => id},
       :group => "roles.id"
     }
   )
  end

and now I can write:

user.roles.by_name(:admin).count

where by_name is an named scope

  named_scope :by_name, lambda { |type| {:conditions => [ "roles.name = ", type.to_s ] } }