Posts Tagged ‘Linux’

what you should know about rbenv and RVM

November 1st, 2011 14 comments

By now you have most likely heard of RVM and rbenv. (There is no mistake in writing one in uppercase and second in lowercase, I’m just following conventions given by their respective authors.) As a contributor in the RVM Project I thought I could find out what really rbenv does. Reading the sources gave me some insights into rbenv – unfortunately there was no new shell trick to learn for me from rbenv. Still I could understand how it works and find differences between the two tools. Both claim, and do provide, a way to switch the currently active ruby.

The main difference between them is when the ruby is switched.

  • For RVM, it’s during changing directories – the process is done once, and from now on the shell already has all ruby and gem binaries available. This can also be triggered manually.
  • For rbenv, all ruby and gem binaries are always available in shell but will load proper code only when in proper context – so switching is done each time any binary is executed.

Going into more details: RVM will take about 60ms on every called cd, which means you would need to type over 16 cd commands to take a whole second – try to do that. During the directory change, RVM will set up environment variables to reflect your ruby and gemset selection allowing it to be separated from other rubies and gems, or binaries. You will not be able to run any other binary or gem available in another project, and any auto-loading mechanisms using plugins will not be able to load them from other project environment.

For rbenv, switching directories with cd takes no time (0ms) so it is faster by default for changing directories. After installing some gems, you need to rehash and it takes 60ms. Using ruby or switching to a project dir does not change the user’s environment almost at all, except for the RBENV_VERSION variable for current – but it does not mean the env is not set at all – it will be set internally every time ruby and any other ruby or gem binary is run – with a overhead of about 50ms for each invocation.

To explain in depth what rbenv does here is a quote I was originally going to put in here but I already posted the message on reddit:

There is one big hole in shims approach – it makes all the gem and ruby binaries available in your shell … just not always functional.

Having something installed in one ruby (like haml) will make it shim available in all rubies – just not working.

Shim approach is working against the system – building another abstraction layer not respecting UNIX mechanisms – like PATH search – just fooling your system (and you).

It is not possible to run a shim/ruby without rbenv loaded – it s required to to make the environment working as expected – in contrary RVM by default builds environment files which sourced once make your system aware of the exact combination of ruby and gems you want to use.

The environment is set and provided to the ran binary in both cases, only the burden of loading it is shifted, and it means:

  • The loading place makes some difference. In RVM, you can check the state of the environment with ‘rvm info’. For rbenv, there are no changes – so there is nothing to check.
  • For RVM, only the selected ruby/gemset related binaries will be available in the shell. In rbenv, all binaries of all rubies and gems all always available in shell – only they will do nothing when called, it is not possible to check if binary exists in system as the checks always will hit the shim binary and report it as existing.
  • For those concerned about loading and execution time, you can not see the difference. It can be only measured. Anything that occurs at or below 300ms in a shell is imperceptible.

Sam Stephenson provided a list of points that make a difference between the tools, lets consider them:

  1. Need to be loaded into your shell. Instead, rbenv’s shim approach works by adding a directory to your $PATH.
    RVM also does not have to be loaded as a function. It allows operation by loading only the environment file, which is done only once and allows single and fast initialization of the current ruby and gemset. RVM also allows operation as a binary when all it’s functionalities are available (except changing environment other than setting default, which behaves identically to rbenv’s only means of operation).
  2. Override shell commands like cd. That’s dangerous and error-prone.
    Overriding of cd is optional. I searchied for almost 8 hours collectively over the last month to find a project that overrides cd – guess what, I could not find one. There are few shell tricks that allow you to override cd, but as cd override is optional, you can do it in your own function merged with the tricks. As for error prone – I see no reports of errors about the cd function. Once it is done, nothing bad can happen. Additoinally RVM provides a security feature whereby you can view the code to be executed ahead of time upon cd and choose to trust it or not.
  3. Have a configuration file. There’s nothing to configure except which version of Ruby you want to use.
    rbenv, at present, allows setting over 4 shell variables which influence the running process, there is no configuration file that could collect them – they have to be set in your shell rc files – for every shell you use.
  4. Install Ruby. You can build and install Ruby yourself, or use ruby-build to automate the process.
    During it’s life RVM collected lots of knowledge how to manage and install rubies – including patches that make ruby work as expected on different environments. Also supporting fancy things like golf ruby – where original ruby binary is replaced with goruby binary. Why would you not want to capitalize on other’s experience?
  5. Manage gemsets. Bundler is a better way to manage application dependencies. If you have projects that are not yet using Bundler you can install the rbenv-gemset plugin.
    Using gemsets is optional but encouraged as it helps in the separation process, some gems force use of bundler but this does not solve all the separation problems. And the pure existence of rbenv-gemsets proves how useful it is. Even bundler is not impervious to plethora of installed system level gems and their oddities of interaction.
  6. Require changes to Ruby libraries for compatibility. The simplicity of rbenv means as long as it’s in your $PATH, nothing else needs to know about it.
    RVM does not require changes to any gems/libraries although it allows some additions that are meant to make life easier – like per project changing PATH for bundler generated binaries, or allowing set :rvm_ruby_string to specify a ruby version instead of setting PATH like rbenv requires. All those things are optional and not required for proper work neither of RVM or the tool concerned. Finally, separation of gemsets in RVM allows skipping calls to bundle exec – which in case of rbenv will be required (unless using my gem rubygems-bundler gem or other such tricks).
  7. Prompt you with warnings when you switch to a project. Instead of executing arbitrary code, rbenv reads just the version name from each project. There’s nothing to “trust.”
    All the warnings appear only when users create the .rvmrc file manually, if it is created automatically with the –rvmrc flag then a lot less warnings are printed. rbenv actually prints a warning when switching to a project which wants to use ruby that is not installed. Trusting of .rvmrc files can be easily switched to not read them at all or to trust any .rvmrc file.

Opposed to Sam Stephenson points above I would like to give my counterarguments for using rbenv:

  1. Execution of any binary is 50ms slower as compared to binaries run in rvm – so if you are using many calls to ruby – this might get in your way of your performance.
  2. Execution of any binary does not require proper ruby/gem to be available, it will silently fail without any warning if it is not available.
  3. It is not possible to check if ruby or gem binary is available in the system, separate calls to gem, binaries or rbenv are required to find out if it is really available – which adds more work you have to do / worry about. With RVM you set environment – and LINUX conventions allow it to work as expected, with rbenv – the environment is faked with shims, you never know (by linux conventions) what will happen.

So summing up the tools are different but they allow developers to do almost exactly the same thing just with different approaches, knowing the differences could help developers chose the right tool for the right job.

Important point mentioned quite often when RVM and rbenv is compared is the size and complication of RVM. Where rbenv is a new tool, it is relatively small and its internals are less complex. rbenv uses the same approaches as RVM and is more readable if only by smaller code base which is due to the fact it is a far less mature piece of software and has far less experience accommodating different environments and platforms. It has not been extensively tested on most available operating systems. While RVM continues to press forward with new features on a solid software foundation, rbenv will continue to grow and progress but will continue to ride RVM’s coattails. It’s simple to take an existing piece of open source software and apply hindsight to create a more slim, less feature packed and less tested piece of software. We wish to see rbenv grow and expand into a great piece of software that provides a different solution to ruby environment management so both projects may learn from one another.

There is one additional thing that Is available for RVM which you will not find for rbenv – Jewelery Box – the official OS X RVM GUI. As rbenv claims to be minimalistic tool it does not seam to need any GUI – but as popularity of JB shows there is a need for such tool.

In closing, I would like to mention that we know that RVM grow big and could use some refactoring to make it easier to hack on it and to make the code base more readable and maintainable – we are working on that by planning RVM2 as an SM extension – SM is another project from Wayne E. Seguin worth checking and big enough to cover few articles. We hear our users loud and clear and have big plans for the future of RVM.

extended less usage

June 28th, 2011 Comments off

Ever used grep and wanted to check that file with less? Now it is easy with wrapping functions, they parse grep outputted lines.

So lets say we search for info in rvm packages:

grep -rn '__rvm_use(' ~/.rvm/scripts

So now you want to see what is inside of ‘.rvm/scripts/selector’ at line 391

less -N +j391 /home/mpapis/.rvm/scripts/selector

But wouldn’t be nice to just write less, double click the found line and middle click:

less /home/mpapis/.rvm/scripts/selector:391:__rvm_use

This will work when using following function:

  local params=() param parsed
  for param in "$@"
    case $param in
      parsed=( ${param//:/ } )
      params+=( "-N" "+j${parsed[1]}" "${parsed[0]}" )
      params+=( "$param" )
  command less "${params[@]}"

Put the above code into ~/.functions and source it in ~/.bashrc

test -s ~/.functions && . ~/.functions || true

Now restart your shell and enjoy you extended less function.

Categories: Development, Linux Tags: , , ,

linux server memory management

February 27th, 2011 Comments off

It has been long time since last post, but finally something useful shown up, today I want to present simple script for monitoring memory in Linux.

But first happy news: our server is now sucessfully migrated to nginx, no more apache, now maximal memory usage is 20MB per process.

So here is script, it is almost simple, find some processes, and kill them if to much RAM used:


memory=$(free -m | awk '{if (FNR==2) print $2}')
count=$(ps aux | awk -v threshold=$threshold '{if (FNR==1 || $4>=threshold) print}' | tee $log | wc -l)
[ $count -gt 1 ] && echo "Showing processes over ${threshold_mb}MB, killing processes over ${kill_at_mb}MB" && cat $log

cat $log | awk -v kill_at=$kill_at '{if ($4>kill_at) print $2" "$4 }' | while read pid mem;
used_mb=$(($(echo $mem | sed 's/\.//')*memory/1000))
# fix rounding problem in awk
if [ $used_mb -gt $kill_at_mb ]; then
kill -9 $pid
echo "pid $pid used ${used_mb}MB - killed"
rm -f $log

Most important in this script are first two variables, threshold_mb which is a limit for showing processes (it is rough comparison) and kill_at_mb which is limit for killing processes, by default this limit is 5 times bigger then showing threshold. Cause killing is more important operation additional check for used memory is more strict to be sure, only processes over the limit are killed.

Save it to disk and in crontab:

sudo crontab -e

add the following lines:

MAILTO=<your @email>
* * * * * /root/bin/

That’s all, now add some filter in your mailbox to ignore mails form server – this is all automatic, but you might want to have this for later check if some important processes are missing.

clean linux tmp

November 2nd, 2010 Comments off

Doing some security check on my laptop I found that some files are kept on disk even I do not want to – /tmp directory. Temp is kind of places in system that should not be kept on encrypted partition, so I have decided to use tmpfs. First define an /etc/fstab entry to keep /tmp on tmpfs:

tmpfs /tmp     tmpfs rw,noexec,noatime,mode=1777 0 0
tmpfs /var/tmp tmpfs rw,noexec,noatime,mode=1777 0 0

Clean all data and mount it:

rm -rf /tmp/* /var/tmp/*
mount /tmp
mount /var/tmp

Now your data is kept only till yours computer is restarted. But wait it keeps growing, I restart my laptop once few months, what then? No problem a handy script might be useful, save it as /root/bin/


: old_minutes:${old_minutes:=60}

  lsof +D "$1" 2>/dev/null | awk 'NR>1{print $9}'

  find "$1" -type f -amin +$old_minutes 2>/dev/null

  diff <(list_used "$1") <(list_old "$1") | awk '$1==">"{print $2}'

  find "$1" -type d -empty

  typeset _tmp_dir
  for _tmp_dir in "$@"
    list_unused_old "${_tmp_dir}" | xargs rm -fv
    list_empty_dirs "${_tmp_dir}" | xargs rm -fvr

clean_all /tmp /var/tmp

What it does? – remove all files older then 60 minutes and not used by any application. Now connect script in root cron (sudo crontab -e):

5 * * * * /root/bin/

Now your temp is secure and clean. You will get list of removed files on your local mail account, to prevent this email remove just v from xargs rm -fv from the script.

Categories: Linux Tags: , , , , , , ,

openSuSE – system that works

April 28th, 2009 Comments off

Recently I got irritated with my Gentoo installation, long story of problems I meet …

So I have decided to go back to openSUSE, the way was not easy, first I have tried to install from openSUSE 11.1 Live KDE CD, but it was not working with my BIOS raid setup. I was a bit upset, SUSE always gave me best working environment in short time, so I have tried with new development build 66.

This was good choice, installation was working perfect, I was also surprised by the improvements, the best example is partitioning screen which gave me feeling that my system knows what I want to archive, just have a look on the screen Raid disk layout

Comparing to Gentoo SuSE is a bit slower, the best example is start time of OpenOffice writer, on Gentoo it was up and ready to work in less then 3 seconds, on openSUSE it takes 5 seconds to show document. But this is not a big issue compared to times spent in Gentoo to configure “make.conf” or just build OpenOffice.

But the most important for me now is set of configuration tools from YAST, I can configure whole system with easy to use GUIs but still have control over my system.

From interesting features I would point out Xen working with nice GUI, will try it out in few days as it should be good alternative to VMWare Server.

On the end small explanation what does build 66 stand fore it is: “openSUSE 11.2-Milestone1“, verion 11.2 is planned for November 2009, so holpfully there will come even more interesting features.

Categories: Linux Tags: , , , , , , ,