Displaying articles with tag ruby

Conditional Compilation of Ruby Code

Posted by rue, Tue Apr 01 23:31:00 UTC 2008

It was one of those silly ideas-nights and I decided to add the ability to have blocks of code that can be omitted from the bytecode depending on a condition as a proof-of-concept. Once the parser is in, we can start working on ‘real,’ more generic sexp and AST manipulators towards something approaching Lisp’s facilities.

I figured it would be a fairly short task and for a change I was correct&emdash;and a good portion of the thanks goes to Rubinius’ already-excellent facilities: I was able to modify the sexp with the full power of Ruby. It took me a bit to figure out the best way to set this up but in the end there are literally about 11 lines of implementation code plus the specs. The end result is a block construct for a special form that allows conditionalising. Currently I only set it up to process global variables but it is trivial to add support for the rest—what may not be trivial is for the user to get other types of conditions set up without confusion over compile-time versus runtime. The invocation is pretty simple:


  # ...

  Rubinius.compile_if($DEBUG) { Logger.debug "Blah blah: #{some_variable}" } 

  # ...

If $DEBUG is false at the time this section goes through the compiler, that entire line disappears from the first R to the last }. If it evaluates to true, the block contents are processed normally&emdash;but we still remove the now-extra wrapper block around it both to avoid the overhead as well as ensure that the variable scope etc. are consistent.

This should be useful for at least removing those generally-unnecessary debug statements etc.; the if $DEBUG does not cost much by itself but it adds up over time. The other easy target are platform/library/etc. specific sections that can be omitted in some cases. It is general-purpose in that the only limiting factor is the condition variable (or possibly expression in the future.)

The commit is here.

0 comments | Filed Under: rubinius | Tags: ruby

Splitting the Specs to a Separate Project with git Submodules?

Posted by rue, Thu Mar 13 01:47:00 UTC 2008

If we have a separate RubySpec repository which is included to Rubinius by means of git’s submodule support (mounted at spec/ruby/) there are two types of committers: those who work with RubySpec only and those who work with Rubinius only. The workflow for a RubySpec-only committer is very simple:

Clone the main RubySpec to a local working copy.
Make changes.
Push changes to RubySpec mainline or make a patch if no commit bit.

That is the end for that committer. Eventually one of the Rubinius committers will pull in the changes to the Rubinius mainline. For a Rubinius committer wanting to make a change to specs, the process is a bit different. One note to make is that having RubySpecs as a separate project imposes one extra step to the normal workflow even if not making any changes to the specs. When first cloning the rbx repo, one must run `git submodule init && git submodule update` to pull in the specs—which is obviously needed to just RUN the specs. Then, if a `git pull` contains submodule changes, another update is usually required with `git submodule update`.

Rubinius Committers

If a person working with the Rubinius repo wants to make changes or add a new spec or whatever, they can of course choose to just work on a separate RubySpec checkout. If the committer does not have a commit bit to RubySpecs, they MUST use a separate repository. That works exactly as described above.

If, on the other hand, we have an individual who is both a Rubinius and RubySpecs committer and wants to work directly in the Rubinius checkout, then the workflow looks like this:

`cd spec/ruby`
`git checkout master`
Make and commit changes. (Use branches if you like but come back to master.)
Push changes to mainline RubySpecs (just `git push`.) <—THIS IS IMPORTANT
`cd ../..`
`git add spec/ruby` <—NO TRAILING /, that overwrites the submodule
Commit.
Push to mainline Rubinius.

At this point the updated submodule becomes available to others working with the Rubinius repo using `git pull && git submodule update`.

Full Example

Here is a long example of the entire process, covering some ground that TFM did not seem to do. Please send in any corrections and/or improvements.


  # We will work from scratch to illustrate the process
  $ cd ~/code/tmp

  # OK, first set up our submodule
  $ mkdir rubyspec
  $ cd rubyspec
  $ git init
   Initialized empty Git repository in .git/
  $ echo 'rubyspec' > README
  $ git add README
  $ git commit -a -m 'Initial import of rubyspec'
   Created initial commit 4c43e66: Initial import of rubyspec
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 README
  $ cd ..

  # OK, now set up the 'public' rubyspec repo (rubyspec.com or whatever)
  $ git clone --bare rubyspec rubyspec.git
   Initialized empty Git repository in /Users/user/code/tmp/rubyspec.git/

  # Next, we create the superproject, obviously the public rbx repo
  # already exists in reality.
  $ mkdir rbx
  $ cd rbx
  $ git init
   Initialized empty Git repository in .git/
  $ echo 'rbx' > README
  $ git add README
  $ git commit -a -m 'RBX import'
   Created initial commit 43b9035: RBX import
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 README
  $ cd ..

  # OK, same deal here, make a 'public' repo (git://git.rubini.us in reality)
  $ git clone --bare rbx rbx.git
   Initialized empty Git repository in /Users/user/code/tmp/rbx.git/

  # Now, let us get rid of the originals so we have a more realistic situation
  $ rm -rf rbx rubyspec
  $ ls
   rbx.git            rubyspec.git

  # Next we want to set up the submodule link, and we need a real rbx repo
  $ git clone rbx.git my_rbx
   Initialized empty Git repository in /Users/user/code/tmp/my_rbx/.git/
  $ cd my_rbx
  $ ls
   README
  # Now we associate the submodule (in reality we would use the URI, not local path)
  $ mkdir spec
  # We can 'mount' the submodule anywhere in the repo, in this case it goes in spec/ruby/
  $ git submodule add ~/code/tmp/rubyspec.git spec/ruby
   Initialized empty Git repository in /Users/user/code/tmp/my_rbx/spec/ruby/.git/
  $ ls -la
   total 16
   drwxr-xr-x  4 user  group  204 Mar 12 19:39 .
   drwxr-xr-x  7 user  group  238 Mar 12 19:37 ..
   drwxr-xr-x  7 user  group  374 Mar 12 19:39 .git
   -rw-r--r--  1 user  group   85 Mar 12 19:39 .gitmodules
   -rw-r--r--  1 user  group    4 Mar 12 19:37 README
   drwxr-xr-x  3 user  group  102 Mar 12 19:39 spec

  # .gitmodules has the tracking info
  $ cat .gitmodules
   [submodule "spec/ruby"]
           path = spec/ruby
           url = /Users/user/code/tmp/rubyspec.git
  # Our rubyspec repo should be in place now
  $ ls -la spec/ruby
   total 8
   drwxr-xr-x  3 user  group  136 Mar 12 19:39 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:39 ..
   drwxr-xr-x  7 user  group  374 Mar 12 19:39 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:39 README
  $ cat spec/ruby/README
   rubyspec

  # Now we need to record the submodule in the superproject
  $ git status
   On branch master
   Changes to be committed:
     (use "git reset HEAD <file>..." to unstage)

         new file:   .gitmodules
         new file:   spec/ruby

  $ git commit -m 'Imported rubyspec submodule'
   Created commit 0ae5cd3: Imported rubyspec submodule
    2 files changed, 4 insertions(+), 0 deletions(-)
    create mode 100644 .gitmodules
    create mode 160000 spec/ruby

  # And then record this over at the mainline
  $ git push
   Counting objects: 5, done.
   Compressing objects: 100% (3/3), done.
   Writing objects: 100% (4/4), 439 bytes, done.
   Total 4 (delta 0), reused 0 (delta 0)
   Unpacking objects: 100% (4/4), done.
   To /Users/user/code/tmp/rbx.git
      43b9035..0ae5cd3  master -> master
  $ cd ..

  # Alright. Now, our test situation is that Joe just hacks on rubyspec and
  # Bob hacks on rubyspec as a part of rbx!

  # Let us create Bob's repo quick first:
  $ git clone rbx.git bobs_rbx
  $ cd bobs_rbx
  $ ls -la
   total 8
   drwxr-xr-x  4 user  group  204 Mar 12 19:47 .
   drwxr-xr-x  9 user  group  306 Mar 12 19:44 ..
   drwxr-xr-x  7 user  group  374 Mar 12 19:44 .git
   -rw-r--r--  1 user  group   85 Mar 12 19:44 .gitmodules
   -rw-r--r--  1 user  group    0 Mar 12 19:45 README
   drwxr-xr-x  3 user  group  102 Mar 12 19:44 spec

  # Now, as you can see, a pristine checkout of a superproject will not actually contain the submodule code
  $ ls -la spec/ruby
   total 0
   drwxr-xr-x  2 user  group   68 Mar 12 19:44 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:44 ..

  # We need to first init to properly link the modules and then update to pull the data in
  $ git submodule init
   Submodule 'spec/ruby' (/Users/user/code/tmp/rubyspec.git) registered for path 'spec/ruby'
  $ git submodule update
   Initialized empty Git repository in /Users/user/code/tmp/bobs_rbx/spec/ruby/.git/
   Submodule path 'spec/ruby': checked out '4c43e665381054478aa7c82da1d1d63b5c258e4b'
  $ ls -la spec/ruby
   total 8
   drwxr-xr-x  3 user  group  136 Mar 12 19:48 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:48 ..
   drwxr-xr-x  7 user  group  408 Mar 12 19:48 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:48 README
  # OK, now it is there. Let us hop out of here for a bit.
  $ cd ..

  # Now, Joe is doing some work on his repo which is JUST the rubyspec submodule
  $ git clone rubyspec.git joes_spec
   Initialized empty Git repository in /Users/user/code/tmp/joes_spec/.git/
  $ cd joes_spec
  $ ls
   README
  # OK, Joe makes a change
  $ echo 'new spec' > spec.rb
  $ git add spec.rb
  $ git commit -m 'Added spec.rb'
   Created commit f380258: Added spec.rb
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 spec.rb
  # Joe pushes to mainline of rubyspec
  $ git push
   Counting objects: 4, done.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 290 bytes, done.
   Total 3 (delta 0), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   To /Users/user/code/tmp/rubyspec.git
      4c43e66..f380258  master -> master

  # Now, let us check in with Bob who has his clone of the rbx repo
  $ cd bobs_rbx

  # Bob does not have Joe's spec yet
  $ ls -la spec/ruby/
   total 8
   drwxr-xr-x  3 user  group  136 Mar 12 19:48 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:48 ..
   drwxr-xr-x  7 user  group  408 Mar 12 19:48 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:48 README

  # Updating does not seem to work either!
  $ git submodule update
  $ ls -la spec/ruby/
   total 8
   drwxr-xr-x  3 user  group  136 Mar 12 19:48 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:48 ..
   drwxr-xr-x  7 user  group  408 Mar 12 19:48 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:48 README

  # Did Joe's change get registered? Let us check..
  $ cd ..
  $ git clone rubyspec.git check_joes_change
   Initialized empty Git repository in /Users/user/code/tmp/check_joes_change/.git/
  $ ls check_joes_change/
   README  spec.rb
  # Yep, it is there! So what is going on?

  # Right, we need to update the mainline rbx. Technically Bob could actually do this
  # too but for now, let us pretend that some access control allows only me to make
  # that update. So back to my repo.
  $ cd my_rbx/
  $
  # Now, we need to grab the change first
  $ cd spec/ruby/
  $ ls
   README
  $ git status
   On branch master
   nothing to commit (working directory clean)
  $ git pull
   remote: Counting objects: 4, done.
   remote: Compressing objects: 100% (2/2)remote: , done.
   remote: Total 3 (delta 0), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   From /Users/user/code/tmp/rubyspec
      4c43e66..f380258  master     -> origin/master
   Updating 4c43e66..f380258
   Fast forward
    spec.rb |    1 +
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 spec.rb
  $ ls -la
   total 16
   drwxr-xr-x  3 user  group  170 Mar 12 19:54 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:39 ..
   drwxr-xr-x  7 user  group  442 Mar 12 19:54 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:39 README
   -rw-r--r--  1 user  group    9 Mar 12 19:54 spec.rb
  # OK, there it is!
  $ cd ../../
  # So, need to record this update in the superproject
  $ git status
   On branch master
   Changed but not updated:
     (use "git add <file>..." to update what will be committed)

         modified:   spec/ruby

   no changes added to commit (use "git add" and/or "git commit -a")
  $ git add spec/ruby
  $ git commit -m 'Updated our rubyspec checkout to newest version'
   Created commit 0d96517: Updated our rubyspec checkout to newest version
    1 files changed, 1 insertions(+), 1 deletions(-)
  # And then push this change to mainline
  $ git push
   Counting objects: 5, done.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 319 bytes, done.
   Total 3 (delta 1), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   To /Users/user/code/tmp/rbx.git
      0ae5cd3..0d96517  master -> master
  $ cd ..
  $ cd bobs_rbx/
  # NOW Bob should be able to access it
  $ git submodule update
  # No? How about this:
  $ git pull
   remote: Counting objects: 5, done.
   remote: Compressing objects: 100% (2/2), done.
   remote: Total 3 (delta 1), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   From /Users/user/code/tmp/rbx
      0ae5cd3..0d96517  master     -> origin/master
   Updating 0ae5cd3..0d96517
   README: needs update
   Fast forward
    spec/ruby |    2 +-
    1 files changed, 1 insertions(+), 1 deletions(-)
  # That by itself does not update yet
  $ ls -la spec/ruby
   total 8
   drwxr-xr-x  3 user  group  136 Mar 12 19:48 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:48 ..
   drwxr-xr-x  7 user  group  408 Mar 12 19:48 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:48 README
  # Now we must update
  $ git submodule update
   Submodule path 'spec/ruby': checked out 'f38025812763dd8ee44beebcc0c8a3997c9aac0f'
  # And now it is there
  $ ls -la spec/ruby
   total 16
   drwxr-xr-x  3 user  group  170 Mar 12 19:57 .
   drwxr-xr-x  3 user  group  102 Mar 12 19:48 ..
   drwxr-xr-x  7 user  group  408 Mar 12 19:57 .git
   -rw-r--r--  1 user  group    9 Mar 12 19:48 README
   -rw-r--r--  1 user  group    9 Mar 12 19:57 spec.rb
  # And there we go, properly updated now

  # Of course, Bob can make a few changes now too. Here is the only potential problem.
  # First, change something in rbx.
  $ echo 'RTFM' > INSTALL
  $ git add INSTALL
  $ git commit -m 'Installation instructions'
   Created commit c0e6667: Installation instructions
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 INSTALL
  $ git push
   Counting objects: 4, done.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 361 bytes, done.
   Total 3 (delta 0), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   To /Users/user/code/tmp/rbx.git
      0d96517..c0e6667  master -> master
  # That was normal. Now, what if Bob wants to make changes to rubyspec?
  # He can make the changes normally. If he has commit access, he can
  # just hop over and do so.
  $ cd spec/ruby
  $ ls
   README  spec.rb
  # One notable thing is that this is a detached by default to avoid some problems, so
  # Bob must go to a real branch.
  $ git status
   Not currently on any branch.
   nothing to commit (working directory clean)
  $ git checkout master
   Previous HEAD position was b292add... Added a spec
   Switched to branch "master"
  $ echo 'my spec' > spec2.rb
  $ git add spec2.rb
  $ git commit -m 'Added a spec'
   Created commit b1e7bdb: Added a spec
    1 files changed, 1 insertions(+), 0 deletions(-)
    create mode 100644 spec2.rb
  # Now Bob can just push it if he has access
  $ git push
   Counting objects: 4, done.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 313 bytes, done.
   Total 3 (delta 0), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   To /Users/user/code/tmp/rubyspec.git
      f380258..b1e7bdb  master -> master
  # As the data shows, we directly updated the rubyspec project
  $
  # Obviously, if Bob did not have commit rights, he would do the normal git-format-patch dance instead
  $
  # Now, the problem
  $ cd ../..
  $ git status
   On branch master
   Changed but not updated:
     (use "git add <file>..." to update what will be committed)

         modified:   spec/ruby

   no changes added to commit (use "git add" and/or "git commit -a")
  # This shows the change we made--and adding it is perfectly safe so long
  # as we do it correctly. This is incorrect:
  $ git add spec/ruby/
  $ git status
   On branch master
   Changes to be committed:
     (use "git reset HEAD <file>..." to unstage)

         deleted:    spec/ruby
         new file:   spec/ruby/README
         new file:   spec/ruby/spec.rb
         new file:   spec/ruby/spec2.rb

  $ git reset --hard HEAD
   HEAD is now at c0e6667... Installation instructions
  # OK, so that is not good. The trailing / has git thinking this is a new
  # path which overwrites the submodule. So, without:
  $ git add spec/ruby
  $ git status
   On branch master
   Changes to be committed:
     (use "git reset HEAD <file>..." to unstage)

         modified:   spec/ruby

  # So that worked fine. Now we can update mainline normally
  $ git commit -m 'New spec in rubyspec, updated here and pushed to rubyspec mainline'
   Created commit 3749f4b: New spec in rubyspec, updated here and pushed to rubyspec mainline
    1 files changed, 1 insertions(+), 1 deletions(-)
  $ git push
   Counting objects: 5, done.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 331 bytes, done.
   Total 3 (delta 1), reused 0 (delta 0)
   Unpacking objects: 100% (3/3), done.
   To /Users/user/code/tmp/rbx.git
      c0e6667..3749f4b  master -> master
  $

0 comments | Filed Under: rubinius | Tags: ruby

mod_rubinius Update

Posted by rue, Wed Mar 12 05:08:00 UTC 2008

I have gotten underway with the actual coding part of mod_rubinius: as expected, quite a bit of brainpower went into trying to figure out the (hopefully) best design possible. The main focus is still ease of deployment, support for shared hosting and, of course, performance. At the simplest, `rake install` and adding the location mapping to httpd.conf followed by a restart should be sufficient to get up and running but there will be quite a few options that can be used to achieve everything we want.

One thing of note is that I will not make any attempt to integrate with MatzRuby, JRuby or any of the other implementations. We believe Rubinius has some unique features that we can take advantage of (although the first draft will certainly do so to its fullest.) The current design is founded on this premise and, as things stand, it will be impossible to use the meat of mod_rubinius with any other implementation—some parts may be useful for others too.

But that is all for now. Once I have something publishable, I will open the development process up.

0 comments | Filed Under: | Tags: ruby

Scripting iTerm with AppleScript

Posted by rue, Wed Mar 12 04:46:00 UTC 2008

After a short embarrassment with SSH I decided I needed a bit of xtermcontrol functionality. On the rest of my machines, I use it to automatically switch to a green background when in an SSH session among other things.

As a complete newb, AppleScript syntax is a bit confusing. The idea is clear, the implementation seems somewhat awkward (particularly since I saw no way to dynamically determine properties.) So I chose to use the rb-appscript abstraction, installed through Gems. It enables accessing a high-level AppleScript-equivalent in Ruby. With this, I created the initial frail and simple draft:


# ~/Scripts/iterm.rb

require 'rubygems'
require 'appscript'

include Appscript

# Map some values or just pass-through
def expand(property)
  case property
  when 'bg'
    'background_color'
  else
    property
  end
end

include Appscript
app('iTerm').current_terminal.current_session.send(expand(ARGV.first)).set(ARGV.last)

This allows, for example, changing the background color with `ruby ~/Scripts/iterm.rb bg blue` etc.

To hook it up, I set up a function in my ~/.bashrc.osx (which is sourced in ~/.bashrc obviously):


  ssh() {
    ruby ~/Scripts/iterm.rb bg blue
    command ssh "$@"
    ruby ~/Scripts/iterm.rb bg black
  }

Obviously a lot of room for improvement but it just wraps any ssh invocation by first changing the bg, then launching ssh itself with whatever arguments were given and then switching back when ssh exits (whether requested or through an error.)

Gives me a nice blue background to easily distinguish ssh sessions among my terminals. This is exceptionally handy to use for su sessions if you use those. It is much harder to accidentally leave a bright red terminal session running than a regular-looking one. (I always only use sudo anyway, but not everyone does.)

0 comments | Filed Under: mac | Tags: ruby

Installing JRuby on OS X Using the SoyLatte Java 6 JDK

Posted by rue, Mon Mar 03 01:00:00 UTC 2008

headius often kindly tries to get me to actually overcome my lingering Java allergies to try to run JRuby and now my excuse of FreeBSD amd64 not being Java-friendly has been stripped away.

In reality JRuby of course is a great project running a pretty impressive platform, the JVM, so this should be an interesting experiment. headius suggested using the SoyLatte version of JDK/JRE and looks like not without cause. It is an implementation of Java 6 based off the BSD port. So, steps taken:

Java
1. Downloaded the amd64 binary from the Soylatte site (Intel Core 2 Duo is, oddly enough, kind_of? amd64—EM64T to be precise)
2. The directory needs to be extracted somewhere, I chose /opt/local
  1. `cd /opt/local`
  2. `sudo tar xjvpf /path/to/tar.bz2`
  3. `sudo mv long_name soylatte16`
3. Set the Java environment variables (these go in my ~/.bashrc)
  1. export JAVA_HOME=/opt/local/soylatte16
  2. export PATH=/opt/local/soylatte16/bin:$PATH (front to make sure it overrides system JREs)
  3. I wanted to try NetBeans which expects certain paths: `ln -s /opt/local/soylatte16 /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home`
JRuby
1. Downloaded the 1.1b binary package
2. Extracted in /opt/local again
  1. `cd /opt/local`
  2. `sudo tar xzvpf /path/to/tar.gz`
  3. `sudo mv long_name jruby`
3. `export PATH=$PATH:/opt/local/jruby/bin` to make it visible, ~/.bashrc again
Pick up the new envs: `source ~/.bashrc`
Test: `jruby -v`
Hooray!

Now to see if I can come up with something sensible to use this for.

0 comments | Filed Under: mac | Tags: ruby

Reinventing the Which Wheel Comes First

Posted by rue, Fri Feb 15 04:49:00 UTC 2008

Aside from research, the past few days I have been putzing with reimplementing Rubinius' Array#sort and Array#sort! because, well, they sucked. The implementation is an iterative three-way partitioning Quicksort that drops down to Insertion Sort for small Arrays or subpartitions. Two observations:

The new implementation is at least one order of magnitude faster in all areas compared to the old one. Pathological cases are much improved from this, even, three orders of magnitude usually. String Arrays are about two orders of magnitude faster on small-medium Arrays and perform at the same scale with (theoretically) infinitely large ones. The old version ran out of stack space around 2000 or so.
MatzRuby is typically still far faster, but particularly the block forms are actually fairly close, in some cases actually within the same order of magnitude (albeit slower.) And the new version is pure Ruby all the way.

Here are some benchmarking figures. Note the disparity between the Array sizes.

For MatzRuby and new Rubinius: Numeric, 1000 elements, 100 loops 146233 strings, each with 20 characters For old Rubinius: Numeric, 100 elements, 100 loops 1462 strings, each with 20 characters #sort MatzRuby OLD* NEW ----------------------------------------------- sorted 0.007812 2.581782 0.602038 sorted block 0.195312 4.064099 1.782695 reversed 0.007812 4.423980 8.090656 reversed block 0.195312 4.909585 26.117707 random 0.023438 0.920588 1.643777 random block 1.914062 0.967784 3.984373 same 0.007812 0.062574 0.462366 same block 1.906250 0.915978 4.052798 two 0.007812 0.099981 0.426605 two block 1.898438 0.902365 3.756180 three 0.000000 0.118842 0.500360 three block 1.898438 0.907539 4.471767 mixed 0.015625 0.635714 1.161138 mixed block 1.859375 0.809821 3.037062 strings 0.015625 5.512026 3.742163 strings block 0.304688 10.301844 7.038649

Next mini-project will be converting Array to use a more efficient storage mechanism. That should be fun.

0 comments | Filed Under: rubinius | Tags: ruby

Platform Abstraction Musings

Posted by rue, Wed Feb 13 03:25:00 UTC 2008

The current platform abstraction in Rubinius is a bit hodgepodge--it works, but the responsibilities and hierarchies are not defined in any predictable manner. What I have been thinking about is a basic idea of a simple hierarchical model where each platform is resolved to a class that provides a complete implementation for it by inheriting and selectively overriding the specifications from its superclasses.

General idea

So, we take POSIX to be one of the top-level implementations (others could be WinNT etc.) POSIX implements the default platform that could run any theoretical operating system that did not deviate from or extend the standard in any (meaningful for us) way. Then, POSIX can be subclassed for example as Linux, which allows us to override the parts of POSIX where a Linux machine will differ or provide extra functionality. This can further be subclassed as Debian < Linux, for example, if anything like that is needed. Furthermore, we can use Ruby modules to represent cross-cutting concerns. So while we could split Linux into Linux32 and Linux64, it may be simpler to just implement Linux as a 32-bit and provide Linux64 (or even just a Generic64 for some things) as a module that can be inserted anywhere it is needed. Further, we can even divide to OS versions.


    # A 64-bit MonkeyLinux 1.8 could look like this
        POSIX
          ^
          |
        Linux
          ^
          |
     MonkeyLinux <<< Linux64
          ^
          |
    MonkeyLinux::V18

The beauty of it is, of course, that it is not necessary to go that deep. If Linux Distro X does not need to specify any behaviour deviating from the base Linux provided, then that is what is used.

Config vs. Runtime

This scheme needs to be split into two components: the build-time config such as struct and constant definitions and the runtime that uses those definitions where needed, uses FFI to attach functions etc. Ideally we would implement both of these in the same class/module hierarchy thereby reducing any mismatch problems but parallel hierarchies work too if they turn out to be considerably easier to implement.

Either way, the first step would be the build-time setup of creating platform.conf, types.conf and so on. This is already done with the tools we have (StructGenerator et al.) but we can make it easier to manage by splitting it up into the above-described hierarchy. When the time comes to build, we just resolve the correct class to use (more about this further down) and ask it to generate the config information or provide the data to do so--not sure what will turn out to be easier here yet. It would also probably be a good idea to codify the exact platform class used here so that it does not need to be re-resolved when the runtime starts. The runtime side of course largely corresponds to what you can see in kernel/platform now, with the exception that, for example, the .attach_method call that ultimately gets run is possibly the result of overrides in the descent through the class hierarchy instead of a hardcoded guess.

Resolving the Correct Platform

The only obvious problem remaining is the actual method of determining which path should be taken in the platform implementation hierarchy. Broadly, two methods can be used: registration or resolution. The former keeps some type of a matching table that selects the correct class based on the key (platform info) given. The latter, which at this point seems favourable, would instead take the approach of testing the platform info against its children's filters/resolvers and passing it further down as necessary. In this scheme, let us say that 'amd64-freebsd6.2' is passed in. The top-level Platform uses POSIX.resolve to determine it is a POSIX platform (initially this 'check' is probably platform !~ /winnt/i or something) and then POSIX peddles it to its subclasses until possibly BSD.resolve accepts it and so on until no subclass can further resolve it. Modules, of course, are taken into account in this whenever inserted.

0 comments | Filed Under: rubinius | Tags: ruby

ORM Persistence and Thinking Outside the Box

Posted by rue, Fri Feb 08 00:08:00 UTC 2008

I took a bit to completely rewrite the tiny application I had been using (and subsequently neglecting) called Oughtve. I will write more about that later, but a part of the rewrite process was to move away from the semi-binary text files I was using and into a real database. I opted to use SQLite as the backend and for ease of use and (possibly) allowing the user to select a different DB(database) at some point, I wanted to use an ORM. This turned out to be the source of never before seen frustration for me.

Initially I wanted to use DataMapper because its interface seemed to be the best of the bunch. It had an issue with namespaced model names not working (which I will ticket as soon as Lighthouse sees fit to allow me to) but the deeper problem was the inability to define the model classes without being connected to the database which is not good when you are trying to control the database setup phase. Sequel had the same problem if using models. For a brief time, I tried to use Sequel just as a “better SQL” but that got old really quick. Eventually I looked at Og which I must admit I had thought dead only to find it quite vibrant (although very poorly documented.) Og does also, indeed, allow defining the models without an existing database connection which got me past my first annoyance.

The second one really drove me up the wall, though. None of the frameworks has an easy (or any) way of exactly defining the point where the database and schema are created. The situation for destroying the database is the same, and additional problems are created by the general inability to be able to definitively drop the database connection. I went through a huge mess of attempts to be able to run my specs in some deterministic way.

I have to take a lot of the blame for the frustration myself, though. While I think that such features would be immensely useful, in truth I was only trying to do this to be able to run my specs! The setup spec actually does require being able to control the creation but in isolation it is quite possible to do (with just a little bit of stubbing to help.) All the other files would run just fine in isolation, too, but when I tried to run multiple the problems started. I am sure that you already see where I went wrong. Truthfully, I was just immensely annoyed at the entire process at this point and I think it clouded my judgement and caused me to forge ahead, pushing against the walls of the box I had built for myself (with the kind assistance of the ORM frameworks.) It took me entirely too long to put my two ducks in a row.

In the application itself, I do not need the arbitrary setup and teardown (the initial setup can be handled easily enough.)
The specs were only failing when run together—or in other words, in the same process than another spec file.

In hindsight, the solution is obvious and I feel stupid for having bashed my head against the wall for as long as I did. Is it perfect? No. Does it address the ORMs’ shortcomings? No. But it works. So I stopped worrying about it and just wrote a little spec runner of my own that still allows running multiple specs at a time, just in different processes. It is reproduced here for posterity, in case anyone ever needs a template:


#!/usr/bin/env ruby

# The spec files are designed to be run one at a time
# because it removes the necessity of completely tearing
# down the DB setup which is *exceedingly* difficult in
# any ORM.

require 'fileutils'

dummy_dir = File.expand_path "#{File.dirname(__FILE__)}/spec/dummy_dir"

FileUtils.rm_r  dummy_dir, :secure => true  rescue nil
FileUtils.mkdir dummy_dir                   rescue nil

# Split any files from options to pass to the spec program
split = ARGV.index('--')
opts  = if split
          ARGV.slice!((split + 1)..-1).join ' '
        else
          '-f p'
        end

files = if ARGV.empty? 
          Dir.glob("spec/*_spec.rb")
        else
          ARGV.pop if split
          ARGV.map {|f| "spec/#{File.basename(f)}" }
        end

begin
  real_home = ENV['HOME']
  ENV['HOME'] = dummy_dir

  files.each do |spec|
    pid = fork {
      puts "\n#{spec}:"

      exec "spec", opts, File.expand_path(spec)
    }

    Process.waitpid pid
    FileUtils.rm_r  dummy_dir, :secure => true  rescue nil
  end
ensure
  ENV['HOME'] = real_home
end

0 comments | Filed Under: | Tags: ruby

Compiler Walkthrough Illustration

Posted by rue, Fri Nov 09 19:49:00 UTC 2007

Note: this is a sort of a draft. See the real document over in the repo.


# Start
[:block, 
  [:newline, 1, "(eval)", 
    [:defn, :foo, 
      [:scope, 
        [:block, 
          [:args, [:a, :b], [], nil, nil], 
          [:newline, 1, "(eval)", 
            [:yield, 
              [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
            ]
          ]
        ], 
        [:a, :b]
      ]
    ]
  ]
]

# into_script
[:script,  
 [:block, 
   [:newline, 1, "(eval)", 
    [:defn, :foo, 
      [:scope, 
        [:block, 
          [:args, [:a, :b], [], nil, nil], 
          [:newline, 1, "(eval)", 
            [:yield, 
              [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
            ]
          ]
        ], 
        [:a, :b]
      ]
    ]
   ]
  ]
]

# Script.create strips :script
[
 [:block, 
   [:newline, 1, "(eval)", 
     [:defn, :foo, 
       [:scope, 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]
     ]
   ]
 ]
]
# out = [<script node>]


# Script.consume
# convert sexp.first ->
# Block.create comp, sexp
 [:block, 
   [:newline, 1, "(eval)", 
     [:defn, :foo, 
       [:scope, 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]
     ]
   ]
 ]
# out = [<block node>]

# Block#consume sexp.shift
 [ 
   [:newline, 1, "(eval)", 
     [:defn, :foo, 
       [:scope, 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]
     ]
   ]
 ]
# out = [<newline node>]

# Compiler#convert_sexp first
# -> Newline.create
# Newline.new.consume sexp.shift
   [1, "(eval)", 
     [:defn, :foo, 
       [:scope, 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]
     ]
   ]

# << out = [1, "eval", <defn node>]
# compiler.convert <defn node>

# Define.create
# Define.new.consume sexp.shift
# name = sexp[0] (:foo), body = sexp[1..-1]
# scope = super(body)
# ClosedScope#consume body
     [
       [:scope, 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]
     ]
# out = []

# Scope.new.consume sexp.shift
# convert args.first, 
# out = [<converted>, locals]
       [ 
         [:block, 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]
         ], 
         [:a, :b]
       ]

# Block.new.consume sexp.shift
# :args => convert
# :newline => convert

         [ 
           [:args, [:a, :b], [], nil, nil], 
           [:newline, 1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
           ]


# out = [<args node>, <newline node>] 

# Arguments.new.consume sexp.shift
[[:a, :b], [], nil, nil]

# Newline.new.consume sexp.shift
            [1, "(eval)", 
             [:yield, 
               [:call, [:lvar, :a, 0], :+, [:array, [:lvar, :b, 0]]], false
             ]
            ]

0 comments | Filed Under: rubinius | Tags: ruby

Kittensoft

Serious talk ab—oooh dust specks!! pounce

Displaying articles with tag ruby

Conditional Compilation of Ruby Code

Splitting the Specs to a Separate Project with git Submodules?

Rubinius Committers

Full Example

mod_rubinius Update

Scripting iTerm with AppleScript

Installing JRuby on OS X Using the SoyLatte Java 6 JDK

Reinventing the Which Wheel Comes First

Platform Abstraction Musings

General idea

Config vs. Runtime

Resolving the Correct Platform

ORM Persistence and Thinking Outside the Box

Compiler Walkthrough Illustration

Recently

Categories

Serious talk ab—oooh dust specks!! *pounce*

Displaying articles with tag ruby

Rubinius Committers

Full Example

General idea

Config vs. Runtime

Resolving the Correct Platform

Recently

Categories

Serious talk ab—oooh dust specks!! pounce