Tomas Varsavsky

I was fortunate enough to go to the JAOO Sydney conference this week. I attended the first Australian JAOO in Brisbane last year which brought out a variety of international heavy hitters so my expectations were high. JAOO did not disappoint, it was a very well put together conference and I had a great time participating in a variety of technical and process related talks presented by local and international speakers. I came back energised and with a long run sheet of technologies and ideas to explore over the next few months.

The highlights for me were:

• Mike Cannon-Brookes‘ talk on the great Aussie success story that is Atlassian. It truly is great what a couple of smart young guys were able to achieve in such a short time. I liked the idea Fed Ex days, a quarterly event where developers are encouraged to flex their creative muscle and ’ship’ working code within 24 hours. I also liked what they’ve done to measure productivity loops within their software development teams (e.g. code change to page refresh time, commit to build feedback time). As a consultant, I often struggle to explain to management the real cost of unproductive tools, technology and process. This sounds like a good way to go about it. On the downside, I disagree with what seems an overly formal code review process. Good teams with smart people that pair and rotate frequently should not need so much emphasis on code review.
• Both Avi Bryant’s talks. The first one was about how to improve the Ruby’s inherent performance. It was a deeply technical talk which mostly went over my head because I don’t operate in that world but it made me wish I did. His second talk followed the creative process they’ve been through to build their latest product. It really showed that a creative, iterative software development process does in the end produce a better product than careful specification.
• Pamela Fox’s talk on Google App Engine. She made what could have been a boring feature walkthrough presentation into a greatly entertaining experience by building the Best Website Ever before our eyes, complete with frames, marquee tags, animated gifs, web ring, pumping techno and spam!
• James Ward’s walkthrough of Adobe Flex was a surprising delighter. I have a suspicion that when building rich client applications we’re pushing html, css and javascript to extremes that they were never designed to go and we are feeling much development pain as a consequence. Could Flex or Silverlight be the answer? I don’t know.

I didn’t go to see talks given by ThoughtWorkers (I’ve seen it all before) but by all accounts Dan North’s Pimp ‘My Architecture’ and Erik Doernenburg’s ‘Builds: From Good to Great’ were both excellent.

I understand that JAOO had some trouble with attendance this year due to many companies cutting back on discretionary spending. It’s a shame, conferences are the best bang-for-buck when it comes to career development. Hopefully the organisers won’t be discouraged by this and come back next year for another great round. Even better, maybe they’ll consider doing a tour of duty through Melbourne!

In my current project we have a requirement to merge simple data into Microsoft Word document templates. Ruby comes with the WIN32OLE library which can manipulate Office documents. WIN32OLE has a few major downsides — it only runs on Windows, it requires Microsoft Office to be installed and it works by sending commands to Word itself to perform operations. Using Word as a back end system for a web application used by 50 people made us nervous so a different approach was needed. We came up with a combination of Ruby, Office Open XML file format, XML processing with Nokogiri and native Zip libraries that works.

Office Open XML file formats

The new Office file formats (.docx, .xlsx, .pptx files) are basically a zipped collection of XML files. We focused on Word files (.docx) but this approach would work with any of the other types of files as well. The specification for the format weighs in at several thousand pages. Producing a file from scratch without a purpose built library that handles all the intricacies of the format would be quite a task. Instead, we drafted the templates in Word and placed markers to tell our templating engine where to insert values. We created document properties which reference data values and added these as fields into the document in the place where the values should be inserted. For example, we could have fields like:

• label_tag #{data[:user].name}
• label_tag #{data[:user].address}
• label_tag #{data[:booking].number}
• label_tag #{data[:booking].items.collect{|i| i.name}.join(’,')}

If it looks a bit like Ruby code, it’s because it is! The expressions get evaluated by our templating engine and the results are inserted into the document. Ruby in Word documents, a world first?

Opening the documents

To read and create documents we need to unzip and re-zip the document. We had trouble using Ruby’s standard RubyZip library. For some reason Word gave a nasty warning when opening files created with RubyZip. Our application has to run on Windows, Linux and Mac so we created an adapter that delegated to standard operating system zip executables based on the host platform. To keep it fast, we extract and re-added only the files that we need to work on. This is important because some documents can become very large when they contain embedded objects such as images.

Processing the template

The document content can be found in the file word/document.xml inside the zip archive. The fields in the template come out as fldSimple tags that look like this:

To process the document.xml we simply need to find all the fields that have the text label_tag in the w:instr attribute:

The rest is simple. We extract the expression in the element text using a regular expression, evaluate it and insert it back into the XML which ends up looking like this:

We add the attribute fldLock with value true to make the field read-only so the user cannot change it when they open the document.

We also have tags to create lists, insert rows into tables and duplicate sections in the document. These are a bit more complicated in their XML manipulation. Beware, we had a few issues dealing with Word’s nasty XML which can vary a bit between versions and sometimes do unexpected things with formatting.

Conclusion

This approach worked really well for us and I would recommend it for simple field merging.

Ruby development was always meant to be done on a *nix platform. For a while now I’ve been putting up with the Windows XP / Cygwin combination on my laptop to give me a pseudo-*nix environment. While this is a workable compromise, it is far from ideal. A lot of the Ruby tools don’t work well on Cygwin: crashes and mysterious hangings are common, execution times are slow and frustration is high.

The common solution to this problem is a walk to the closest Apple shop to get a shiny new Mac, but I’m not cool enough for a Mac so I decided to try something more befitting my coolness level: A Linux virtual machine running on my Dell laptop.

These are the steps you need to get a Ubuntu Virtual Machine running Rails over MySQL on your Windows machine.

Step 1: Setup a Virtual Machine (VM)

First we need to get a virtual machine configured and running with a *nix Operating System. I’ve chosen Ubuntu Server because all I need is shell access. You can use Ubuntu Desktop if you plan to use the VM as your development desktop.

1. Download the latest version of VMware Server or VMware Player. VMware Player is a lighter download but can only operate VMs, VMware server can operate and create VMs. Either will work for our purposes, I used VMware Server.
2. Download and unzip the latest Ubuntu Server VMware image. I used version 7.10 Gutsy Gibbon.
3. Start VMware Server, point it at the downloaded image and start it.
4. The virtual machine will start up and should come up with a login prompt. Hit Enter a couple of times on the console if it doesn’t.

Step 2: Get access

In this step we create a user on the VM and configure it for external shell access through SSH.

1. Login using the username “notroot” and the password “thoughtpolice”.
2. Install OpenSSH.

> sudo apt-get install ssh

3. Create yourself a user.

> sudo adduser –shell /bin/bash <user_name>

4. Add the user to the admin group (so they can do sudo, you can play around with sudoedit instead if you like).

> sudo usermod -g admin <user_name>

5. Get the IP address of the virtual machine.

> sudo ifconfig eth0

6. Fire up your favourite terminal program (I use PuTTY) and SSH to the VM using the user you created above.

Step 3: Install Ruby, Ruby on Rails and MySQL

Now we install some core development packages.

1. Update the package index.

> sudo apt-get update

2. Install Ruby and related tools.

> sudo apt-get install ruby rubygems irb ri rdoc ruby1.8-dev build-essential

3. Install Ruby on Rails.

> sudo apt-get install rails

4. Install MySQL.

> sudo apt-get install mysql-server

Step 4: Create a Rails project and fire it up

We have everything we need, let’s create a Rails project and access it from Windows.

1. Create a Rails project in your home directory.

> cd ~/projects
> rails killerapp
> cd killerapp

2. Create a database.

> mysqladmin -u root -p create killerapp_development

3. Configure the database in your Rail application config/database.yml.
4. Start the WEBrick server. You have to bind it to the external IP address (not localhost) to make it available though windows.

> ./scripts/server -b <vm_ip_address> &

5. On your favourite Windows browser, navigate to: http://<vm_ip_address>:3000/.The default Rails landing page should come up! You can add this IP address to the Windows hosts file to avoid having to remember it.

Step 5: Set up Samba (optional)

It’s useful to be able to share files between the VM and Windows, especially if you plan to use a Windows based IDE like Eclipse to craft your code. VMware has a “Shared Folders” feature but I gave up trying to get it to work after several hours of bashing my head against Google. I reverted back to Samba, which is trivial to configure.

1. Install Samba.

> sudo apt-get install samba

2. Add the user to the smbpasswd file.

> sudo smbpasswd -a <user_name>

3. Open the Samba configuration file for editing.

> sudo vi /etc/samba/smb.conf

4. Remove the ‘;’ from the line that says “security = user,”.
5. Add the following to the end of the file.

[<user_name>]
path = /home/<user_name>
valid users = <user_name>
read only = No
create mask = 0777
directory mask = 0777

6. Save and close the file.
7. Restart Samba.

> sudo /etc/init.d/samba restart

8. On Windows Explorer, Map Network Drive \\<vm_ip_address>\<user_name>, login using “<user_name>”.

That’s it! I haven’t used this environment in anger yet so there are bound to be some holes, let me know if you fall into any.

Steve Yegge (one of my favourite bloggers) recently posted about the maintenance problems exhibited by a large Java code base. I agree that software bloat is one of the biggest problems in our industry right now.

His proposed solution is to re-implement the 500,000 lines of Java code into 150,000 lines of a more concise language such as Ruby or JavaScript (ES4). This approach seems misguided to me because software complexity is not directly proportional to number of lines of code. The fact that 5 lines of Java can sometimes be compressed into 1 line of Ruby does not make the logic less complex or easier to maintain if they are essentially doing the same thing. In fact, I would argue that often 1 line of clever Ruby is actually less maintainable that 5 lines of Java because it requires a deeper understanding of what’s going on underneath the covers.

A good case study is Perl where developers seem to be a mind set of “how clever can I be” and write one liners that become indecipherable even by the author within days. You might be able to get away with this if you’re a gun developer working on your own but not if you’re working in an average team of average developers. I think Ruby has actually struck a nice balance between readability and conciseness and this is one of the reasons why its popularity has surged. Still, the thought of a 150,000 line Ruby code base scares me and would keep me up at night if it was left in the hands of average developers.

A code base with 1000 Java classes is just a complex as a code base with 1000 equivalent Ruby classes even if the number of lines per class is less. Reducing complexity and therefore increasing maintainability is all about applying good software engineering principles and frequent doses of my favourite refactoring — Delete.

In my current project we have some deployment scripts written in Ant. We can deploy a new version of the application as built by CruiseControl by running the command “ant deploy-war -Dwarfile=http://cruise-box/artifacts/killer_app/BUILD_TIMESTAMP/killer_app.war“.

Most of the time we want to deploy the latest successful build. We manually get the timestamp of the build from the CruiseControl dashboard and substitute it into the command above. It would be nice to default the warfile property to the latest successful build. For that we need CruiseControl to give us a well known link to the latest successful build.

I solved this problem using symbolic links. We can do this because our CruiseControl server runs on Linux but we could have achieved a similar result with a file system copy or an artifact publisher under Windows. I created the following helper Ant script to create a symbolic link using the Symlink task.

I then added a call to the helper script in the CruiseControl config.xml as an Ant publisher executed after a successful build.

I had to do it in an Ant script rather than calling the ln command directly with an execute publisher because I couldn’t work out how to access the build timestamp from within CruiseControl’s config.xml.

We can now access the latest successful build through the URL http://cruise-box/artifacts/killer_app/latest-successful/killer_app.war. The last step is to default the warfile property in our deployment scripts to this URL. We can now run “ant deploy-war” to deploy the latest successful build, or provide the property as before to deploy a different build.