Tomas Varsavsky

I was fortunate enough to go to the JAOO Sydney conference this week. I attended the first Australian JAOO in Brisbane last year which brought out a variety of international heavy hitters so my expectations were high. JAOO did not disappoint, it was a very well put together conference and I had a great time participating in a variety of technical and process related talks presented by local and international speakers. I came back energised and with a long run sheet of technologies and ideas to explore over the next few months.

The highlights for me were:

• Mike Cannon-Brookes‘ talk on the great Aussie success story that is Atlassian. It truly is great what a couple of smart young guys were able to achieve in such a short time. I liked the idea Fed Ex days, a quarterly event where developers are encouraged to flex their creative muscle and ‘ship’ working code within 24 hours. I also liked what they’ve done to measure productivity loops within their software development teams (e.g. code change to page refresh time, commit to build feedback time). As a consultant, I often struggle to explain to management the real cost of unproductive tools, technology and process. This sounds like a good way to go about it. On the downside, I disagree with what seems an overly formal code review process. Good teams with smart people that pair and rotate frequently should not need so much emphasis on code review.
• Both Avi Bryant‘s talks. The first one was about how to improve the Ruby’s inherent performance. It was a deeply technical talk which mostly went over my head because I don’t operate in that world but it made me wish I did. His second talk followed the creative process they’ve been through to build their latest product. It really showed that a creative, iterative software development process does in the end produce a better product than careful specification.
• Pamela Fox‘s talk on Google App Engine. She made what could have been a boring feature walkthrough presentation into a greatly entertaining experience by building the Best Website Ever before our eyes, complete with frames, marquee tags, animated gifs, web ring, pumping techno and spam!
• James Ward‘s walkthrough of Adobe Flex was a surprising delighter. I have a suspicion that when building rich client applications we’re pushing html, css and javascript to extremes that they were never designed to go and we are feeling much development pain as a consequence. Could Flex or Silverlight be the answer? I don’t know.

I didn’t go to see talks given by ThoughtWorkers (I’ve seen it all before) but by all accounts Dan North‘s Pimp ‘My Architecture’ and Erik Doernenburg‘s ‘Builds: From Good to Great’ were both excellent.

I understand that JAOO had some trouble with attendance this year due to many companies cutting back on discretionary spending. It’s a shame, conferences are the best bang-for-buck when it comes to career development. Hopefully the organisers won’t be discouraged by this and come back next year for another great round. Even better, maybe they’ll consider doing a tour of duty through Melbourne!

In my current project we have a requirement to merge simple data into Microsoft Word document templates. Ruby comes with the WIN32OLE library which can manipulate Office documents. WIN32OLE has a few major downsides — it only runs on Windows, it requires Microsoft Office to be installed and it works by sending commands to Word itself to perform operations. Using Word as a back end system for a web application used by 50 people made us nervous so a different approach was needed. We came up with a combination of Ruby, Office Open XML file format, XML processing with Nokogiri and native Zip libraries that works.

Office Open XML file formats

The new Office file formats (.docx, .xlsx, .pptx files) are basically a zipped collection of XML files. We focused on Word files (.docx) but this approach would work with any of the other types of files as well. The specification for the format weighs in at several thousand pages. Producing a file from scratch without a purpose built library that handles all the intricacies of the format would be quite a task. Instead, we drafted the templates in Word and placed markers to tell our templating engine where to insert values. We created document properties which reference data values and added these as fields into the document in the place where the values should be inserted. For example, we could have fields like:

• label_tag #{data[:user].name}
• label_tag #{data[:user].address}
• label_tag #{data[:booking].number}
• label_tag #{data[:booking].items.collect{|i| i.name}.join(‘,’)}

If it looks a bit like Ruby code, it’s because it is! The expressions get evaluated by our templating engine and the results are inserted into the document. Ruby in Word documents, a world first?

Opening the documents

To read and create documents we need to unzip and re-zip the document. We had trouble using Ruby’s standard RubyZip library. For some reason Word gave a nasty warning when opening files created with RubyZip. Our application has to run on Windows, Linux and Mac so we created an adapter that delegated to standard operating system zip executables based on the host platform. To keep it fast, we extract and re-added only the files that we need to work on. This is important because some documents can become very large when they contain embedded objects such as images.

Processing the template

The document content can be found in the file word/document.xml inside the zip archive. The fields in the template come out as fldSimple tags that look like this:

Template Field: User Name

To process the document.xml we simply need to find all the fields that have the text label_tag in the w:instr attribute:

xml.xpath(“//w:fldSimple[contains(@w:instr, 'label_tag')]“).each do |element|
# process each element here
end

The rest is simple. We extract the expression in the element text using a regular expression, evaluate it and insert it back into the XML which ends up looking like this:

Tomas Varsasvky

We add the attribute fldLock with value true to make the field read-only so the user cannot change it when they open the document.

We also have tags to create lists, insert rows into tables and duplicate sections in the document. These are a bit more complicated in their XML manipulation. Beware, we had a few issues dealing with Word’s nasty XML which can vary a bit between versions and sometimes do unexpected things with formatting.

Conclusion

This approach worked really well for us and I would recommend it for simple field merging.

Last weekend I had a blast attending my first BarCamp held at the ThoughtWorks office in Melbourne. The concept is powerfully simple: get a bunch of like minded people together and interesting conversations will ensue. The attendees represented a cross section of disciplines in the IT industry that you don’t usually find in technology specific user groups, producing a variety of interesting presentations.

Many thanks to Ben and the team for organising, I’ll be back next year.

I keep a list of “interesting things to look sometime maybe” which I add to when I come across something that I should learn more about. This list has grown over the last few months because I’ve been busy with client work, the birth of my daughter, landscaping my back yard and <insert more excuses here>. But today is my lucky day! I finished a client assignment last week and I have a few days up my sleeve before the next one starts.

Here are the high priority items on the list as it stands now.

• Scala
• Erlang
• RMock
• RSpec Story Framework
• Wicket
• Seam
• Ruby 1.9 new features
• Virtualisation with VMware and VirtualBox
• Git and Mercurial
• Read some of the books collecting dust on my bookshelf (Lean Thinking, User Stories Applied, Agile Estimating and Planning, Blink, The Tipping Point, Freakonomics)

Let’s see how many I can get through before consulting reality spoils the party.

Ruby development was always meant to be done on a *nix platform. For a while now I’ve been putting up with the Windows XP / Cygwin combination on my laptop to give me a pseudo-*nix environment. While this is a workable compromise, it is far from ideal. A lot of the Ruby tools don’t work well on Cygwin: crashes and mysterious hangings are common, execution times are slow and frustration is high.

The common solution to this problem is a walk to the closest Apple shop to get a shiny new Mac, but I’m not cool enough for a Mac so I decided to try something more befitting my coolness level: A Linux virtual machine running on my Dell laptop.

These are the steps you need to get a Ubuntu Virtual Machine running Rails over MySQL on your Windows machine.

Step 1: Setup a Virtual Machine (VM)

First we need to get a virtual machine configured and running with a *nix Operating System. I’ve chosen Ubuntu Server because all I need is shell access. You can use Ubuntu Desktop if you plan to use the VM as your development desktop.

1. Download the latest version of VMware Server or VMware Player. VMware Player is a lighter download but can only operate VMs, VMware server can operate and create VMs. Either will work for our purposes, I used VMware Server.
2. Download and unzip the latest Ubuntu Server VMware image. I used version 7.10 Gutsy Gibbon.
3. Start VMware Server, point it at the downloaded image and start it.
4. The virtual machine will start up and should come up with a login prompt. Hit Enter a couple of times on the console if it doesn’t.

Step 2: Get access

In this step we create a user on the VM and configure it for external shell access through SSH.

1. Login using the username “notroot” and the password “thoughtpolice”.
2. Install OpenSSH.

> sudo apt-get install ssh

3. Create yourself a user.

> sudo adduser –shell /bin/bash <user_name>

4. Add the user to the admin group (so they can do sudo, you can play around with sudoedit instead if you like).

> sudo usermod -g admin <user_name>

5. Get the IP address of the virtual machine.

> sudo ifconfig eth0

6. Fire up your favourite terminal program (I use PuTTY) and SSH to the VM using the user you created above.

Step 3: Install Ruby, Ruby on Rails and MySQL

Now we install some core development packages.

1. Update the package index.

> sudo apt-get update

2. Install Ruby and related tools.

> sudo apt-get install ruby rubygems irb ri rdoc ruby1.8-dev build-essential

3. Install Ruby on Rails.

> sudo apt-get install rails

4. Install MySQL.

> sudo apt-get install mysql-server

Step 4: Create a Rails project and fire it up

We have everything we need, let’s create a Rails project and access it from Windows.

1. Create a Rails project in your home directory.

> cd ~/projects
> rails killerapp
> cd killerapp

2. Create a database.

> mysqladmin -u root -p create killerapp_development

3. Configure the database in your Rail application config/database.yml.
4. Start the WEBrick server. You have to bind it to the external IP address (not localhost) to make it available though windows.

> ./scripts/server -b <vm_ip_address> &

5. On your favourite Windows browser, navigate to: http://<vm_ip_address>:3000/.The default Rails landing page should come up! You can add this IP address to the Windows hosts file to avoid having to remember it.

Step 5: Set up Samba (optional)

It’s useful to be able to share files between the VM and Windows, especially if you plan to use a Windows based IDE like Eclipse to craft your code. VMware has a “Shared Folders” feature but I gave up trying to get it to work after several hours of bashing my head against Google. I reverted back to Samba, which is trivial to configure.

1. Install Samba.

> sudo apt-get install samba

2. Add the user to the smbpasswd file.

> sudo smbpasswd -a <user_name>

3. Open the Samba configuration file for editing.

> sudo vi /etc/samba/smb.conf

4. Remove the ‘;’ from the line that says “security = user,”.
5. Add the following to the end of the file.

[<user_name>]
path = /home/<user_name>
valid users = <user_name>
read only = No
create mask = 0777
directory mask = 0777

6. Save and close the file.
7. Restart Samba.

> sudo /etc/init.d/samba restart

8. On Windows Explorer, Map Network Drive \\<vm_ip_address>\<user_name>, login using “<user_name>”.

That’s it! I haven’t used this environment in anger yet so there are bound to be some holes, let me know if you fall into any.