The Measure of a Design

We all want to write good software rather than bad software. What makes software good? I am not going to try and answer that question today. Instead I am going to talk about something that I've observed over and over in my software career - How can you judge whether software is designed well?

how easy is it to change the software?Note, I am not talking about whether the software is good or not. The ultimate metric of that is, does it solve the problem it is supposed to. My question is, is the architecture and design good? The empirical measure of that is, when new requirements come up, how easy is it to change the software?

Ease of changing software consists of various factors: how much code has to be written/changed, how easy is it to find the appropriate place(s) to change the code, and how confident can you be that the changes didn't break other functionality.

How Much Code
This measure isn't about absolute lines of code. It's about lines of code relative to expectations. If you are adding complex functionality, you'd expect it to be a decent amount of code. On the other hand, if you are just fixing a typo in an output, you'd expect that to be a really small amount of work.

There was one project I worked on that included a report of employees and their salaries. A request came in that the report have the employees sorted by salary. My expectation for this change is it should just be a matter of putting a sort call in the appropriate place, and possibly having to write a custom comparator. I.e. a really quick change. In reality, this change required some major reworking, and had to be delayed because it was going to take too long to accomplish.

When simple changes take a lot of work, this is evidence that your design is less than ideal.

Appropriate Place
When writing software, most tasks can be accomplished in a myriad of ways. A good architecture will lead you to choose one way over another. When the architecture guides you to a specific place for putting in new features, this is a good thing. Not only does it help with consistency, it means that when your new feature has to be changed, the next developer will know exactly where to go to find what you did, and know where to make the change.

This is something I really like about Ruby on Rails. It is an opinionated framework which sometimes feels very limiting, but means that most times I know exactly where to go find the code that I am looking for.

Unexpected Changes
One fear that is common when changing existing software is that you'll break something unintentionally. A hallmark of a good design is that this rarely happens. Well structured code with clean decoupled interfaces can go a long way to giving you confidence that a new change won't have unintended consequences. Yes, having a large suite of automated tests is really helpful (whether or not your design is good), but no test suite is completely thorough. I find that when I am making changes to code, I usually have a pretty good feel of how risky the change is. When most changes feel risky, this is a sign that the design can be improved.

Example
A coworker recently came to me with a change request from our client. We have an electronic form system. Form instances are allowed to be edited by the form submitter. Well, this one form type includes a list of people (as a field in the form) that the client wanted to also be allowed to edit the form. I.e. we had to make a change in the security system, but just for part of the application.

To fix this, what we had to do was to override the can_edit? method that is in the base form class, with the logic required in the derived class. Something like:

class SpecificForm < BaseForm
  def can_edit?
    super || form_editors.include?(current_user)
  end
end

The current design was a good one for this change by the above metrics. The new code required was very simple, it was obvious that "can_edit?" was the appropriate place to put the new logic, and since the change exists as a new method only in the one specific class (i.e. the base class's logic didn't change), it is unlikely to affect any other portion of the application.

Refactoring
Code is rarely written in ideal situations. Requirements aren't all known up front. You don't always have an adequate amount of time to do a good job. As a result, existing code bases don't have the best design. When you find that more and more of your changes take more time than expected, or it isn't obvious how they should be done, or each change introduces new bugs, it is time to think about refactoring.

It can be hard to find the time to do this, because by definition refactoring doesn't add any new functionality or fix any existing bugs. Yet it is important to do at times. A month ago, the "can_edit?" change mentioned above wouldn't have been nearly so simple. The permissions functionality had been spread across multiple classes and methods and instance variables in a very inconsistant manner. As such, it was easy to get confused as to the right way to do things, easy to miss a change that needed to be made, and easy to introduce bugs. However, a couple of weeks ago, I finally got fed up with all these (and many other) problems, and refactored the code. While that meant a couple of days where nothing new got fixed or added, that time will be more than made up over the next couple months as new features and bug fixes get made much more quickly.

One last thought. A, often overlooked, key to successful refactoring is pride - don't have too much pride in your code. Many times I've seen developers who are happy to refactor other people's code, but don't want to admit that there could be anything less than perfect about their own design and so resist any changes. Almost everything can be improved. Have enough security in yourself to allow your own designs to be improved and you will become a better developer.

Simple Spaceship in HTML 5

Today's post extends last week's orbit by allowing you to control the spaceship.

Demo

Mult:    Refresh: ms   

JavaScript
This code uses the same 3 classes as the previous demo. To keep the math simpler, I assume that the spaceship starts out 1 unit from the center of the solar system and has a speed of 1. Unlike the previous demo, the acceleration follows the inverse square law to improve realism.

Ship
The ship is still a position, velocity, and acceleration, but now it also has a direction and a concept of a thruster.

  1 var Ship = function() {
  2     this.pos = new Vector();
  3     this.vel = new Vector();
  4     this.acc = new Vector();
  5     this.color = "255,255,255";
  6     this.dir = 0;
  7     this.max_dir = 12;
  8     this.thrusterOn = false;
  9     this.thrusterForce = 0.1;
 10     this.size = 0.01;
 11 }
 12 
 13 Ship.prototype.turnRight = function() {
 14     this.dir = (this.dir + 1) % this.max_dir;
 15 }
 16 
 17 Ship.prototype.turnLeft = function() {
 18     this.dir = (this.dir + this.max_dir - 1) % this.max_dir;
 19 }
 20 
 21 Ship.prototype.angle = function() {
 22     return this.dir * 2 * Math.PI / this.max_dir
 23 }
 24 
 25 Ship.prototype.adjust = function(time) {
 26     if (this.thrusterOn) {
 27         this.acc.x += this.thrusterForce * Math.sin(this.angle());
 28         this.acc.y -= this.thrusterForce * Math.cos(this.angle());
 29     }
 30     this.vel.adjust(this.acc, time);
 31     this.pos.adjust(this.vel, time);
 32 }
 33 
 34 Ship.prototype.draw = function(ctx) {
 35     ctx.save();
 36     ctx.translate(this.pos.x, this.pos.y);
 37     ctx.rotate(this.angle());
 38     ctx.strokeStyle = "rgb(" + this.color + ")";
 39     ctx.scale(this.size, this.size);
 40     ctx.beginPath();
 41     ctx.moveTo(0, -10);
 42     ctx.lineTo(5, 10);
 43     ctx.lineTo(-5, 10);
 44     ctx.lineTo(0, -10);
 45     ctx.stroke();
 46 
 47     if (this.thrusterOn) {
 48         ctx.fillStyle = "rgb(" + this.color + ")";
 49         circle(ctx, 0, 11, 1);
 50         circle(ctx, -2, 13, 1);
 51         circle(ctx, 2, 13, 1);
 52         circle(ctx, -4, 15, 1);
 53         circle(ctx, 4, 15, 1);
 54         circle(ctx, -1, 15, 1);
 55         circle(ctx, 1, 15, 1);
 56     }
 57     
 58     ctx.restore();
 59 }

Simulation
The simulation has the same responsibilities as before, but also scales the display so that a radius of 1 in ship space corresponds to halfway out on the canvas.

  1 var Simulation = function(ctx, size) {
  2     this.ctx = ctx;
  3     this.size = size;
  4     this.reset();
  5 }
  6 
  7 Simulation.prototype.drawStar = function() {
  8     this.ctx.fillStyle= "#FF0";
  9     circle(this.ctx, 0, 0, 20);
 10 }
 11 
 12 Simulation.prototype.draw = function() {
 13     this.ctx.fillStyle = "rgba(0,0,0,1.0)";
 14     this.ctx.fillRect(0, 0, this.size, this.size);
 15     
 16     var dim = this.size/2;
 17     
 18     this.ctx.save();
 19     this.ctx.translate(dim, dim);
 20     this.drawStar();
 21     this.ctx.scale(dim/2.0, dim/2.0);
 22     this.ship.draw(this.ctx);
 23     this.ctx.restore();
 24 }
 25 
 26 Simulation.prototype.step = function() {
 27     var ship = this.ship;
 28     var acc = ship.acc;
 29     var angle = Math.atan2(ship.pos.x, ship.pos.y);
 30     var d2 = ship.pos.x*ship.pos.x + ship.pos.y*ship.pos.y;
 31     acc.x = -Math.sin(angle) / d2;
 32     acc.y = -Math.cos(angle) / d2;
 33     ship.adjust(this.mult);
 34     if (d2 > 16) {
 35         this.stop();
 36         alert("Ship Left Orbit!");
 37     }
 38 }
 39 
 40 
 41 Simulation.prototype.reset = function() {
 42     this.stop();
 43     var ship = new Ship();
 44     ship.pos.x = 1.0;
 45     ship.pos.y = 0.0;
 46     ship.vel.x = 0.0;
 47     ship.vel.y = -1.0;
 48     this.ship = ship;
 49 }
 50 
 51 Simulation.prototype.run = function(mult, refresh) {
 52     this.stop();
 53     this.mult = mult;
 54     var self = this;
 55     this.timer = setInterval(function() {
 56         self.step();
 57         self.draw();
 58     }, refresh);
 59 }
 60 
 61 Simulation.prototype.stop = function() {
 62     clearInterval(this.timer);
 63 }

Helper Function
Both the above classes call the helper function "circle" for drawing filled in circles.

  1 var circle = function(ctx, x, y, rad) {
  2     ctx.beginPath();
  3     ctx.arc(x, y, rad, 0, 2*Math.PI, false);
  4     ctx.fill();
  5 }

Onload
We need a few more controls, so the onload portion has a little extra functionality.

  1 $(function() {
  2     var canvas = document.getElementById("canvas");
  3     var size = 500;
  4     canvas.width = size;
  5     canvas.height = size;
  6     var ctx = canvas.getContext('2d');
  7     var sim = new Simulation(ctx, size);
  8     sim.draw();
  9 
 10     $("#run").click(function() {sim.run($("#mult").val(), $("#refresh").val())});
 11     $("#stop").click(function() {sim.stop()});
 12     $("#reset").click(function() {sim.reset(); sim.draw();});
 13     $("#ccw").click(function() {sim.ship.turnLeft(); sim.draw();});
 14     $("#cw").click(function() {sim.ship.turnRight(); sim.draw();});
 15     $("#thruster").mousedown(function() {sim.ship.thrusterOn = true; sim.draw();});
 16     $("#thruster").mouseup(function() {sim.ship.thrusterOn = false; sim.draw();});
 17 });

HTML
Again, the HTML is just a canvas along with some controls.

  1 <canvas id="canvas"></canvas>
  2 <br />
  3 Mult: <input id="mult" type="text" value="0.01" size="4" />
  4   
  5 Refresh: <input id="refresh" type="text" value="15" size="3" />ms
  6   
  7 <button id="run">Run</button>
  8 <button id="stop">Stop</button>
  9 <button id="reset">Reset</button>
 10 <br />
 11 <button id="ccw">CCW</button>
 12 <button id="cw">CW</button>
 13 <button id="thruster">Thruster</button>

Going in Circles via Html5

Today's post is just a simple little HTML5 animation using canvas that moves a spaceship in orbit around a star (not to scale).

Demo

Mult:    Refresh: ms   

JavaScript
This may be over-engineering for such a simple program, but there are 3 javascript classes used to accomplish this.

Vector
(not a Java Vector but a math vector)

  1 var Vector = function(x, y) {
  2   if (arguments.length == 0) {
  3     x = 0.0;
  4     y = 0.0;
  5   }
  6   this.x = x;
  7   this.y = y;
  8 };
  9 
 10 Vector.prototype.adjust = function(v, mult) {
 11   this.x += v.x * mult;
 12   this.y += v.y * mult;
 13 }

Ship
A ship consists of 3 vectors (position, velocity, and acceleration), and also knows how to draw itself.

  1 var Ship = function() {
  2     this.pos = new Vector();
  3     this.vel = new Vector();
  4     this.acc = new Vector();
  5     this.color = "255,255,255";
  6 }
  7 
  8 Ship.prototype.adjust = function(time) {
  9     this.vel.adjust(this.acc, time);
 10     this.pos.adjust(this.vel, time);
 11 }
 12 
 13 Ship.prototype.draw = function(ctx) {
 14     ctx.save();
 15     ctx.translate(this.pos.x, this.pos.y);
 16     ctx.strokeStyle = "rgb(" + this.color + ")";
 17     ctx.beginPath();
 18     ctx.moveTo(0, -10);
 19     ctx.lineTo(5, 10);
 20     ctx.lineTo(-5, 10);
 21     ctx.lineTo(0, -10);
 22     ctx.stroke();
 23 
 24     ctx.restore();
 25 }

Simulation
A simulation consists of the ship, knows how to accelerate the ship around in an orbit, and has the commands for actually starting, stopping, and running the simulation.

  1 var Simulation = function(size) {
  2     this.size = size;
  3     this.reset();
  4 }
  5 
  6 Simulation.prototype.drawStar = function(ctx) {
  7     ctx.fillStyle= "#FF0";
  8     ctx.beginPath();
  9     ctx.arc(0, 0, 20, 0, 2*Math.PI, false);
 10     ctx.fill();
 11 }
 12 
 13 Simulation.prototype.draw = function(ctx) {
 14     ctx.save();
 15     ctx.fillStyle = "rgba(0,0,0,1.0)";
 16     ctx.fillRect(0, 0, this.size, this.size);
 17     ctx.translate(this.size/2, this.size/2);
 18     this.drawStar(ctx);
 19     this.ship.draw(ctx);
 20     ctx.restore();
 21 }
 22 
 23 Simulation.prototype.step = function() {
 24     var ship = this.ship;
 25     var acc = ship.acc;
 26     acc.x = -ship.pos.x;
 27     acc.y = -ship.pos.y;
 28     ship.adjust(this.mult);
 29 }
 30 
 31 
 32 Simulation.prototype.reset = function() {
 33     this.stop();
 34     var ship = new Ship();
 35     ship.pos.x = this.size / 4;
 36     ship.pos.y = 0;
 37     ship.vel.x = 0;
 38     ship.vel.y = -this.size / 4;
 39     this.ship = ship;
 40 }
 41 
 42 Simulation.prototype.run = function(ctx, mult, refresh) {
 43     this.stop();
 44     this.mult = mult;
 45     var self = this;
 46     this.timer = setInterval(function() {
 47         self.step();
 48         self.draw(ctx);
 49     }, refresh);
 50 }
 51 
 52 Simulation.prototype.stop = function() {
 53     clearInterval(this.timer);
 54 }

On Load
Then there is the little bit of code to kick everything off, which leverages jQuery. This just sets the size of the canvas, and links the input controls from the HTML to the Simulation object.

  1 $(function() {
  2     var canvas = document.getElementById("canvas");
  3     var size = 500;
  4     canvas.width = size;
  5     canvas.height = size;
  6     var ctx = canvas.getContext('2d');
  7     var sim = new Simulation(size);
  8     sim.draw(ctx);
  9 
 10     $("#run").click(function() {sim.run(ctx, $("#mult").val(), $("#refresh").val())});
 11     $("#stop").click(function() {sim.stop()});
 12     $("#reset").click(function() {sim.reset(); sim.draw(ctx);});
 13 });

HTML
The HTML is just a canvas along with some inputs to control the simulation.

  1 <canvas id="canvas"></canvas>
  2 <br />
  3 Mult: <input id="mult" type="text" value="0.01" size="4" />
  4   
  5 Refresh: <input id="refresh" type="text" value="15" size="3" />ms
  6   
  7 <button id="run">Run</button>
  8 <button id="stop">Stop</button>
  9 <button id="reset">Reset</button>

Thoughts on Interviewing

I've been on both sides of the interviewing table a number of times. Currently I'm on the less stressful side, that of trying to find a good candidate. So how do you find a good candidate?

Joel on Software has two criteria: Smart, and Gets Things Done. His Guerrilla Guide to Interviewing does a good job of explaining what he means, so I won't repeat it. The question is, how do you determine if someone is smart and/or gets things done?

The Google/Facebook style of interviewing where you have to solve problems on a whiteboard answers the "smart", at least for a certain type of smart, and assuming that you don't freeze up under the pressure of interviews. But what about "gets things done"? The ability to solve problems doesn't mean that you will work diligently. So the best way I know to determine that is to investigate your track record. An internship or trial hire period would be better, but since that isn't typically an option, it instead means trying to figure out from someone's resume and their answers to questions about it. This is very much, an inexact science.

What about smart? Different positions require different degrees of intelligence. As my boss reminded my, the work that we currently do doesn't require super genius intelligence. However, I maintain that no matter how mundane the job is, it is better to have higher caliber employees doing them. Do you really want to be maintaining the software of someone who is just barely qualified to do the job? Are you really going to go to that person for help when an issue arises? How are you going to learn and improve while working with someone who just wants to get by?

Therefore, while I am not looking for super-geniuses, I do want someone who is more than just barely competent. So how do I personally test for that?

Based on Joel's article we now make candidates write some programs on the whiteboard. We are hiring a programming position. If you want the job you should be capable of turning thoughts and ideas into code. I realize that between modern IDEs and the internet, modern day programmers have access to unprecedented levels of help. However, you shouldn't be dependent on that help. If you don't know the difference between AND and OR without looking it up, or you don't know any collection types other than an ArrayList, how can I be confident that you will write quality maintainable code?

How about theoretical knowledge from a CS program? How important is that? I realize how much importance you put on that will depend on your own skills and experiences. While it may not be important to actually know that the time complexity of a bubble sort in O(n²), at some point you will run into code that is running unacceptably slow. If you don't show any knowledge of time complexity, what evidence do I have that you will actually pick out the bottlenecks of a slow program and know how to fix them? (or even better, how to avoid them in the first place?)

Unfortunately for me, every example above came from recent interviews I've done. Am I wrong to think that a Java developer should be familiar with more than just ArrayList from the java.util Collection classes? Is it random trivia to remember the time complexity of bubble sort? I think these are examples of things people who care about the craft of software development should know, but maybe I am biased by my own experiences?

What do you think?

Saved By The VM

One of the most exhilarating, but also frustrating, aspects of working in computers is how fast things move. Things that would've been considered disgracefully wasteful years ago, now provide us with solutions that I never would've imagined then. For example the editor that I am using to write this is running inside of my browser. In effect, the browser is acting as a virtual machine, and the editor is just one application running on it. How much CPU "wastage" is there by running in a VM rather than a native application? Who knows, but it doesn't matter, as my laptop isn't even breaking a sweat. And this is just one of dozens of tabs that I have open in the browser, and the browser is just one of over a dozen applications running currently (plus who knows what the OS is doing).

Not only is CPU time cheap, but so is storage. At work, our network storage takes automatic snapshots, which makes it trivial to retrieve from backups. This would've seemed incredibly wasteful to me not that long ago. This, combined with VMs, saved me a ton of work this past week.

Torquebox 2.0.0 was released recently, and I wanted to get it installed on our development server. While I was doing this, I decided to upgrade the version of Java running on the machine. I downloaded the latest version and installed it. Unfortunately, I wasn't paying attention and I let it install in the default location, rather than our group's standard location. In a moment when I was apparently flashing back to the days of yore, I thought I could just move the Java directory structure to the location where I wanted it. Since the registry settings did not get updated in the process, bad things happened. My first attempt to fix things was to just delete the Java directories, and reinstall it. Unfortunately, Java thought it was still installed, and refused to reinstall. So then I asked Windows to uninstall Java, but a DLL was missing (probably because I had already deleted it), so it couldn't uninstall either.

What was I to do? I spent some time fiddling with the Windows Registry, but that was getting me nowhere. Finally, I came to my senses and remembered what year it was. Our servers are running on VMs which means it's just an image on a network disk somewhere, which has regular snapshots taken. So I just wandered down the hall and asked one of our system administrators to restore the VM to an image from the day before. A few minutes later, my mess was cleaned up.

So yes, VMs are wasteful and inefficient, but they sure save time sometimes. And what's all this extra CPU power for, if not to make our lives easier?

Pair Programming - My Experiences

I've mentioned pair programming before in the context of the psychology of extreme programming, but I haven't said too much about it. This is because I've had very little first hand experience with it. Well, in the last couple of months I've spent a lot of my time pair programming, and here are my preliminary thoughts.

Distractions
I have argued that pair programming can cut down on distractions. I find this to be true. When working on my own if I execute a command that I know will take 5 or more seconds, it is very tempting to take this break to check my email or read a slashdot article. Unfortunately, while I justify this as being efficient, in reality it turns a 5-10 second pause into a couple minute pause. The change of context also flushes some registers, err.. short term memory, making it harder to be as focused when I first switch back to my development tasks.

When someone is looking over my shoulder, I am not going to make that context switch.

Motivation
I find that my enthusiasm for projects ebbs and flows. I'll be gung-ho for a while, and then my motivation will slip away. Its easier to keep working in a pair, both from the peer-pressure that you don't want to be slacking, and just the fact that its more fun.

Efficiency
I've always read that pair programming is more productive than one person working alone, but not twice as productive. (measured in terms of how much code gets written per unit time). My experience of the past couple months agrees with this. Of course this ignores other benefits from pair programming, like potentially higher quality code, and the implicit cross-training that happens.

Correctness
Pair programming definitely cuts down on mistakes. While the person on the keyboard is directly thinking about what they are doing, the backseat driver can focus on this big picture. These two different views helped us realize shortcomings that would not have been found until production had either of us been working alone. Also, when bugs were encountered, having two people (who think differently) looking at them, meant we were able to resolve them quicker.

Drawbacks
There are a few problems that I encountered with pair programming. The most obvious one is scheduling. The two of us paired up have different schedules in our personal lives, so we don't show up at the same time, or leave at the same time, or even take our lunch breaks at the same time. Plus we both have other professional responsibilities that sometimes require one or the other's attention at various times at the day. Thus an eight hour work day results in only four or five hours of useful pairing time.

The problem that I didn't expect was energy drain. After a couple of weeks of doing primarily pair programming I felt mentally exhausted. While we got a lot done, it wasn't a pace that I can maintain indefinitely. I think the little breaks that I normally build into my day, which were missing when pair programming, helped keep me fresh. I suspect that I can address this problem by intentionally scheduling breaks when pairing up.

it is most effective to work alone for a whileThe third, and most important, problem is problem solving. There are obviously times when two heads are better than one. But there are also times where the two heads need to work on their own. To solve difficult design problems, or to come up with innovative solutions to a problem, I truly believe it is most effective to work alone for a while. Then, after I've had a chance to think about it in my own way and come to my own conclusions, is the time to meet and discuss the ideas with others. i.e. if we work together on a solution, we will come up with one solution. If we work independently, we will come up with two solutions, from which we can then draw a third solution that is better than what either of us came up with independently, or if we had worked together all along.

Conclusion
Pair Programming has a place in your toolkit of software development skills. I think it can be really helpful when working on software that you want to be reliable, you want more than one person intimately familiar with the code base,  and which doesn't require too much innovative problem solving. However, like most things, moderation is the key. There are definitely times when pair programming will hinder you.

If you have the opportunity, give it a try. However, be open to the fact that it isn't best for all situations, so don't force it if it doesn't fit.

Chasing Technology

Maybe I'm just getting old, but keeping up with technology seems to be getting harder and harder. As I've mentioned before, I am a big fan of learning your libraries, languages, and tools. But how are you supposed to do that, when the libraries keep upgrading and there are constantly new tools to use? Just this past week there was a post on slashdot suggesting that tech skills have a 2 year half life. The comments on that thread seem to mostly disagree with the hypothesis, but there is definitely some truth to it.

The two languages that I use the most at work did not exist when I started college. However, languages are easy to keep up with. They tend to move slowly and are very well documented. Tools, libraries, and frameworks, on the other hand, are a nightmare to chase. They are constantly being upgraded, and often times the best documentation are the question/answers out there on the web at places like stackoverflow. The problem with these sites is that they are only useful after a tool has been out long enough for a critical mass of people to use.

So what's a person supposed to do?

Well, first, as many commenters from the slashdot article stated, the most valuable skills a software engineer has is knowing how to design, develop, and debug software. These skills don't go away just because you are using a new language or library. So play to your strengths. If you design and write clean code, it'll be easier to fix/update as you learn the language/library/tool better.

Second, choose your upgrades carefully and intentionally. Just because there is a newer version of something out there, doesn't mean you need to use it. Are the benefits worth it to upgrade now? Does it make more sense to wait until it has been popularly adopted? Or maybe it makes sense to skip a version or two.

Third, don't become an expert in everything. Most of us have finite amounts of energy and time. So, while I'm a big believer in being an expert in the tools you are using, that isn't always practical. If a particular tool/library/framework is bound to be upgraded/replaced in the near future, it's probably ok if you use the existing library in a less than optimal way. Focus your energies on the timeless (design and coding skills) or the long lasting (languages).

constantly learnFourth, and most importantly, constantly learn. If your employer doesn't provide opportunities for professional growth, take it anyway. Just as you don't ask for permission for every single test that you write, or every whiteboard design that you draw, you don't need permission to explore new technologies and tools. Whether they know it or not, your employer pays you to be a good and competent software developer. This requires you to be constantly learning new and better ways of doing things. If you are not taking ten to twenty percent of your time exploring and learning, you are shortchanging yourself and making yourself obsolete. I realize that it seems there is never time to do this, but like other aspects of good software engineering (appropriate design, testing, documentation), in the long run you will be worse off if you don't make the investment now.

Vexing Bugs

While bugs are a part of development, there are a few types of bugs that I find particularly vexing: intermittent bugs, library bugs, and bugs that only happen in production.

Intermittent Bugs

It's annoying to perform the exact same set of steps 4 times, have it work 3 of the times, and fail once. When Murphy has his way, it works when you, perform the steps or are watching, but fails when the user does it on their own. How do you diagnose a bug that you can not reliably reproduce?

Library Bugs

The majority of library bugs aren't actually bugs in the library, but rather a bug in how you are using the library. Either way, having cryptic error messages coming out of the bowels of a library, when there is no apparent connection between the error and your code, is not fun. I find the process of googling error messages or library usages to be much more frustrating than tracking down errors that are entirely in my own source code.

Production Bugs

While we strive to have our development environment similar to the production environment, there are certain discrepancies that always creep in. Things like debug information, optimization level, database source, etc. When in development, you don't want to be messing with production data, and your willing to give up some performance to better track the code. But since it isn't the same, code that works fine in development doesn't always work in production. Ugh.

This Week

So what prompted this post? Well, this week I had a bug that was at the intersection of these bugs. We had deployed a new version of our application, and suddenly I started getting server emails about errors. It was some cryptic error happening within the library we use for making Web Service calls. And, of course, the errors only happened sometimes. Even more frustrating, when trying to reproduce the error on my local box using WEBrick in development mode, the error never happened.

After setting up a development server on my local box that was configured just like the production box, I was able to intermittently reproduce the error. Using my standard debugging technique of adding printfs, I eventually noticed output that looked like:

Starting Web Service Call 1
Starting Web Service Call 2
Exception Caught from Web Service Call 2
Ending Web Service Call 1

Unfortunately, it didn't catch my eye right away, but I eventually noticed that every time there was an error, the web service calls were getting interleaved. i.e. Web Service Call 2 was starting before Web Service Call 1 started. And now it suddenly made sense. 

The two web service calls were being made as the result of two separate AJAX callbacks from a web page. Since the the two web service calls were being done in two different web requests, they were being handled in parallel resulting in a race condition. As it turns out, the web service library we were using is not thread safe. However, apparently WEBrick in development mode was serializing these requests (i.e. not processing 2 until 1 was completely handled), and so in development there were no threading issues. 

As with most bugs, once the problem had been diagnosed, it wasn't that hard to fix. In this case, we put the web service calls in a critical section, forcing them to be serialized. For our particular use case, the higher latency of serializing the web service calls was acceptable, allowing for a fairly simple solution.

Moral

This post wasn't really intended to have a moral. It was mostly just me talking about my week, but I suppose there are a couple good ideas.

• If you think its a library bug, you are probably misusing the library. (i.e. Learn your libraries)
• Intermittent errors are indicative of threading issues.
• Know the differences between your development and production environments.
  oh, and maybe most importantly of all
• When frustrated, take a break. I didn't actually track down the bug until I walked away from it for a while and then came back to it with fresh eyes.

Learn Your Libraries

Story 1
I still remember the month long final project from my first ever CS class. Actually, I remember almost nothing from the project except for a bug that caused me to have to throw away my first three weeks of work. I had just learned about enumerations in Pascal and thought they were the neatest thing since sliced bread. I designed my entire project around enumerations. Users would enter their menu choices via enumerations. I would display the enumerations as options. All of this depended very heavily on my understanding of enumerations which, as it turned out, was in no way related to what enumerations actually are.

Unfortunately, I didn't find this out until I had spent two weeks writing the entire program and tried running it. It didn't work at all. Like any good developer I randomly changed various lines of code until it worked. A week later, with the project deadline looming, my project was not any closer to working. It was time for drastic action. Since I couldn't seem to get the enumerations to do what they were "supposed" to do, I actually read up on them. Imagine my dismay when I realized that all the assumptions that I had made were wrong. And not just a little bit wrong. The reality (enums are a typesafe way to have named constants) and my vision (they'll do everything I want, including cooking dinner and solving the halting problem) were completely unrelated. I spent the rest of the day griping and complaining about why would anybody create such a useless design feature. When I realized that that wasn't getting me any closer to a program that I could actually turn in, I finally sat down and rewrote the entire program using only Pascal features that I actually understood, and eschewed enumerations entirely.

Story 2
The other day a friend asked me what I knew about Java Serialization. After asking for details, it turns out that he was working on a project where they were doing bit-diddling on the Serialized output of an object. One of the assumptions that they had was that if you had an objects instance1 and instance2, and you serialize both of them, that the serialized byte data would be identical if instance1 and instance2's fields are all identical. They had run a bunch of experiments and it seemed like their assumption wasn't always holding. I think the question to me was hoping that I would point out something they had missed that would give them an easy fix.

Upon check the Serialization Spec, it turns out their assumption was not valid. My suggested solution was to not use Java's Serialization, but rather write their own conversion to bytes that would fit their specific needs. This may seem like reinventing the wheel, but as I've mentioned before, I believe that rolling your own is often the way to go.

Story 3
On a project I once worked on, the decision was made that all data was going to be handled as XML. All of the data was stored as an XML document in a single column in a database. To process individual data items, XPath queries would be made. HTML reports would be created by running giant XSLT transforms on the data. The assumption was that because XML has such great support, it would be easy to add new data (just turn it into XML and merge it with the other data), and create new reports (just write a new XSLT).

The reality is that just because you can create a general tool to handle syntactically correct XML, you still need to customize the logic for the semantics. Which meant every new data source added still required code and logic changes. As for generating the HTML via XSLT transformations, its very difficult to use modern HTML features, like Ajax, this way. And by making such a large portion of the code base XSLT and XPath, we couldn't take advantage of the IDE's ability to do easy code completion, navigation, and refactoring, or leverage the development team's OO skill to create small, easily testable, and reusable components. And then there's the performance issues, both in time and space, that were encountered with trying to manipulate and store large quantities of large XML documents.

Moral
The moral of these stories is that you need to know your tools. Enumerations, Serialization, and XML are all great, when used appropriately. When not... well, invalid assumptions lead to bugs. Invalid assumptions about crucial components of your software lead to large scale rewrites. To avoid the large scale rewrite, we often have a period of denial where we try to make the unworkable work. The worst case is that with enough bubble gum, chicken wire, and ingenuity we do make it work, at least some of the time. This results in a maintenance nightmare keeping it working and large scale rewrites are even less likely to happen once you've got "working" code.

While we all know that assumptions can be bad, the danger is when we don't realize we are making assumptions. In the first story I thought I know what enumerations did. The developers in the second story had used Java Serialization on many projects and thought they fully understood it. In the third story the benefits of XML were considered without awareness of all of the limitations and restrictions that came along. i.e. We didn't know what we didn't know.

The conservative solution to this problem is to only use technologies you have already used successfully on all your projects. While this seems to be favored by many people, I am too much a fan of shiny objects and so like to try new things. The key to using new technologies (or existing ones in new ways) is to limit the scope until you have used it successfully. If it is going to be integral to your design, create a small throw-away prototype first to make sure you find the dark corners and sharp edges first.

The most important solution, though, is to be willing to admit you were wrong. Its impossible to get very far in life without making assumptions, and despite your best precautions, sooner or later, you will be wrong about an assumption. Rather than try to patch on hack after hack to try to make a bad assumption viable, be willing to break from the past and rewrite/redesign code given your new knowledge. Even if it means throwing away a lot of work.

RSpec Lessons Learned

Previously I showed some unit tests written in RSpec. Those test were fairly ugly, partially because I didn't really understand RSpec. In an effort to have better unit tests, I have learned a little more about RSpec.

Read the Latest Documentation
You'd think this would be obvious, but given the lack of links to the documentation on the various "here's advice on using RSpec" blogs, it must not be done a lot. Worse, somehow I had been only looking at the old documentation for version 1 of RSpec. So, if you are going to use RSpec, I suggest you read latest documentation, which as of when I am writing this is RSpec 2.6. This will definitely be helpful.

Make Tests Self Documenting
Each test (it or specify block) can take a description. Sometimes this is necessary, but any time you have documentation (and that's what this description is), you risk having the documentation be out of sync with the code. While not everyone agrees, I am a fan of letting the tests document themselves, whenever possible.

Make Use of Context
Use the "context" keyword to describe what you are doing in a "before" block to set up the state necessary for testing.

describe Thing do
  let(:thing) {Thing.new}
  subject {thing}
  context "empty object" do
    #insert tests
  end
  context "with inherited object" do
    let(:base) {Thing.new}
    before(:each) do
      base["foo"] = 5
      thing.prototype = base
      thing["baz"] = 15
    end
    #insert tests
  end
end

Make Use of Describe
Use the "describe" keyword to describe either the noun that is being tested, or the actions that are under test. i.e. if you have actions in a "before" block that are the actions being tested, you should you use "describe" rather than "context"

describe Thing do
  let(:thing) {Thing.new}
  subject {thing}
  context "empty object" do
    describe "when assigning via properties" do
      before(:each) do
        thing["foo"] = 5
        thing["bar"] = "hello"
      end
      # insert checks
    end
  end
end

Make Use of Subject
If you have multiple tests on a single object, make it the RSpec Subject and put it in a describes block. This way, all of your "should" comparisons will be implicitly on this object.

describe Thing do
  let(:thing) {Thing.new}
  subject {thing}
  context "empty object" do
    describe "keys" do
      subject {thing.keys}
      it {should_not be_nil}
      it {should be_empty}
    end
  end
end

Use Its
Often times you want to test the properties of an object. You can use the "its" method to have the implicit subject of "should" comparisons be the result of the method or array dereference specified.

describe Thing do
  let(:thing) {Thing.new}
  subject {thing}
  context "empty object" do
    its(['newProperty']) {should be_nil}
    its('newMethod') {should be_nil}
  end
end

Let and Subject are Lazy Loaded
The "variables" defined in "let" calls and the "subject" aren't actually evaluated until they are used. So if you never reference a variable specified in a "let", then that code is never executed. This also means that order isn't important.  i.e. the following will work:

let (:a) {b + 1}
let (:b) {5}
specify("show using let") {a.should == (b+1)}

Shared_examples and shared_contexts are Global
I actually haven't found this documented, so I may be doing something wrong. But I have found that if I have two different rspec files that each have a "shared_examples_for 'test this object'", this causes problems. I can test each rspec file in isolation and it is fine. But if I try to test both at the same time, I get an error saying that a shared example already exists with the name "test this object".

There are two different solutions to this problem that I have found. If the shared examples are the same, pull them into a common Module that is included. Note, this is better than having repeated code anyway. If the examples are different, then you have to be more unique with the names of the shared examples.

Final Thoughts
It seems that the approach of RSpec is to write tests such that they are self descriptive and so that each test tests exactly one thing. While in theory, this sounds good, I am finding that this results in very verbose test files. That's even with the cleaning up that I have done after learning RSpec better.  i.e. I like a lot of what RSpec does, but I am not convinced that it is the best way to write unit tests.

Rewritten Tests
Using what I have learned, I have rewritten the unit tests from before. Here is what they look like now:

require 'spec_helper'

describe Thing do
  let(:thing) {Thing.new}
  subject {thing}

  shared_examples_for "simple object" do |map, self_keys|
    describe "keys" do
      subject {thing.keys}
      it {should have(map.size).items}
      it {should include(*(map.keys))}
      it {should_not include("noSuchProperty")}
    end
    describe "self_keys" do
      subject {thing.self_keys}
      before(:each) { self_keys ||= map.keys}
      it {should have(self_keys.size).items}
      it {should include(*self_keys)}
      it {should_not include("noSuchProperty")}
    end
    describe "fields" do
      map.each do |k, v|
        its([k]) {should == v}
      end
      its(['noSuchProperty']) {should be_nil}
    end
    describe "methods" do
      map.each do |k, v|
        its(k) {should == v}
      end
      its('noSuchMethod') {should be_nil}
    end
    describe "to_hash" do
      subject {thing.to_hash}
      it {should_not be_nil}
      it {should have(map.size).items}
      it {should include(*map.keys)}
      it {should include(map)}
    end
  end
  
  context "empty object" do
    describe "keys" do
      subject {thing.keys}
      it {should_not be_nil}
      it {should be_empty}
    end
    its(['newProperty']) {should be_nil}
    its('newMethod') {should be_nil}
    describe "to_hash" do
      subject {thing.to_hash}
      it {should_not be_nil}
      it {should have(0).items}
    end
    
    describe "when assigning via properties" do
      before(:each) do
        thing["foo"] = 5
        thing["bar"] = "hello"
      end
      it_should_behave_like "simple object", 'foo'=>5, 'bar'=>"hello"
    end
    
    describe "when assigning via methods" do
      before(:each) do
        thing.foo = 5
        thing.bar = "hello"
      end
      it_should_behave_like "simple object", 'foo'=>5, 'bar'=>"hello"
    end
  end
  
  context "with inherited object" do
    let(:base) {Thing.new}
    before(:each) do
      base["foo"] = 5
      base["bar"] = "hello"
      thing.prototype = base
      thing["baz"] = 15
      thing["bye"] = "bye"
    end
    it_should_behave_like "simple object", 
      {'foo'=>5, 'bar'=>"hello", 'baz'=>15, 'bye'=>"bye"}, ['baz', 'bye']
    describe "when overriding values" do
      before(:each) do
        thing["foo"] = 25
        thing["bar"] = "hola"
      end
      it_should_behave_like "simple object", 'foo'=>25, 'bar'=>"hola", 'baz'=>15, 'bye'=>"bye"
    end
  end
end

Below is what the output looks like. As you can see, if you read it, it describes the tests that are being run more clearly than the old version of the tests.

$ rspec spec/models/thing_spec.rb 

Thing
  empty object
    keys
      should not be nil
      should be empty
    ["newProperty"]
      should be nil
    newMethod
      should be nil
    to_hash
      should not be nil
      should have 0 items
    when assigning via properties
      it should behave like simple object
        keys
          should have 2 items
          should include "foo" and "bar"
          should not include "noSuchProperty"
        self_keys
          should have 2 items
          should include "foo" and "bar"
          should not include "noSuchProperty"
        fields
          ["foo"]
            should == 5
          ["bar"]
            should == "hello"
          ["noSuchProperty"]
            should be nil
        methods
          foo
            should == 5
          bar
            should == "hello"
          noSuchMethod
            should be nil
        to_hash
          should not be nil
          should have 2 items
          should include "foo" and "bar"
          should include {"foo"=>5, "bar"=>"hello"}
    when assigning via methods
      it should behave like simple object
        keys
          should have 2 items
          should include "foo" and "bar"
          should not include "noSuchProperty"
        self_keys
          should have 2 items
          should include "foo" and "bar"
          should not include "noSuchProperty"
        fields
          ["foo"]
            should == 5
          ["bar"]
            should == "hello"
          ["noSuchProperty"]
            should be nil
        methods
          foo
            should == 5
          bar
            should == "hello"
          noSuchMethod
            should be nil
        to_hash
          should not be nil
          should have 2 items
          should include "foo" and "bar"
          should include {"foo"=>5, "bar"=>"hello"}
  with inherited object
    it should behave like simple object
      keys
        should have 4 items
        should include "foo", "bar", "baz", and "bye"
        should not include "noSuchProperty"
      self_keys
        should have 2 items
        should include "baz" and "bye"
        should not include "noSuchProperty"
      fields
        ["foo"]
          should == 5
        ["bar"]
          should == "hello"
        ["baz"]
          should == 15
        ["bye"]
          should == "bye"
        ["noSuchProperty"]
          should be nil
      methods
        foo
          should == 5
        bar
          should == "hello"
        baz
          should == 15
        bye
          should == "bye"
        noSuchMethod
          should be nil
      to_hash
        should not be nil
        should have 4 items
        should include "foo", "bar", "baz", and "bye"
        should include {"foo"=>5, "bar"=>"hello", "baz"=>15, "bye"=>"bye"}
    when overriding values
      it should behave like simple object
        keys
          should have 4 items
          should include "foo", "bar", "baz", and "bye"
          should not include "noSuchProperty"
        self_keys
          should have 4 items
          should include "foo", "bar", "baz", and "bye"
          should not include "noSuchProperty"
        fields
          ["foo"]
            should == 25
          ["bar"]
            should == "hola"
          ["baz"]
            should == 15
          ["bye"]
            should == "bye"
          ["noSuchProperty"]
            should be nil
        methods
          foo
            should == 25
          bar
            should == "hola"
          baz
            should == 15
          bye
            should == "bye"
          noSuchMethod
            should be nil
        to_hash
          should not be nil
          should have 4 items
          should include "foo", "bar", "baz", and "bye"
          should include {"foo"=>25, "bar"=>"hola", "baz"=>15, "bye"=>"bye"}

Finished in 0.0831 seconds
78 examples, 0 failures