Comparing Two Decoupling Approaches

At work this week I suggested an alternative to how we currently decouple code.  I got asked the eminently reasonable question, "How is that way any better?"  This post discusses the two approaches.  Note that I think that the approach I favor can be improved upon, but this is a starting point.

The Problem

Imagine you have a Frobulator class.  You use this object in many places in your code.  However, you can foresee a situation where you might replace the Frobulator class because of a change in the underlying data source.  If and when this occurs you don't want to have to go through your entire code base and change every occurrence of Frobulator to FrobulatorPrime.  In short, you want to decouple your code base from the Frobulator object.

Solution 1

Create a GenericFrobulator class.  The GenericFrobulator class has the exact same methods as the Frobulator class.  Its implementation is to have an instance of a Frobulator and for every method to just redirect the call to the Frobulator class.  The rest of your code base uses the GenericFrobulator object.  If and when you replace the Frobulator class, all you have to do is change the one class.

Cons

This requires writing a bunch of "dummy" code.  I.e. one method for every method in Frobulator each of which is a single line of pass through code.

There is also a performance hit of the extra call - though realistically this is almost definitely negligible.  Also, on errors or debugging stack traces will have an additional level in the stack, but this is only a minor level annoyance.

Pro

Successfully decouples the code base from the Frobulator object.  Only one class has to change (in addition to the creation of the new FrobulatorPrime class) when a new implementation is used.

Solution 2

Create an IFrobulator interface (we're working in C#, hence the 'I' prefix on interfaces) that has the same interfaces as the Frobulator object, and make the Frobulator class implement this interface.  Create a FrobulatorFactory class which has a single method which returns an IFrobulator object.  The implementation creates a Frobulator object.  The rest of the code base uses this FrobulatorFactory class to create the IFrobulator that it uses.

Cons

You have to create two additional classes (IFrobulator and FrobulatorFactory), one of which doesn't actually do anything.

Pros

It successfully decouples the code base from the Frobulator implementation.

Comparison

On the surface, these two approaches are pretty similar.  They both require similar amounts of code.  With the Interface approach you don't have to create "pass through" methods for every single method, but on the other hand you do have to create an extra class - the Factory class.  Both approaches involve a similar amount of work to change out the Frobulator class -- create the FrobulatorPrime class, and either modify the FrobulatorFactory or the GenericFrobulator class.

Real Difference

The real benefit to the interface approach is the primary benefit to OO in general.  Its not that it saves code, but rather it models your intent.  By having an Interface and a Factory, it is explicit what the purpose of each class is.  It is obvious that the Factory is for creating objects, and the Interface is for decoupling.

It also guards against new developers breaking the design.  With the GenericFrobulator approach, it is always possible that down the road some developer will decide to insert logic (besides a simple call-through) into this class.  Now all the decoupling power is lost.  Interfaces (at least in languages like C# and Java) can't contain any code, so this can't happen.

The Interface method also supports a hybrid model.  Imagine that you have a scenario where part of your code wants to use the Frobulator class, and part of it wants to use the FrobulatorPrime class.  How would you do this with the GenericFrobulator approach?  With the Interface and Factory approach, you can either create a new Factory, which some classes use, or you can have an additional Factory method/parameter to determine which object gets created.  (or you could pass in the IFrobulator object and make it the calling code's responsibility).

I also feel that the Interface and Factory approach is closer to the real panacea of decoupling, though this is a fuzzier argument.

Failed Decoupling Goals

There are other goals of decoupling, that both of these approaches fail at.  Both can be modified to potentially achieve them, though I feel that the Interface/Factory approach more naturally fits these modifications.

The first goal is to make it possible to change the Frobulator class out without having to make any changes to the code base.  I.e. without having to change the Factory class or the GenericFrobulator class.  An example of this is the java.db packages which you get via putting the right jar file in your classpath and using the right configuration string to get your db classes.  (you are using a config file for this, right?)

The second goal is to be able to test the rest of your code base without having a Frobulator instance at all.  Sometimes it is nice to be able to have a StubFrobulator class which creates test data, and use that to test the rest of the object.  If your code is properly decoupled, this should not be difficult.

Summary

In short, I think the Interface and Factory approach is the better approach because it more directly describes the intent, more succinctly solves the problem, and is easier to modify to handle more advanced decoupling issues.

Is there yet a different way to decouple software?  How do you do it?

Don't Try To Write Reusable Software

First a story

Early in my first job, another fresh out of school co-worker and I decided that we were going to rewrite this one library we used to make it more reusable for all of the upcoming projects.  When we asked for advice from a more experienced developer, he basically told us not to bother, because we were going to fail.  We scoffed at his cynicism, knowing that our design skills and knowledge of the requirements were going to result in this awesome library.

Fast forward a year and the "reusable" library that we spent so much time and effort on still had to go through considerable customizations to handle cases that we hadn't even envisioned when we started.

Did we fail because of incompetence?  While our skills probably weren't quite as good as we thought at the time, no.  We failed because it required the unknowable - knowing what the future would bring.  The Agile premise of YAGNI summarizes this.

The Moral

I summarize this experience with the title to this post, don't try to write reusable software.

I can hear your CS hearts shudder at this. 

How can this be?  Isn't writing reusable software what is taught in our programming classes?  You can't really be saying that we should be cutting and pasting code all over the place.  Besides there are plenty of successful reusable libraries out there, like printf or java.util.*.  This must be a false premise!

Again, I say, "Don't try to write reusable software!"

But...but...that goes against everything I learned.

The Real Moral - Discover Reusable Software

Don't try to write reusable software, discover it instead.

Huh?

Obviously you shouldn't be cutting and pasting code.  Obviously you should still be using methods and classes.  The DRY principle is a good one and it implies reusable code.  However, reusable code is the means, not the goal.  

Let me repeat that, because it is important.  Writing code that is reusable (methods, classes, etc.) is the means to achieving the goal of a clean design.  But your goal isn't the reusability, your goal is to only specify everything once.  Only write your algorithm once.  Only specify a value once.  Only put your business logic on one place.  If you strive to achieve those goals, you will end up with reusable components.  

Even better, you will end up with code that is maintainable and written in a timely manner.

Madking is back

It has been almost 2 years since I last posted here.  During that time I changed jobs twice.  Once to take a job that was too good to be true.  Then again when it turned out that it was too good to be true and they cancelled the project, leaving me looking for a job.

Not only have I not blogged in 2 years, but I also haven't competed in programming contests in that time frame.  I am finally getting back on that hobby horse.  I am participating in the TopCoder Open 2009, and as of right now I am one of 120 still standing (one of 9 Americans, if I counted right) in the Algorithm contest.  Tomorrow they narrow down to 45, so we'll see how that goes.

I figured if I could get back to the programming contests, I could also get back to the blog.  I am not going to promise any particular schedule, so I'd suggest subscribing to the RSS feed if you want to actually read what I write.  I will try to write more often than every 2 years.  :)

One other development since I last posted, is that we have gotten our own domain, and I picked a hosting site that specializes in hosting Ruby on Rails applications.  I have a couple applications that I am playing with on the site.  I am envisioning that as I work with them, they will inspire me for things to blog about.

I never know how to end posts, so I  am just going to end this one with a statement about ending posts.

Don't Soft Code

You have probably been taught to not hard code parameters in your code. For example rather than saying:

if (connections > 50) { // do something

You should probably write:

if (connections > MAX_CONNECTIONS) { // do something

and if you are feeling particularly ambitious, MAX_CONNECTIONS should be set by a config file, so that a sys admin or operator can change this value at deployment time.

This has been a driving force behind my designs for some time now. I try to not hard code any values in my programs. Sometimes expediency leads me to do it anyway, but at least I feel guilty for doing it. Unfortunately, I have found that avoiding hard coding values is not the panacea that is promised. You now have code that is effectively split up (part in the code, part in the config file) which can make it harder to read/understand the code. I have also seen that if the config files get large/complicated enough, no one but the programmer is going to be willing to touch this file, and it really becomes just part of the program.

I am writing this now, because I recently read an article on worsethanfailure.com about Soft Coding. The article makes a very good point - sometimes "Hard Coding" values is better than the alternative. Rather than reiterate why, I'll just tell you to go read the article. The main moral of the story though, is to remember that every design decision involves trade-offs, and it is important to consider the pros and cons of every choice. In the specific case of choosing when to hard code a parameter - you should hard code it if it will be at least as easy to maintain/change the code as it would be to change/maintain/understand the "soft coded" parameters. It is up to you, the developer, to make this determination.

Gone Programmin' (back next week)

The current TopCoder marathon match runs through this Wednesday and only the top 50 (out of 200 from last week) advance. I am currently in approximately 70^th place so I haven't put any time into this blog this week. The good news is that the Topcoder algorithm tournament first round was also this weekend, and I advanced to compete again. Next week, I will hopefully have something more interesting to say on my blog.

No, Really. Don't Get Stuck in a Rut.

The TopCoder Marathon Match that just finished was a poker problem. They defined a very simple game of poker and a probabilistic strategy their server would follow. My task was to write a program that played 10,000 hands of poker against a specific strategy and try to win. The problem description was couched in terms of trying to learn the server's strategy.

My solution observed the play of every hand and built tables which calculated the probability of the other player's actions. Using these probabilities, my program chose actions that maximized expected value at every point. When I tested this approach and allowed it to play millions of hands, this worked great. Unfortunately, 10,000 hands was not enough hands to get good enough information about the other players actions in all situations.

I spent the rest of the match trying to figure out what short cuts I could make. Things like, "well I haven't seen this betting pattern exactly, but I've seen similar actions, so I will use what I saw in the similar (but not exact) situation as a predictor of the computer's strategy." The other thing I did was hard code a couple rules. I noticed that it took a long time for my solution to learn to fold the bad hands. Over the course of 10 million hands it would learn to fold about 30% of them. In the first 10,000 hands it only folded about 1% of them. I've played enough poker to know that not folding enough is the mistake of most weak players. To address this, I just hard coded the program to fold the weak hands.

This combination of strategies were enough to let me finish in around 60^th place. Since the top 200 advance this round, this is good enough.

So what does this have to do with ruts? Well it turns out that a number of the people who did better than me did not create learning strategies. Instead of trying to figure out how to observe and adapt appropriately, they spent their time coming up with an algorithm that would be good against all strategies. Some of them were able to come up with some very simple strategies which were, nevertheless, able to do better, on average, than my more complicated learning strategy. If we played millions of hands rather than 10,000, mine would almost definitely be better, but that wasn't the problem we had to solve.

I got stuck in the rut of figuring out a learning algorithm, I just never realized it. While it occurred to me to figure out a good fixed strategy, I only thought of that in the context of what my learning program should do until it learned the opponent. Since this felt like just a tweak to my approach, I never got around to doing it.

Moral of the story: sometimes it can be a good idea to try different ideas even if you don't realize you are in a rut.

Don't Get Stuck in a Rut

I was solving a practice TopCoder problem the other day, and was having difficulty. First the problem:

Imagine you are multiplying to positive integers A and B (A >= B) using long multiplication with no carry. For example if A = 89 and B = 76:

       8  9
x      7  6
------------
      48  54
+ 56  63
-------------
  56 111  54

The input to your program would be the array {56, 111, 54}. You have to return A, in this case 89. If there are multiple A's that satisfy the input, you have to return the one that minimizes A-B. To make sure the problem is feasable they constrain it by saying that the result of A*B < 10¹⁴.

I decided to approach the problem by noticing that the the most significant digit (MSD) of the answer was equal to the MSD of A times the MSD of B. I would try each combination of digits that makes this work, and then move on to the next digit. I also noticed that the number of digits in the answer equals the number of digits of A + number of digits of B - 1. So I tried every combination of lengths of A and B that satisfied these constraints, when picking digits. I then returned the one that worked that satisfied the tie breaking condition (i.e. smallest A that is >= B).

This passed all the examples provided, so I submitted it. Unfortunately it timed out on some of the system tests. Since this was a practice, I had access to the system test and copied it locally. I ran this test locally and let it run for a couple hours. When I came back it still hadn't finished. Given that the program has to solve the problem in less than 2 seconds, this was a wee bit too slow.

I finally gave upI looked at my code and found some obvious optimizations. This improved runtime from hours to 10's of seconds. Better, but still too slow. I worked at it a while longer, but just couldn't figure out how to improve my code enough to get it to run fast enough. So I finally gave up and read TopCoder's solution to the problem.

Their solution - just try every value from 1 to sqrt(N) for B. Set A to be N / B. Check if it satisfies the constraints. You can check 10⁷ choices for B in under 2 seconds, and their bounds guarantee that B won't be larger than that.

This solution is much easier to comprehend and code up than what I came up with. Yet, I didn't see it. I was stuck in a rut. This comes up repeatedly when trying to solve a problem, whether it is a small contrived problem like TopCoder or much larger "real-world" problems. Once you see an approach that looks like it might work, it can blind you to other approaches which might be better. This is really just another example of "Don't Want That", where what you need to stop wanting is your current approach.

How do you get out of this rut? The first thing that has to happen is that you have to notice you are in a rut and be open to the idea that other solutions may exist. Given this, how do you find these other solutions? One of my favorite ways is to present the problem I have to someone else without telling them my current approach. A fresh mind often comes up with a fresh solution. What if you don't have someone to discuss it with or, as in a competition, you can't discuss it with people? Then you have to try and clear your mind of everything you know. Start at the problem from scratch, considering every salient fact, paying particular attention to any that your previous approach ignored. Try to come up with the most off the wall things you can think of and pay attention to the reasons they don't work. If you have the luxury of time, put the problem down and don't think about it for a while.

Unfortunately, sometimes it is hard to describe a problem to a fresh person without sullying it by hinting at our own approach. And I often find that no matter how hard I try to come at a problem from a new angle, my mind just slips back into the old rut. So how do you get unstuck from a rut?