Unit Testing - the Bad

Previously I talked about some of the benefits of unit testing. However, it's not all peaches and cream. There are costs to unit testing.

Time
The most obvious cost is the time taken to write the unit tests. If you are trying to test a component which manipulates very complex objects, you first have to generate these complex objects and then you have to validate them after the fact. If you are trying to test a component that is being used in a multi-threaded environment you have to create a mock environment with multiple threads. In these types of scenarios it can take significantly more time and effort to write meaningful unit tests than it did to write the actual component in the first place. The testing environment can be complex enough that it'll also take more time to debug the tests than it does to debug the code.

Dependencies
If you are writing unit tests, you are probably using a framework like JUnit or TestNG. You may also be using additional tools to help reduce some of the complexity discussed in the previous paragraph. However, each of these tools adds a dependency. It's frustrating enough when you can't build a project because you are missing a library. Not being able to build because the tests are missing a library is worse. This can lead to developers just deleting/commenting/ignoring tests so they can go forward, meaning all that time you spent is wasted.

Inertia
One of the biggest challenges to developing software is maintaining code as requirements change. A promise of automated unit tests is that they allow you to refactor code without fear of breaking things. However, a large enough suite of unit tests can actually make it more work to restructure code. When you decide that you need to completely restructure entire packages, destroying classes, and creating new ones, there's obviously a lot of work in modifying the rest of the code to be consistent with these changes. Unit tests can be a large part of your code base, and with large structural changes they will have to also be drastically changed to appropriately test the new code. While logically this is no different than the cost of writing the code and unit tests to start with, psychologically it is different. This added work can encourage you to just tack on changes in the most expedient way. i.e. A large suite of unit tests can actually cause the exact opposite effect that is desired, making you less likely to make valuable architectural changes rather than more likely.

Example
Imagine that you have modified your phone number code from before so that phone numbers are passed around as PhoneNumber objects rather than as Strings. To work with this you modify the PhoneFormatter's format method to take a PhoneNumber object rather than a String. Well, now all of your PhoneFormatter unit tests have broken. While this particular example isn't too much work to fix, especially if you refactor the test code first, it's still extra work. I'm sure you can imagine how larger changes on more complex code could require major reworking of unit tests.

  1 import static org.junit.Assert.*;
  2 import org.junit.Test; 
  3 import org.junit.Before; 
  4  
  5 public class PhoneFormatterTest {
  6  
  7   private PhoneFormatter mFormatter; 
  8  
  9   @Before public void setup() {
 10     mFormatter = new PhoneFormatter(); 
 11   } 
 12  
 13   @Test public void testDirectNumber() {
 14     check("(757) 555-1234", "7575551234"); 
 15     check("(757) 555-1234", "17575551234"); 
 16     check("555-1234", "5551234"); 
 17     check("911", "911"); 
 18   } 
 19  
 20   @Test public void testDialNine() {
 21     check("(757) 555-1234", "97575551234"); 
 22     check("(757) 555-1234", "917575551234"); 
 23     check("911", "9911"); 
 24   } 
 25  
 26   private void check(String expected, String number) {
 27     PhoneNumber pn = new PhoneNumber(number);
 28     assertEquals(expected, mFormatter.format(pn)); 
 29   } 
 30 }