At Rally, we’ve stayed committed for the last 7 years to never telling the business that the product has to halt active development to pay down technical debt. For us, the “big rewrite in the sky” has always been off the table. Instead, we prefer to incrementally refactor and improve the existing elements of the codebase, gradually getting it to where we want it to be without ever halting feature development completely.

This has never been an easy task. The truth is, the codebase is quite old by engineering standards, many parts of it written before standards emerged in the industry. Our entire persistence layer, for example, was created by hand well before libraries such as Hibernate existed (or at least, before they became standard practice). This means that there are many, many areas of the code that we’d like to attack and improve.

One thing we’ve started doing is holding lunchtime “refactotums,” where we take a look at some code and try to improve it. Of course, with such a large codebase, and so many areas of the code that have not been touched since they were first written, we’ve often wondered how to most effectively spend our refactotum time. This led us to wonder “what are the areas of the code where a refactoring would have the greatest impact?”

We started keeping track of our codebase using Sonar which was extremely helpful in keeping track of metrics at a class level, but it didn’t give us quite enough information. For example, if Class A has a complexity score of 500 and Class B has a complexity score of 200 (higher is worse), you’d think that spending some time cleaning up Class A would be the most effective thing to do. But what if Class B is used by 50 times as many other classes as Class A? Or what if Class B, complicated as it is, has never been modified in the history of the project? It would be nice to clean this class up, but it might be a better use of time to focus on Class A

So our idea was to combine two metrics: the class complexity (based on McCabe Cyclomatic Complexity) and the number of times a file has been modified in the codebase. The idea here was, every time a file has to be modified, it has to be read. A complex file is harder to read, making the engineer more likely to misunderstand or make a mistake while editing the file.

Luckily, git makes it quite easy to count the number of modifications to files in the codebase:

git whatchanged | grep '^:.*alm-webapp/src/main/java/.*\.java$' | cut -c65- | sed -e 's/\//./g' -e 's/.java//' | sort | uniq -c | sort -r

Obviously, the grep and cut would have to be modified slightly depending on your codebase structure. The grep is used to ensure only our code is looked at, excluding any third-party code in the system. The cut and seds are mostly just about munging the filenames into “probable” classnames, since that’s more useful (and what Sonar stores).

This command yields some high-traffic files, but it needs to be combined with high-value targets as well. For this, we turned to Sonar.

We were unable to use the Sonar web services as thoroughly as we were hoping, so we wound up writing a query directly against the database where the Sonar data is stored. The query for a given metric (in our case, metric 12, class complexity) looks like this:

    SELECT projects.kee, project_measures.value, project_measures.metric_id
    FROM snapshots, project_measures, projects
    WHERE islast = 1
      AND project_measures.snapshot_id = snapshots.id
      AND project_measures.metric_id = 12
      AND snapshots.qualifier = 'CLA'
      AND snapshots.project_id = projects.id

We wrote a Ruby script that pulled the data from these two sources and correlated them together, deriving an eventual “weighted rank” from the two. We went on to include a few more metrics in the final equation. Output from the script looks like this:

Calculating complexity data from sonar...Done!
Calculating revision data from git...Done!
Calculating weighted rank data...Done!
Current formula:
   ( 0.15 * ( complexity / average_complexity ) )
 + ( 0.40 * ( function_complexity / average_function_complexity ) )
 + ( 0.30 * ( efferent_coupling / average_efferent_coupling ) )
 + ( 0.40 * ( revisions / average_revisions ) )

Rank    Score  Revisions  Complexity  F.Complexity  E.Coupling  Class
----  -------  ---------  ----------  ------------  ----------  -----
   1    14.12         60         507             2          67  com.rallydev.ClassG
   2    13.10         93         301             2          71  com.rallydev.ClassQ
   3    12.53         80         318             2          70  com.rallydev.ClassR
   4    11.25         11          45            45          27  com.rallydev.ClassO
   5     9.65          9          43            43           8  com.rallydev.ClassC
   6     9.48         72         194             2          52  com.rallydev.ClassZ
   7     9.24         60         205             2          56  com.rallydev.ClassN
   8     8.79         42         218             2          59  com.rallydev.ClassK
   9     8.75         31         260             3          53  com.rallydev.ClassF
  10     8.63         54         167             2          59  com.rallydev.ClassT
  11     8.38         65         194             2          39  com.rallydev.ClassO
  12     8.29         44         207             3          49  com.rallydev.ClassY
  13     7.81         40         225             2          41  com.rallydev.ClassP
  14     7.48         18          26            26          23  com.rallydev.ClassK
  15     7.44         60         164             2          34  com.rallydev.ClassA

We’ve found it pretty useful to know where our efforts would be most useful. If you’re using git and sonar, feel free to modify this script and try it on your own codebase (note, you will need the mysql client binary in your path for it to work).

#!/usr/bin/env ruby
# Authors: Rod Hilton and Ryan Scott
# Site: www.rallydev.com

METRICS = {12=>:complexity, 14=>:function_complexity, 74=>:efferent_coupling}
WEIGHTS = {:revisions=>0.4, :complexity=>0.15, :function_complexity=>0.4, :efferent_coupling=>0.3}

def print_around(title, &block)
  print title + "..."
  STDOUT.flush
  yield block
  print "Done!\n"
end

def add_class_data(class_data, class_name, data_key, data_value)
  class_data[class_name] = {} unless class_data.has_key?(class_name)
  class_data[class_name][data_key] = data_value
end

def add_complexity_data(class_data)
  complexity_query = <<-EOF
    SELECT projects.kee, project_measures.value, project_measures.metric_id
    FROM snapshots, project_measures, projects
    WHERE islast = 1
      AND project_measures.snapshot_id = snapshots.id
      AND project_measures.metric_id in (#{METRICS.keys.join(",")})
      AND snapshots.qualifier = 'CLA'
      AND snapshots.project_id = projects.id
    EOF

  print_around "Calculating complexity data from sonar" do
    complexity = `echo "#{complexity_query}" | mysql -h alm-build --user=YOUR_SONAR_USERNAME --password=YOUR_SONAR_PASSWORD --database=YOUR_SONAR_DATABASE`

    complexity.each do |complexity_line|
      match = complexity_line.match(/^(\S*)\s+([\d\.]+)\s+(\d+)$/)
      if(match)
        name = match[1]
        metric = match[2].to_f
        metric_id = match[3].to_i
        metric_name = METRICS[metric_id]
        add_class_data(class_data, name.match(/^.*:([^:]*)$/)[1], metric_name, metric)
      end
    end
  end
end

def add_revision_data(class_data)
  print_around "Calculating revision data from git" do
    file_changes = `git whatchanged | grep '^:.*alm-webapp/src/main/java/.*\.java$' | cut -c65- | sed -e 's/\\//./g' -e 's/.java//' | sort | uniq -c | sort -r`
    # file_changes = `cat modified.txt`

    file_changes.each do |file_change_line|
      match = file_change_line.match(/^\s*(\d+)\s+(.*)$/)
      if(match)
        revisions = match[1].to_i
        name = match[2]
        add_class_data(class_data, name, :revisions, revisions)
      end
    end
  end
end

def average_for_data(class_data, key)
  values = class_data.values.collect{|val| val[key]}
  values.inject{ |sum, el| sum + el }.to_f / values.size
end

def add_weighted_rank_data(class_data)

  print_around "Calculating weighted rank data" do
    averages = {}
    WEIGHTS.each do |key, value|
      average = average_for_data(class_data, key)
      averages[key] = average
    end

    class_data.each do |classname, data|
      data[:weighted_rank] = 0
      WEIGHTS.each do |key, weight|
        chunk = weight * (data[key] / averages[key])
        data[:weighted_rank] = data[:weighted_rank] + chunk
      end
    end
  end
end

def remove_classes_without_all_data(class_data)
  num_keys = class_data.values.map{|val| val.size}.max
  class_data.delete_if{|key, val| val.size < num_keys}
end

# --- Main ---

class_data = {}

add_complexity_data(class_data)
add_revision_data(class_data)

class_data = remove_classes_without_all_data(class_data)

add_weighted_rank_data(class_data)

#Sort by rank, descending
class_data_sorted = class_data.sort{|a,b| b[1][:weighted_rank] <=> a[1][:weighted_rank]}

puts "Current formula: "
pieces = []
WEIGHTS.each do |key, weight|
  pieces << sprintf( "( %3.2f * ( %s / %s ) )", weight, key.to_s, "average_"+key.to_s)
end
puts "   " + pieces.join("\n + ")

puts "\n"
puts "Rank    Score  Revisions  Complexity  F.Complexity  E.Coupling  Class"
puts "----  -------  ---------  ----------  ------------  ----------  -----"
class_data_sorted[0..49].each_with_index do |file, index|
  class_name=file[0]
  class_data=file[1]
  printf("%4u  %7.2f  %9u  %10u  %12u  %10u  %s\n", (index+1), class_data[:weighted_rank], class_data[:revisions], class_data[:complexity], class_data[:function_complexity], class_data[:efferent_coupling], class_name)
end

Feel free to make any improvements or changes to this script that you wish. It’s not the most efficient in the world, but it was thrown together pretty quickly to get a decent idea of our highest-impact refactoring targets.

Suggestions for improving the formula? Made your own changes to the script? Feel free to post in the comments below, we welcome the discussion!

Through hours of trial and error, I found something that seems to work pretty well. Here are the caveats:

1. It does not work in Internet Explorer
2. It requires CSS3
3. It uses alpha colors, which means it’s a bit more CPU-intensive than text normally is.

Otherwise, it seems like a good solution. This works in Firefox for Linux and Mac and Chrome for Linux and Mac. It does not work in IE at all, and it doesn’t work in Safari (last time I checked).

Here it is:

How It Works

Let’s start with some basic text in a div:

<style>
.threed {
  font-size: 250%;
  text-align: center;
  border: 1px solid black;
  margin: 1em;
  border: 1em;
}
</style>

<div class="threed">
Hello world!
</div>

First, you can apply a text-shadow using a CSS3 property called text-shadow like so:

.threed {
  font-size: 250%;
  text-align: center;
  border: 1px solid black;
  margin: 1em;
  border: 1em;
  text-shadow: .5em 0 .2px rgba(0%, 100%, 100%, 0.6)
}

Here we put a text-shadow on our text that is offset .5em horizontally and 0em vertically. It has a ‘blur’ of .2px, which means virtually none at all. The text color is a cyan, at .6 alpha transparency.

The text-shadow element actually allows us to specify multiple text-shadows, so we can just add a second shadow now, this one pink, still .6 alpha transparency, but offset .5em to the LEFT instead of the right (using a negative offset)

.threed {
  font-size: 250%;
  text-align: center;
  border: 1px solid black;
  margin: 1em;
  border: 1em;
  text-shadow: .5em 0 .2px rgba(0%, 100%, 100%, 0.6), -.5em 0 .2px rgba(100%, 0%, 0%, 0.6);
}

The last thing we’ve got to figure out how to do is hide the black text itself. This is actually easy, since you can apply alpha transparency to any color using CSS3. In this case, we’re simply going to make the text fully transparent.

/* 3D CSS Effect: www.nomachetejuggling.com */
.threed {
  font-size: 250%;
  text-align: center;
  border: 1px solid black;
  margin: 1em;
  border: 1em;
  text-shadow: .5em 0 .2px rgba(0%, 100%, 100%, 0.6), -.5em 0 .2px rgba(100%, 0%, 0%, 0.6);
  color: rgba(40%, 40%, 40%, 0.0);
}

This effect tends to work best for:

• Small font sizes
• Lots of text
• Normal weight (bold text looks bad)

You can see this technique used in a lengthy parody of the movie Clash of the Titans here.

Degradation

This seems to degrade really well in Internet Explorer, which supports neither alpha transparency on text nor text-shadow. As a result, it shows up as just plain black text.

It degrades very poorly in Safari however, which supports alpha transparency on text but not text-shadows, meaning that the text is rendered invisible without the shadow, so it just shows up as a big blank box.

Improvements

I fiddled with a lot of different parameters to try to perfect this, and what you see here is as close as I was able to get. The trick with 3D text is that you want the overlapping areas between the pink and cyan to be dark enough to show up for both eyes, the result of multiplying the cyan and pink colors together. With the CSS, it’s a bit lighter than I’d like, but it seems to do the job.

I also tried to do it in a way that simply made the main color pink, with a single cyan shadow (and vice versa) but I found that, without that slight blur parameter (.2px), the colors didn’t seem to blend quite as well, hurting the effect.

If anyone out there figures out a way to improve this, please post a comment here with your tips.

In the post, Shore basically argues that certifications, particular those relating to Agile software development, have no value. In fact, he goes on to argue that they offer counter-value, that some employers are LESS likely to hire someone with a certification.

The thing is, the main reason this bothers me is that I have a few certifications from Sun (pre-Oracle days), and the process I went through to acquire these certifications is one I found immensely useful. It’s hard for me to see something as meaningless when it holds so much value to me.

This is not to say I don’t understand the point. The certification merely proves that I took a test, and considering how low a score you can get and still “pass” the test and receive the certification, it proves little more than that I read a book or took a course. I agree with this basic assessment of certifications: as a token of proof to offer someone else that I know my stuff, they have very little value.

I like certifications, however, because they help me prove to myself that I learned something. When I was studying for the SCJP exam, I did so not through a course, but through a book (two books, in fact). As I studied, I was constantly learning things about Java that I didn’t know – low-level details of the language, its syntax, the way it handles inheritance, and so forth. Same thing when I studied for the SCWCD exam: I was, at the time, at a job that did mostly ColdFusion for web layer, and I had no experience whatsoever with Servlets, JSPs, custom tags, or anything in that area. The process of reading a book (again, two books) for the SCWCD exam was, primarily, where I learned about web components in Java. Because of the knowledge I gained during that process, I was able to later get a new job that actually used J2EE technology for the web, and I knew my way around enough to be productive. If I had not read those books, I probably wouldn’t have survived the interview, and if I had I wouldn’t have known what I was doing well enough to do my job.

But really, this is the value of reading, not of certification. Wouldn’t I have gotten the same benefit from merely reading a book on Java or J2EE? Not necessarily. Most books on something like Java are focused on specific applications of it – they are positioned as “here’s how to use Java to do so-and-so” or “here’s how to use J2EE to write the kind of web apps you may write in real life.” Books like these don’t do a deep-dive in the core of the language itself or the standards. Books that are intended to help you study for a certification exam, however, are forced to leave no stone unturned in the subject area – forced in a way that a typical book may not be. An book author writing about JSPs may simply have never written a JSP custom tag in Java (perhaps he or she only worked with tag files) and, as a result, they may only devote a handful of pages to a surface level exploration of that topic. The SCWCD study guide had to dive all the way into that subject in order to set readers up with the best chance to pass. Obviously, a really good book would cover all of the material, but the vast majority of books aren’t going to qualify as “good” in my opinion, and finding that perfect book can be challenging. In the other hand, you can be relatively confident that a certification study guide will cover the entire subject in complete detail.

But again, this shows the value of reading books, just very specific ones. Why bother with the exam? For the same reason you bothered taking tests or getting grades in school – to receive an objective evaluation of how well you understood the material. If I study for the SCJP exam, then get a 65% (passing, humorously enough), then you can be reasonably certain you understand approximately 65% of the material. If that’s not enough for you, the exam actually tells you the specific areas in which you are weak (when I took the SCJP exam, I was helpfully informed my weakest area was with knowledge of the standard library) so you can focus on those areas if you wish. The value of a test is that someone who knows more than you about a subject can give you a reasonable assessment of how well you’ve learned that subject. That’s all a certification exam is: a test on a specific subject.

I find myself much more aligned with Bob Martin’s view. A certification has value in that it is a mechanism for instruction. Whether you take a course or read a book, you are learning, and learning has value, always.

Personally, I much prefer a certification that requires an exam, rather than merely a course. A certification without the examination seems to miss the point, at least for me. My Certified ScrumMaster certification has less value to me than my Sun Certified Java Programmer certification, because the former was given to me after attending a class for two days, while the latter was something I studied for over the course of a couple months, and ended with an exam proving to me that I understood the material.

Essentially, I view a certification as a thing I do for myself primarily, to help me learn a subject at a deep level. Why bother even listing it on my resume? Because what it shows, at least in my opinion, is that I’m self-motivated and I like to learn and improve on my own. My certifications prove, at the very least, that I’m interested enough in programming that I took time out of my normal life to study a topic in depth and take a test. It’s not exactly the promise of the typical certification, but that combined with how much I enjoy learning when I am studying for a certification makes them valuable to me.

I certainly hope Shore’s posts about people passing on candidates with certifications is hyperbole. I would never suggest that a certification should serve as a substitute for a real interview; I may have the SCJP certification, but if you want to know if I really know my Java you should ask me lots of Java questions in an interview. It bothers me to think that something I’ve greatly enjoyed doing – something that I feel has made me substantially more knowledgeable in certain areas – would reflect poorly on me as a job candidate.

I’m going to continue operating as though Shore is exaggerating a bit to illustrate a point. There are other Sun certifications (Oracle certifications? Weird.) I plan on studying for and earning, and there may be other certifications beyond those. As far as I’m concerned, learning always has value, and examinations always help one know how well they are learning. And hey, if you get a spiffy-looking piece of paper afterwards, that’s a bonus.

Sketch out the object model for a x

Generally this question is asked to feel out someone’s OO skills. Do they understand inheritance, polymorphism, abstraction, etc? Can they describe the relationship between a plant, flower, and petal (a flower is-a plant, and has-a petal)?

I used to ask this question myself when I was first starting out doing interviews. I would ask the candidate to describe the object model for a Chicken. I was expecting the candidate to sketch something resembling the following:

Now, I wouldn’t expect it to be perfect UML or anything like that, but I wanted to see the basic idea that the candidate knew that a Chicken was probably a Animal or a Bird, and maybe Bird is an Animal, and maybe Chickens have Feathers or Beaks, or maybe those belong to Bird, etc. I thought this was a great question for gauging a candidate’s comfort with Object Oriented Programming.

I was wrong.

This is actually a terrible question. I stopped asking it years ago because I never seemed to get an answer I liked, which told me that it was a bad question. Only recently, however, have I really come to a full understanding of why it’s such a bad question.

Evolutionary Design

A good software developer does not sit down and sketch out all of the objects in the system and their relationships. Doing so would be a surefire way to add unnecessary complexity to an application. One particularly interesting example of this can be found here, a semi-transcript of Bob Koss and Bob Martin sitting down together to create a small program to keep track of bowling scores.

When the Bobs start, they sketch out a Game class, Frame class, and Throw class. But when they start to write their tests for the Throw class, they realize they have none because it has no behaviors. They start to write some tests for the Frame class and discover none there as well. They finally start by writing tests for the Game class itself, which helps them drive all the way to completing the application. In the end, they only had two classes: Game and Scorer.

When trying to think of the classes outside of the context in which they would be used, the two Bobs failed. Once they began writing tests to drive out the behavior, they found that the responsibilities they needed belonged not to three different classes, but to two. The behavior of the system drove the design.

This is a fundamental principle of Agile software development (and, more specifically, XP): Evolutionary Design. Good programmers follow the practice of Evolutionary Design because it prevents the system from being over-engineered and overly complex while incrementally making the system more and more resilient to change. This is a good practice, and I want to hire people who follow it, but when I asked them design the object model for a Chicken, I am essentially asking them to do Big Design Up Front, something I’d prefer them not to do if they join the team. I’m setting myself up for failure, because anyone who answers this question well may have a tendency to do something I don’t really like.

Context is Key

Imagine asking a similar question to the Chicken question above. You request the candidate “design the object model for a Car.”

Maybe you’re expecting the candidate to write something like this:

Seems reasonable. But imagine that the application for which this Car has been designed in a full-fledged CAD-like application for a car designer. The Car will be used by the designer to help determine how the car looks, how the exterior works, how many doors it has, and so on. It may look like this:

Or imagine instead, that the Car class will be used in a traffic simulator program that’s really meant to test traffic congestion, stop light placement, and so on. In that application, a Car is really just something with a direction and a velocity, so it would look more like this:

In the former, the future changes your design should guard against are new types of doors, or new components for a car. There is no danger of a different kind of Vehicle, such as a Plane, because this is for a Car company. In the latter, you may need to guard against additional vehicle types such as Bicycle or Scooter. The design that gets evolved by these two systems will be fundamentally different, so why ask the question without providing information about the system itself?

Ultimately, the problem here is that there is a spectrum of answers to this question, ranging from extremely simple to extremely complex. But depending on the context, any one of those designs can be the right design. When an interviewer asks the question, theres a specific place on this spectrum where he or she would like the candidate to answer, and anything below that indicates a poor understanding of OO, while anything above it indicates a tendency to overdesign. The interviewer has set up the candidate to fail.

How To Ask This Question

If you want to know how comfortable a candidate is with OO, ask them the question in a way that resembles real-life OOP a bit more closely. Start simple, by providing a single simple requirement. Though the candidate won’t be able to write and run tests, the requirement serves as a driver for the design.

A good indicator that the question is taking you and the candidate off-track is if the boxes he or she draws on the whiteboard lack method names. If there are no methods, there are no behaviors, which means the candidate is designing first, regardless of requirements. Steer the candidate back by asking them to write method names.

For example, ask the candidate to design the object model for a simple bookshelf. A bookshelf is so simple that there’s virtually no way to overcomplicate it, so you can say that you simply need to be able to add a book to the bookshelf, nothing else. Once they have done this, give the candidate another “test”, by adding a requirement. Now say that you want the bookshelf to be a “smart” bookshelf, where you can look up a book by title and get a reference to the book. The candidate will make some changes to their design. Continue adding requirements to the thing until you’re satisfied with the evolution of the candidate’s design.

• Modify the design to allow me to search the bookshelf by Title, Author, or ISBN
• Allow me to use the bookshelf as an ad-hoc library, so I can “check out” a book, removing it from the bookshelf. Make it possible to look up books that have been “borrowed”.
• Require a user provide their name when checking out a book. Give them a return date. Make it so the bookshelf can generate a list of overdue books and who has them. Where does that method belong? Should it be on its own object that Bookshelf uses?
• Make the bookshelf more concrete, so that there is only a certain number of books that can fit on any given shelf in the overall bookshelf. Make it possible to ask which shelf a book is on. Make it possible to add a new shelf.

You can obviously pick your own model (Car, Elevator, Fridge, etc) and drive the direction the design should go using requirements. Have the candidate write method names (but not implementations) where appropriate. Keep asking if the method is on the right object, or it belongs on another one.

How To Answer This Question

Of course, not everyone reads this blog, so if you’re in an interview you may get the “design an object model for a x” question. If you find yourself in this situation, you essentially want to ask questions of your interviewer to pull requirements out of them. The reality is, they probably DO have a particular kind of system in mind when they ask the question, so you need to find out what that is.

If you’re supposed to design the Car class, ask if there are other types of vehicles in the system. If not, don’t make a Vehicle parent class, and explain that you see no need unless the system required additional vehicles. Before you draw a box named Tire, ask if the car needs to be able to support snow tires or some other kind of interchangeable tire. If not, why would you make a class or interface for them? Start as simply as you can and only draw a new box when you’ve extracted enough information from the interviewer to deem it warranted. If you jump up to the whiteboard and draw two or three boxes before asking any questions, you’re placing your interview at risk because the interviewer may be picturing a completely different usage of this system than you are.

Of course, it’s always possible that the interviewer will keep asking for the same information, arguing that you should show your OO skills off without requirements that drive that design out. If you want the job, your best bet there is to take a random stab at the appropriate level of complexity and hope that you strike somewhere near what the interviewer wants. In that situation, however, I’d imagine your interviewer doesn’t know much about designing good software, and their codebase might be an over-engineered mess, so you may not want that job.

One of the most common sources of tension between product owners and developers is when product owners are surprised at how high an estimate for a story might be. Usually this tension is easy to resolve by reiterating that the product owners really have no concept of how much something should cost. However, there is one scenario I see over and over again: when a product owner protests the estimate of a story because it seems, to the PO, like it’s simply re-using an aspect of the system somewhere else.

“I already paid for that!” he or she might say, quite correctly. A simple example recently for us here was our column picker. Our grids allow you to select which columns you wish to see on any given grid. Later on, we created the ability to have grids inside of panels on user dashboards. Our product owner wanted to put the column picker on the settings pane of a grid panel. To him, this was simply reusing the column picker that had already been created for an earlier story. He was quite shocked when our estimate for the effort was just as high as it was the first time.

His frustration is completely understandable. How is this possible? Why isn’t it easy to just drop that component into another place and have it work? After all, if we were good engineers, wouldn’t we have written that component to be reusable?

Actually, no. It’s because we were good engineers that it wasn’t written in that way. This is one of the hardest things for product owners to understand, and I hope to explain it in this post.

Evolutionary Design

Once upon a time, development teams worked in specific phases. First they would design all of the code they were going to write; boxes would be drawn on whiteboards, sequences captured in documents, and so on. Once the design was finished, the developers would go off and write the code for it. Luckily, our industry has realized that this is a problematic way to write software.

It’s simply not likely that the team will discover all of the quirks of the design during the design phase. It’s even more likely that requirements and features will change during the long span of time this process takes, forcing the developers to throw away work and go “back to the drawing board” on the design. Splitting the design phase from the coding phase is a recipe for disaster.

Instead, it’s generally preferable to practice evolutionary design; allow the design to grow organically as the code is written. This, combined with testing and refactoring, seems to be the best known way to ensure that the quality of the code remains high as a product grows.

Doing this makes it easier to deal with changes that effect design, and it prevents the team from wasting time going “back to the drawing board” in the event a problem is discovered.

YAGNI

One principle to follow as you code is the YAGNI principle. YAGNI means “You Ain’t Gonna Need It” and essentially it means that developers should discourage themselves from adding additional code to support something that is not, at the time, required. Developers have a strong desire to add code here or there while editing code, and YAGNI helps us remember to resist the urge to add superfluous code or functionality.

Following this principle helps eliminate waste, and helps keep developers focused on adding value that product owners want. But why?

If I’m editing Module A to add Functionality X, why not just add Functionality Y while I’m there? It may delay the delivering of Functionality X in the short-term, but later when it’s decided that Functionality Y must be supported as well, it will be dirt cheap, right?

The trick to understanding why YAGNI helps is realizing that there are thousands upon thousands of potential Module As in a system. The truth is, Functionality Y may simply never be required. And even if it is, Functionality Z may not be. Adding that functionality would be a waste of time and resources if it’s never used.

YAGNI Applied to Design

The same principle behind YAGNI can be extended to aspects of high-level design. When we originally designed the Column Picker component, it was not designed to be dropped into any part of the system. This was intentional: if we had spent the time to make the Column Picker completely generic and reusable, but then had never needed to drop it into another part of the system, the additional effort required to make it super-generic would have been a wasted effort. This is true for any aspect of the system being designed: we can spend the extra time and effort to make something extremely generic, but if that generality is never needed, that time is waste. Many times developers will refer to something that’s overly generic as “over-engineered.”

This means that the description of the component can often be misleading. We never built “a column picker” – we built “a column picker for a grid page”. The column picker was written in a way that assumed it was part of a grid page. Again, it could have been written more generically, but doing so would have potentially created waste (this does not mean that it’s alright to design code that violates principles of good design. Code can be well-designed but still not fully generic).

So when it came time to put the component in a completely different context than the one for which it was designed, it was still a lot of work: almost as much as was originally required to make the first pass. Though some time was saved simply because it was a known and well-understood problem for which we would leverage newly-developed knowledge, it still effectively required writing a whole new component: one that was much more generic and utilized what it could of the previous increment.

Essentially, to avoid having developers waste time, components should be “generic on demand”. Once a component needs to be reused, then it should be refactored into something extremely generic, but not before. The end result is that a product owner will have to pay for a component twice before it’s cheap.

This may seem counterintuitive to product owners, but it’s important to understand. Because the team approaches coding in a way that causes the second time to be nearly as expensive as the first, the team is able to deliver everything else quickly.

The first time, it is costly because the developers have to solve unknowns and create something new from scratch.

The second time, it is costly because it has to be molded into something generic and reusable.

The third time, however, is cheap.

It’s the result of many months of research, I hope others may find it interesting and informative.

Abstract

Test-Driven Development is a Software Engineering practice gaining increasing popularity within the software industry. Many studies have been done to determine the effectiveness of Test-Driven Development, but most of them evaluate effectiveness according to a reduction in defects. This kind of evaluation ignores one of the primary claimed benefits of Test-Driven Development: that it improves the design of code. To evaluate this claim of Test-Driven Development advocates, it is important to evaluate the effect of Test-Driven Development upon object-oriented metrics that measure the quality of the code itself. Some studies have measured code in this manner, but they generally have not worked with real-world code written in a natural, industrial setting. Thus, this work utilizes Open Source Software as a sample for evaluation, separating Open Source projects that were written using Test-Driven Development from those that were not. These projects are then evaluated according to various object-oriented metrics to determine the overall effectiveness of Test-Driven Development as a practice for improving the quality of code. This study finds that Test-Driven Development provides a substantial improvement in code quality in the categories of cohesion, coupling, and code complexity.

Download

If you’d like to read it, you can download it here:

Quantitatively Evaluating Test-Driven Development by Applying Object-Oriented Quality Metrics to Open Source Projects

Many developers think of unit tests as tests that test a single class. In fact, I myself once thought this way. If I wanted to write unit tests for a two-class system in which a class used another class, I’d write two unit tests. After all, if I created instances of both classes in my test, that wouldn’t really be a unit test, would it?

In recent years, I have revised my stance on this matter. This distinction between what is and is not a unit test is one I no longer draw in the same way.

Unit Tests need to be:

• Fast – Unit Tests should never run so slowly that they discourage developers from running them all the time.
• Focused – Unit Tests should focus on a single area so that they can isolate problems to that area when they fail.

Once upon a time, I thought ‘focused’ meant that it should be focused on a single class. Instead, it should focus on an area, but that area can include multiple classes.

An Example

Let’s say we have a simple Calculator class with some tests (normally we’d write the tests first, but since we’re talking about the tests in this post we’re going to write them afterwards). As with any example fabricated for demonstration purposes, the classes in this post have little need for the level of testing I will be applying to them, but they serve an illustrative purpose.

package com.rallydev.engblog.unittesting;

public class Calculator {

  public int calculate(char operation, int operand1, int operand2) {
    switch (operation) {
    case '+':
      return operand1 + operand2;
    case '-':
      return operand1 - operand2;
    default:
      throw new UnsupportedOperationException(String.format(
          "'%s' is not a known operation", operation));
    }
  }

}

Here is the test for this class:

package com.rallydev.engblog.unittesting;

import static org.junit.Assert.assertEquals;

import org.junit.Before;
import org.junit.Test;

public class CalculatorTest {

  private Calculator calculator;

  @Before
  public void setUp() {
    this.calculator = new Calculator();
  }

  @Test
  public void shouldAdd() {
    assertEquals(5, calculator.calculate('+', 2, 3));
  }

  @Test
  public void shouldSubtract() {
    assertEquals(1, calculator.calculate('-', 5, 4));
  }

  @Test(expected = UnsupportedOperationException.class)
  public void shouldThrowExceptionForUnknownOperation() {
    calculator.calculate('Q', 1, 2);
  }
}

But after thinking about it, we realize that the responsibility for parsing what the caller wants to do and the responsibility for actually performing that operation should be separated.

So we’re going to create a new class, CalculationProcessor, which actually handles the calculations themselves.

package com.rallydev.engblog.unittesting;

public class CalculationProcessor {
  public int add(int operand1, int operand2) {
    return operand1 + operand2;
  }

  public int subtract(int operand1, int operand2) {
    return operand1 - operand2;
  }
}

The tests for this class should be obvious, so we won’t go into them.

We will also modify our Calculator class to be injected with CalculationProcessor at construction time, which it will use to perform the operations. That looks like this:

package com.rallydev.engblog.unittesting;

public class NewCalculator {
  private CalculationProcessor calculationProcessor;

  public NewCalculator(CalculationProcessor calculationProcessor) {
    this.calculationProcessor = calculationProcessor;
  }

  public int calculate(char operation, int operand1, int operand2) {
    switch (operation) {
    case '+':
      return calculationProcessor.add(operand1, operand2);
    case '-':
      return calculationProcessor.subtract(operand1, operand2);
    default:
      throw new UnsupportedOperationException(String.format(
          "%s is not a known operation", operation));
    }
  }
}

So the question is, what should the test for NewCalculator look like?

If we follow the strict “One Unit Test, One Class” paradigm, we have to create a mock for the CalculatorProcessor and pass it into the Calculator.

One possible way it could work is by mocking out the add and subtract methods so that they have mock implementations which actually perform the operations. That solution really only works here because this example is so simple (the implementation of add is trivial). Were this problem more real-world, mocking out the responses based on the inputs may not be feasible, so let’s imagine that we cannot do this for the purposes of this post (when mocking out the responses is possible, most of this post is rendered moot, as that is usually the best way to go).

The other way to write these tests would be to create a mock and verify that our NewCalculator delegates to CalculationProcessor appropriately. It would like this:

package com.rallydev.engblog.unittesting;

import org.junit.Before;
import org.junit.Test;

import static org.mockito.Mockito.*;

public class NewCalculatorBadTest {
	private NewCalculator calculator;
	private CalculationProcessor mockCalculationProcessor;

	@Before
	public void setUp() {
		this.mockCalculationProcessor = mock(CalculationProcessor.class);
		this.calculator = new NewCalculator(mockCalculationProcessor);
	}

	@Test
	public void shouldAdd() {
		calculator.calculate('+', 2, 3);
		verify(mockCalculationProcessor).add(2,3);
	}

	@Test
	public void shouldSubtract() {
		calculator.calculate('-', 2, 3);
		verify(mockCalculationProcessor).subtract(2,3);
	}

	@Test(expected=UnsupportedOperationException.class)
	public void shouldThrowExceptionForUnknownOperation() {
		calculator.calculate('Q', 1, 2);
	}
}

I hate this test. This test does little but double-check the implementation of NewCalculator.

This test is extremely brittle. If the calculator had more responsibilities (say, multiplication, exponents, etc.) and the tests were more thorough (boundary checking, looking for division by zero, and so on) then changing the implementation of NewCalculator would require changing dozens of broken tests.

The tests are there to make us feel safe about refactoring the code, but virtually any refactoring breaks them, which requires us to manually fix the tests. When this happens, a refactoring effort loses its test harness: the code and the tests must both change together, rather than one at a time with the one verifying the correctness of the other.

Instead, I would prefer to leave the test as it was, only modifying the setUp method to correctly construct the NewCalculator with real instantiations of its dependency.

package com.rallydev.engblog.unittesting;

import org.junit.Before;
import org.junit.Test;

import static org.junit.Assert.assertEquals;

public class NewCalculatorBetterTest {
  private NewCalculator calculator;

  @Before
  public void setUp() {
    CalculationProcessor calculationProcessor = new CalculationProcessor();
    this.calculator = new NewCalculator(calculationProcessor);
  }

  @Test
  public void shouldAdd() {
    assertEquals(5, calculator.calculate('+', 2, 3));
  }

  @Test
  public void shouldSubtract() {
    assertEquals(1, calculator.calculate('-', 5, 4));
  }

  @Test(expected = UnsupportedOperationException.class)
  public void shouldThrowExceptionForUnknownOperation() {
    calculator.calculate('Q', 1, 2);
  }
}

There are a few things to notice about this test. First and foremost, the bodies of the three test methods are now identical to how they were before the refactoring. This means that the refactoring could have taken place while using the test to ensure it was correct, though the setUp method would have had to change.

Secondly, even though a real CalculationProcessor was instantiated for the test, it was not assigned to a field. It is scoped exclusively to the setUp method. This is because the test really has no reason to know about the class beyond the fact that it needs it to construct the NewCalculator.

Blurring the Line

Is this a unit test, or is it something else? Unit-testing pedants would argue that it’s not a unit test, but an integration test. Michael Feathers may disagree:

“In the industry, people often go back and forth about whether particular tests are unit tests. Is a test really a unit test if it uses another production class? I go back to the two qualities: Does the test run fast? Can it help us localize errors quickly?” (Source: Working Effectively with Legacy Code)

The test above is extremely fast, but how effectively does it help us localize errors? If it were the only failing test for a system full of classes and tests, it would help us narrow down the error pretty quickly: there is either a problem parsing the operator, or a problem performing the operation.

If we had a separate test for the CalculationProcessor itself, but it was passing, then we would know the error is in the parsing. It didn’t help us localize the problem quite as quickly as the mocking test would have, but it is also a far stronger test, less likely to break under the strain of a refactoring.

It’s a trade-off, of course, but I now tend to lean toward creating instances of the dependency classes in the setup of the test in these situations. I have simple guidelines to help me determine when I should create a mock versus a concrete instance of a class.

Verify() is a Crutch

When you’re writing a test, you go through three phases, which BDD proponents refer to as the Given, When, and Then. First you write the Givens, which attempt to fabricate a pre-existing situation for the purposes of your test. Then you write the Whens, which are the operations being tested. Last you assert some Thens, which are the expected results of performing the operation.

In the case of our simple unit test above, the Given is that a NewCalculator exists. The When is that calculate is called with some parameters. And then Then, of course, is that we get the desired result. But how do we write our Thens?

My feeling is that, if we reach into our toolbox and pull out a verify to write our Thens, we are cheating our tests out of the chance to be truly valuable. When we reach for verify, we are creating brittle tests that resist refactoring rather than enable it. We are using verify as a crutch, but we should only reach for it as a last resort.

Instead, reach for concrete instantiations of classes where possible. Create a mock that will be used to verify method invocation for one of two reasons:

1. Speed – If creating an instance of the class will significantly slow down the test, it should be mocked out instead. This is likely to be the case for classes that clearly turn tests into integration tests, such as classes that facilitate a connection to a database or over a network. This may also be the case for classes that perform a great deal of processing.
2. Complexity – Create a mock if creating an instance of a class requires creating a number of additional classes that it depends on, which in turn depend on more classes, and so on. You do not want your setUp methods to contain thousands of lines of object instantiation to support your tests. I recommend creating real objects as deeply down the dependency tree as possible, then mocking out the rest. How far down the tree should you go? That’s really a matter of taste, but I recommend trying two layers deep and no more. When you do this, you are likely to find that you have no need to verify the calls on the mock objects (other tests already cover those scenarios), but instead can set up mock objects to respond in specific ways to method invocations, simulating their behavior for the purpose of the test so that your tests can verify actual results.

Going Forward

Writing tests without the hard Unit/Class restriction allows us to write tests that enable the safe refactoring of classes, but come at the potential price of slowing down our tests and complicating the setUp methods for those tests. There is a delicate balance there, but in general I recommend trying to err on the side of concrete class instantiation in tests, at least over method invocation verification (mocking out responses is always still preferred).

This view is, of course, subject to change (as it was when I felt that the restriction was valuable), but since blurring this line my tests have remained fast and, more importantly, allowed me to code faster by letting me refactor when necessary.

Agree? Disagree? Leave a comment.

But for the last year I’ve been a very happy T-Mobile G1 user. I love the Android operating system, I love the Google integration, I love the (relative) openness of the device and the near perfect openness of the application market. I cannot imagine using a phone that doesn’t run Andoid any time soon.

Despite loving the software on the G1, I’ve never been completely crazy about the hardware itself. The screen sliding feels a little cheap to me, the phone generally feels a bit sluggish, and the trackball becomes nearly worthless after significant use. Overall, these are minor annoyances, but annoyances nonetheless. The G1 always felt like something of a knockoff iPhone; it was the thing you got when you couldn’t afford a slick, sleek iPhone from Apple. It has features like an iPhone, but generally inferior hardware.

So when I started seeing the Verizon Droid, I got excited. I was eager to switch back to Verizon (I had switched from Verizon to T-Mobile to get the G1 when it first came out) due to Verizon’s superior network coverage and the fact that my wife is a Verizon user. Moreover, the phone looked simply stunning to me. Solid, industrial, sleek, thin (or at least thinner than the G1). Overall, a bad-ass looking phone.

Unsurprisingly, I purchased a Droid the day it was released. In fact, I suffered the early termination fee with T-Mobile in order to get it.

I’ve had the phone for a day now and I wanted to offer some comparisons between it and the G1.

The Good

The hardware of the Droid is solid. It feels heavy and sturdy, sliding the screen to reveal the keyboard differs from the cheap spring of the G1 a great deal (it’s almost, but not quite, difficult to slide). The phone is extremely fast and responsive. Applications start quickly, process quickly, and generally seem speedier. The screen of the Droid is significantly larger than with the G1, and the clarity and resolution of the Droid’s screen are simply stunning. My wife’s contact picture actually looks like her.

The phone is only slightly thinner than the G1, but it makes a huge difference – it feels FAR less bulky in my pocket. The obnoxious upward curve of the bottom of the G1 is gone, replaced by a solid boxy square on the Droid. The front of the phone is devoted almost entirely to the screen. The trackball of the G1 (which becomes less and less responsive over time) has been replaced by a directional pad, and that pad can be accessed only by sliding the keyboard out, devoting even more of the phone to the screen itself.

The power button is in a more convenient place on the Droid as well. Whereas the power/lock button on the G1 was somewhat awkward to use(I always worried about dropping the phone when pressing it to darken the screen before putting it back in my pocket), the Droid’s power/lock button is easy to push and encourages holding the phone in a more solid hand configuration. The volume keys have been moved to the opposing side of the phone, but that doesn’t seem to make a difference either way.

Additionally, the vibration seems a bit stronger on the Droid. I always leave my phone in vibrate mode, but the G1 vibration was pretty weak. Often I’d get a text or a phone call and not even feel it in my pocket. So far, that hasn’t been a problem with the Droid, whose vibration is powerful without being loud. Battery life is far better with the Droid as well – a full day of use with the G1 left the battery dead if I didn’t charge it during the day at least a bit. A full day of use with the Droid doesn’t even get the battery into the yellow, and that’s with the screen significantly brighter.

One feature I particularly like is that the Droid comes with a devoted headphone jack. Listening to podcasts on the G1 was a pain, as it required using an awkwardly long adapter cable plugged into the USB port. This also prevented me from charging the device while listening to something, something I often wanted to do in the car. The Droid has a separate charging port and headphone jack, which is far easier to work with.

Most importantly, the phone connects to Verizon, so I have significantly better 3G coverage than I did with T-Mobile, especially in conference rooms at work (where I need the comfort of Google Reader the most).

The Bad

The Droid isn’t without it’s faults, unfortunately. The Droid keyboard is vastly inferior to the G1’s keyboard. Firstly, there are only 4 rows of keys on the Droid, as opposed to the G1’s five. This means pressing numbers requires using an ‘ALT’ key. Not hugely irritating, but mildly so. Far worse, however, is the fact that the keyboard keys have no gaps between them as they do on the G1, and the buttons themselves are flat rather than being slightly rounded. Typos are far, far more common with the Droid phone, it’s nearly impossible for me to get through a simple text message without fat-fingering something. To make matters worse, because the backspace key is DIRECTLY above the enter key, I find myself accidentally hitting enter when I’m trying to backspace, which means I’m sending a LOT of text messages that end in a typo, only to be followed by another text-message correcting the mistake.

Fuck you, Motorola.

Another irritation is that the Droid does not use a standard USB port. It sure looks a hell of a lot like one, but it isn’t. I could plug the G1 into anything that had a Mini-USB connector on the end. Wall chargers, car chargers, even the cables my wife uses for HER phone (very convenient if I happened to need a quick charge in her car). The Droid uses some proprietary connector, so you have to buy adapters for the thing if you want to charge it. This is completely obnoxious and unnecessary – it’s clearly Motorola trying to suck a little bit more money out of me. The real kick in the balls, however, is what the phone comes with in terms of chargers. It comes with a basic USB connector to connect it to your computer, and a wall adapter FOR THE USB CABLE. If you want to plug it into the wall, you must sacrifice the USB cable. This is particularly obnoxious for me because I had my G1’s usb cable (a standard USB with mini adapter on the end) in my office downstairs, a wall charger in the bedroom so it could charge at night, a wall charger at work so I could charge at work, and a car charger in my car if I needed to make a call but had let my battery die. The Droid’s supplied chargers give me ONE of these at any given moment, and I must separately purchase what I need for the other three.

UPDATE: A commenter has explained that the plug the Droid uses IS, in fact, an open standard, called Micro-B. A quick search on Newegg revealed many cheap Micro-B cables that I could use with the phone. So that’s much less annoying. Still would have bee nice to have a separate wall charger that didn’t use up the one USB cable that comes with the phone, though.

The camera for the Droid is 5 Megapixels. Seems impressive, except that so far I’ve had some pretty crappy luck with it. In fact, the picture of the charger took me five tries to get right, and still came out blurry, even when I switched the focus mode for closeup shots. The flash is basically worthless as well, it never seems to flash at the correct time, and it is effectively a “make this photo blurry” toggle.

Call quality on the Droid seems significantly worse as well. I have not yet determined if the speaker is the problem, but so far it’s much more difficult to hear people during a call on the Droid.

Another annoyance is how close the row of four hardware buttons are to the bottom of the screen on the Droid. I was using Twidroid and reading some private messages on my twitter account, then I hit menu. The only menu item on this screen is for “Delete all messages”. I decided I didn’t want to do that, and hit “back” to exit out of that menu. Unfortunately, the “back” button is flush with the bottom of the screen, so my attempt to go “back” registered on the touchscreen as a click on the menu item, so Twitdroid proceeded to permanently delete all of my private messages. I had to actually rip the battery out of the phone in a panic to interrupt it quickly enough, and I still lost my most recent thirty or so messages. Annoying.

Overall

Many of the nicest features of the Droid are actually features of the Android 2.0 operating system, the next version of the OS running on the G1. It remains unknown at this point if the G1 will get this version of the OS, but it’s what contains most of Droid’s coolest features, such as the turn-by-turn directions and other various improvements to Google Maps. The 2.0 upgrade also comes with a cost: a few of my favorite apps such as “reddit is fun”, “K-9 Mail”, and “Pure Calendar” don’t actually work correctly, but that will likely be fixed in due time.

I also do not yet have the Multimedia Station charger/dock for the Droid, which I think would be awesome on my bedroom nightstand, nor do I have the car mount, both of which I want and I think will make me appreciate the Droid more.

As it stands though, I’m not sure the Droid is compelling enough on its own to warrant the switch from T-Mobile to Verizon. I love the increased coverage, the devoted headphone jack, the screen improvements, and the overall feel of the phone, but I really dislike being nickel-and-dimed on chargers and I kind of really hate the keyboard (I’m hoping I get used to it).

But a few days ago, I saw the payoff: Google Reader Recommendations. Recommendation engines are one of my favorite things about the new social web – Amazon’s book recommendations and Netflix’s movie recommendations have led me to find things I never would have discovered on my own. So when I saw that Google Reader had recommendations of other blogs or RSS feeds for me, I was excited.

I figured, like most other recommendation engines, Google Reader now looked at what feeds I subscribe to and found other feeds that were subscribed to by people who subscribed to the feeds I like. This is generally how recommendation engines work: if I love “Back to the Future” and 90% of Netflix users who rate that movie highly also highly rate “Ghostbusters” highly and I’ve never seen it, Netflix will recommend Ghostbusters to me.

But when I looked at my recommendations, a strange thing caught my attention.

There is a recommendation for a blog related to TeX in there. If I scroll down, there are actually five or six other TeX resources.

This stood out because I don’t subscribe to ANY RSS feeds that related to TeX in any way, not even tangentially. So why did Google so strongly suggest them?

Well, I don’t subscribe to TeX feeds, but I am working on my Master’s thesis, and I’ve had to do a lot of Googling for help on LaTeX. A very large portion of searches I do in Google are LaTeX related, simply because I often need to troubleshoot a problem or look up how to format something.

So Google Reader’s recommendations are not merely based on feeds. They are based in my search results as well, even though that’s “separate” from Reader. It wouldn’t surprise me if people were getting recommendations based on the content of their Gmail accounts either.

Creepy.

The photos from my wedding are up on Flickr. Check ‘em out here. Our photographer was excellent. I highly recommend her.

The wedding was great. Virtually nothing went wrong – we even just barely avoided some rain but avoided it nonetheless. I was kind of dreading having to be social for a whole evening, but even the reception turned out to be a good time (though I had to go hide in order to eat my slice of cake).

Overall, being married doesn’t feel too much different from not being married. I’ve known I’d marry Julia since second week we dated, and I knew she’d make a beautiful bride (she did).

It was great seeing family and merging my 7-person family with her 100-something-person family. I also really loved how many of our friends from college were able to make it, including one of my professors. It was great seeing everyone again.

My friend Dennis was a fantastic Best Man, and Julia’s sister Olivia did an wonderful job as Maid of Honor. It was a truly unforgettable time.