From Top to Bottom - TDD for Web Applications

I am a "testing fanatic" but I don’t understand test-driven development. Or so I thought.

I have read a few books and numerous blog posts about it, and all of them have one major problem:

The examples are always too simple.

Those examples might be helpful if you write simple text editors or calculators for a living. I don’t do that and the odds are that you are in the same boat with me.

As you might already know, I write web applications with Spring Framework for a living. Let’s find out what kind of problems I had with TDD.

Let's Build a House

When I started working on a new task, I typically followed this workflow:

  1. Create the domain model objects and the required database tables.
  2. Implement the repository layer.
  3. Implement the service layer.
  4. Implement the web layer.

As you can see, I liked to build things from bottom to top, and I felt that TDD was not a good fit in for this approach. Because I was working my way from bottom to top, it was often impossible to link a test case with a business requirement. This made me feel that a test case had no "real life" value. Its value was purely technical (ensure that that the code is clean and that it works).

And yet, my number one priority as a software developer is find the correct problem and solve it. TDD could not help me to do that when I followed this workflow. Maybe that is one reason why I felt that I had no incentive to use it.

Every time when I tried TDD, I felt that figuring out the test cases before writing any code was mentally draining. Eventually I stopped trying and wrote the tests after my code was done. It was super easy. After all, when my code is done, I knew exactly what should be tested.

This might not be a good thing.

When I built my applications from bottom to top, I missed an essential piece information which often bite me in the ass later on.

What About the Requirements?

A few days ago I was walking back to home from gym and thinking about this tweet by Kent Beck:

tdd is a great excuse to think about the problem before you think about the solution

Suddenly it all made sense to me (working out seems to be good for your brain). I realized that TDD made feel confused because my workflow was flawed. Instead of thinking about the problem before the solution, I worked my way up from the solution to the problem.

This means trouble.

If we think about the different layers of a web application, we notice that the layer which is "closest" to the solved problem is the web layer. It seems almost obvious that this is the layer where we should start our work.

Let's think about what happens if we flip my old workflow around and start working from top to bottom. This means that

  1. Each layer specifies the requirements for the layer below it. This means that we don't write code which is not required. Of course, nothing is final. Requirements might change and we have to change our code. However, this approach eliminates changes caused by misunderstanding the requirements.
  2. We can concentrate on the requirements and transform them into test cases. When all test cases pass, we have a software which fulfils the given requirements. Also, since we understand the requirements of a specific component, it should be a lot easier to figure out the required test cases even if we haven’t written any code yet.

The more I think about this, the more sense TDD makes to me. I have already tried this “new” approach with TDD and it looks really promising. The most notable improvement is this:

Figuring out meaningful test cases is no longer hard. It is easy.

Why Should You Care About This Shit?

If you are not interested in challenging yourself, you should stop wasting your time and STOP READING NOW. I have already wasted enough of your precious time.

I have noticed that sometimes developers think that the source code of the application is the most important deliverable of a software project.

The problem is that source code has no value unless we can prove that it

  1. Solves the right the problem.
  2. Works as expected.

That is why writing automated tests is important.

It saddens me to see that many developers are constantly seeking ways to improve their coding skills but only a few of those developers are looking for new ways to write tests.

Wake up people! Tests are code too!

7 comments… add one
  • Ee Sep 15, 2013 @ 11:15

    Hey,

    Coincidentally I once again have a thought to share on this matter and it's through a practical example. I recently was given the possibility to refactor one of our internal web services that had a serious case of code rot going (in all honesty, it was caused by yours truly) so given the chance I wanted to do things correctly this time around. First two hours of the refactoring I actually spent on making sure that I can run my tests properly and this time instead of point-cutting unit tests I started to build system tests first in a top-to-bottom fashion. It took me a few hours more to get the first simple "REST API /foo should return bar" test to work but then I had pretty much constant speed in adding new stuff.

    Now, about two months later the refactor/rewrite is just about complete and the end results are almost magical:

    - the web service itself works for its relatively complex purpose
    - system tests spawned a prebuilt Java client library for using the service
    - ...and that library spawned a fully asynchronous HTTP client that can be used for all kinds of HTTP things in general
    - the domain model is visible in all aspects of the service and it makes sense

    So, I can definitely agree that top-to-bottom is definitely a great way to approach the problem and its solution since it forces you to walk through all the layers every single time you modify something. Good blog post, Petri.

    • Petri Sep 15, 2013 @ 17:11

      Thanks for sharing your story!

      I think that stories like this are extremely valuable because many developers are in the same situation than you. I have been in this situation too and I know that it can be extremely hard to figure out how to get started. That is why sharing our success stories or thoughts is important.

  • Antti S Sep 15, 2013 @ 12:42

    Yeah, I think the main "thing" here is to realize that TDD isn't a way of creating unit tests, but actually a way of designing and implementing the software altogether.

    • Petri Sep 15, 2013 @ 17:07

      Good point. I am not sure if I understand that yet but at least I am going to the right direction.

  • Anonymous Coward Sep 19, 2013 @ 14:28

    My experience is that when you start with TDD you get a nicely structured architecture (usually MVP) out of the blue, without any design effort. Which IMO is a huge benefit.

Leave a Reply