One of the misconception that is often associated with agile software development is that agile teams won’t write any technical documentation.
I assume that this misconception is so common because the agile manifesto states that we should value working software over comprehensive documentation. Also, since some of us have experience from writing long technical documents that are not read or updated after we finished them, it is kind of natural to think that all technical documentation is waste.
However, this is a lie that has serious consequences. If you have ever tried to maintain an application that has no technical documentation, you know what these consequences are.
Once again we have to find a tradeoff between two bad options. Let's get started.
Writing Good Documents
Before agile software development became popular, we spent considerable amount of time for writing long documents that were not read by anyone after we finished them. It was quite common that a software project had a lot of documentation, but most of them were useless because they were hopelessly outdated.
It is clear that these traditional practices create a lot of waste, and it really makes no sense to write documents that are abandoned after the project is finished. There must be a better way.
We can find a better way by answering to the question: what is a good document?
I think that a good document fulfills these requirements:
- It has a "customer" who needs its information. For example: a developer needs the deployment instructions when he is deploying the application to the production environment.
- It is as short as possible but not shorter. A good document provides the required information to the reader as fast as possible. It must not contain unnecessary information that might disturb the reader AND it must not miss any relevant information.
- It is up-to-date.
If we want to write technical documents that fulfill these requirements, we should follow these rules:
- We should not write a document that is written only because the process requires it. If the information found from the document is not needed by anyone, we should not write it.
- We should keep the documentation as light as possible. Because shorter documents are easier to update, it is more likely that these documents are really updated. Also, because shorter documents are faster to read, we won't be wasting the time of the persons who read them.
- We should put the documents to the place where they are needed. For example: the documents that are read (and written) by developers should be committed to the version control. This way every developer can access them and we can use code reviews for ensuring that these documents are updated when the code is changed.
- Every document that is committed to the version control system must be written by using a text based format. My favorite tool for the job is Asciidoctor but Markdown is a good choice as well.
Let's take a look at concrete examples that demonstrate what these rules really mean.
What Kind of Documents Do We Need?
If we want to figure out what kind of documents could be useful to us, we have to follow these steps:
- Figure out what we have to do.
- Find out what information we need so that we can do these things.
If we think about a typical software development project or an application that is currently in the maintenance phase, we need to:
- Install or deploy our application. We can write instructions that describe how we can install (or deploy) our application. If we have to install other applications before we can install (or deploy) our application, these instructions must describe how we can install the required applications.
- Configure our application. If our application has a complex configuration (and real apps often do), we need instructions that describe how we can configure our application. The simplest way to write such instructions is to add comments to the configuration files of our application, but sometimes we have to write additional "tutorials" that describe the most common scenarios.
- Make changes to the code written by other developers. Before we can make changes to a piece of code, we have to understand two things: 1) how it is supposed to work and 2) how it works at the moment. Technical documentation cannot help us to understand how the code is supposed to work, but it must help us understand how it is working at the moment. Surprisingly, we can write the necessary documentation without writing a single document. We can document our code by adding Javadocs to our classes and transforming our tests into executable specifications.
- Solve the problems that occur in the production environment. If we would live in a perfect world, we would ensure that we don’t have to solve the same problem twice. However, because we cannot always ensure this, it makes sense to ensure that we can identify the common problems and solve them as fast as fast possible. One way to do this is to create a FAQ that describes these problems and their solutions. Every FAQ entry must describe the problem and provide the information that is required to identify it. It should also describe the steps that are required to solve the problem. The person who solves a new problem must add a new FAQ entry to the FAQ.
- Help new developers to familiarize themselves with the codebase. If our codebase has good Javadocs and clean tests, we don’t necessarily need to write new documents. However, often our codebase is so large and complex that it is really hard to understand the big picture. That is why we often end up writing an architecture specification document that becomes outdated because no one bothers to update it. We can try to avoid this situation by keeping this document as thin as possible. If I have to write an architecture specification, I write a document that provides a brief description of the overall architecture, describes the modules and their responsibilities, describes how the cross-cutting concerns (authentication, authorization, error handling, validation, and transactions) are implemented, and describes the integrations.
It is a somewhat easy to think that I try argue that we should always write these documents. However, this would be a mistake.
Do We Really Need All These Documents?
It depends. Every software project is different and it is impossible to say what kind of information is needed.
That is why I think that putting our faith into best practices or processes, which specify what document we must write, does not help us to be more agile. It only ensures that most of the documents we write are waste.
We must stop looking for a silver bullet. We must stop following best practices. In fact, we must stop thinking about documents.
If want to eliminate the waste caused by writing obsolete documents, we should think about the information that we need and figure out a way to distribute this information to our existing and future team members.
That is agile.
I would argue that writing Javadocs tends to lead to outdated documentation. I'm not saying that you shouldn't write Javadocs but I think that it should be kept at a reasonable level. It is very easy to end up with outdated method parameters etc.
The best way to document code itself in my opinion is to write self documenting code. Executable specifications are one part of it, proper refactoring of code blocks into small enough self-describing methods and thought-out interfaces and classes that describe their intention are the other part. When the code and tests document itself, they are always up to date.
I advocate for writing JavaDocs for interfaces at least. The implementation should be self-describing, but the interface might need a little bit of context too.
I agree with Petri on the mandatory documents. We always have a Confluence page for:
Sometimes we write a document when we want to design a new functionality and there should be a debate regarding the best approach. A short document is a good starting point and it's even better when the final design decision is being documented (It's much easier to remember why we've taken a certain decision).
I have a love-hate relationship with Javadocs. I love them because I can read them while I am coding (no context switch required). On the other hand, I also know that they tend to become outdated when the code is updated, and outdated Javadocs are harmful because they contain incorrect information.
However, I think that Javadocs are outdated mainly because developers aren't disciplined enough to keep them up-to-date. Code reviews can help you to solve this problem. For example, you can review every commit before it ends up to your master branch, and the reviewer must ensure that Javadocs are correct and up-to-date.
Nevertheless, I think that the team has the final say. If they think that Javadocs aren't useful to them, they shouldn't write them. We should remember that they can always start writing Javadoc documentation if they notice that they need it after all.
I agree on the points of both of you. Javadocs give great assistance in IDE but when they are outdated they just give you a headache. I think that the problems related to software development too often come down to lack of discipline. This also applies to writing any other kind of documentation. Also, software developers tend to give in to management on taking shortcuts while they really should be standing up for themselves. Too often tasks are "finished" the minute first working version has been done while this would be the time to clean up the mess that we actually ended up with.
I agree. It is very sad that some developers choose to easiest path even though it will hurt them in the future.
Yes, and at the same time developers are blaming managers for this. Like you said, developers should stand up for themselves and take responsibility for their work. Too many developers are blaming managers when they should take a look in the mirror instead.
It depends though.
I guess generalization is the culprit here. But at a minimum there should be Javadocs & possibly a package-info.java class describing the API.
Forget documentation, agile process talks about a minimum viable feature in the first iteration & a continuous improvement of that feature in the subsequent iterations. But this fact is almost always ignored. The result is that the product is made up of a series of minimum viable features, with a few bugs fixed here and there. The project manager never has time to revisit a feature, it's like they have never even accounted for refactor & improvement in the original estimate.
Writing documentations is in many cases usefull at the beginning or at the end of a project. But we should stop creating classic documents, these are to fast outdated. Documentations should be available via the web so they can be always up-to-date. For that i use Markdown too in combination with DBook.org, which is made for that and can handle versions too.
I think that the most important thing is to find the level of documentation that solves your problems. Often this means that you simply have to ignore blog posts that describe what kind of documents you should write (this one included).
By the way, thank you for mentioning Dbook.org. It looks pretty interesting => I will take a closer look at it.