Matching operational foundations to knowledge foundations

Knowing what to do and when is the key to systems and teams that perform well, and getting to that point requires a shared understanding of how all the pieces work together. As I mentioned in the last post having a logical basis for systems enables clear communication of the ideas and visions needed build good operations but also good systems. It's quite easy to fall into the trap of getting a bunch of engineers together and then building out a complex application or system of applications only to then quickly get overwhelmed with just keeping things going. A little forethought and time spent writing out what you are doing and how you expect it to work pays off massively by helping alleviate problems that occur from miscommunication and thousands of little learning curves for every twist and turn of a complex system.

This leads us to the documentation problems so common in high tech systems. What do I document? How do I convey my understanding to others? What do I need to write to ramp up new engineers and developers? What do I need to write to refresh my memory when I come back to fix or upgrade this five years from now? What do I need to ask other engineers to document to help me with my job? And so many other little questions that dictate how effective the documentation is, questions that unfortunately are often answered on the fly by engineers working on the documentation story with little thought to how it all fits together.

So I recently came across Diátaxis which is a technical documentation framework which can make answering these questions much easier and can greatly improve the consistency, voice, and therefore the effectiveness of supporting documentation. It has four simple categories of documentation to break up what to write and provide clear targets for how to write it.

• Tutorials: Probably the hardest to write of the four categories, these are focused on learning a given system through doing. These are also the essential answer to the question of "How do I ramp up new engineers/developers?"
• How-to Guides: The most common documentation type, these are synonymous with "Run Books" and "Play Books". Sort of essential step by step instructions to achieve various tasks.
• Explanation: The high level look at the system from different angles as to how it is supposed to work to broaden understanding. This sort of documentation is critical to helping maintain the course when drastic changes need to be undertaken to support new features or fundamental flaws are found that need to be corrected without breaking the overall solution.
• Reference: This is the detailed informational content, like a library or API reference, it is information dense without regard to what the reader may or may not already know about the system. It is intended to provide clarity of technical detail in a complete way.

That's pretty much it, every well documented system will have those 4 categories of documentation when using this framework. This simplifies things like hand offs from one engineer to the next, such as if you are losing one engineer you can ask them for a tutorial for their future replacement as well as make sure to capture any how-to guides based on recent events. Which is a much better kind of direction than the vague "Make sure to update your documentation before you switch teams." Or for a long running system being considered for refactoring or replacement you can ask for engineers to review and update the explanation documents before the architects start analyzing and solutioning.

For me finding Diátaxis comes at just the right time as my team is bringing existing systems into a DevOps workflow and developing new systems in that workflow. A simple documentation framework like this let's us look at what we have and rationalize what we need. And in the process of fixing and writing documentation we can identify gaps and problems with some consistency. For instance

I already write CONOPS which can serve as an "Explanation" document, probably not the one and only, but it fits the definition pretty well. Also, my team has run books which match with the "how-to guides". So we are already half way there to a complete set of documentation.

What I don't have is tutorials, which in retrospect seems pretty critical if I want to ramp up new engineers and developers on the systems we have. Especially as one of my personal goals on my team is to achieve a decent level of cross training, then tutorials would be the perfect documentation fit.

The somewhat interesting gap I have in documentation is the reference materials. Some of the systems we have are vendor solutions, so those should point to their provided reference material. But there is a significant amount of custom code, so when I look at that I start to consider auto-generation systems for internal and external API's. So I'll want to contribute to the organizations coding standards so our code can self document without being inconsistent with the rest of the organizations code base. I want reference documentation to exist inside the code as much as in a more easily accessible place like a wiki. For me I think it's important to have documentation suggestions come up in the IDE but also be searchable across the organization to help prevent drifting duplication of solutions and schemas. A little glue code in the CI/CD can push auto-generated docs to the wiki to achieve that goal. And then we can have some confidence that our docs are well rounded.

So with the last blog post and this one I think I've covered an initial plan of attack for building good foundations, the one thing missing (I should make this a series), is the architecture documents. That's next, I have my opinions on how architecture should work and a long history of watching what actually happens, so soon I'll post about rationalizing all this into one complete system.