How to Improve Documentation Practices in Engineering Teams

I saw an interesting question asking how to improve documentation practices in a Slack forum and thought I’d answer it here in longer form.

The OP wanted to know:

  • How are you handling documentation?
  • Where in the development pipeline should documentation take place?
  • Do you work on documentation during the project or at the end of the project?
  • Who is responsible for seeing documentation produced?

Let me pick these questions apart and see if I can’t offer some ideas and experiences from platform engineering with high performance software teams.

A Culture of Documentation

I’m usually hired by clients to refactor their workloads and/or improve the speed at which software is developed and released. One of the first things I ask for upon joining a team is their documentation. I love reading about the history of what exists, the teams that built infrastructure and services, and what plans describe the future state.

What I find is usually less than complete, out of date, and siloed across numerous platforms, security barriers, and private repositories. It’s pretty normal to have a culture of crappy documentation. It’s depressingly normal.

Reasons Behind Defunct Documentation

And there’s a few reasons for this!

Organizations are fluid and ever changing – it’s not possible to design the “perfect” architecture or documentation that works over a long curve of time. Documentation remains siloed and “best effort” towards the start of the life of a product because the product can ship without documentation (in most cases).

As the product evolves, documentation workstreams get out of alignment. User facing documentation is typically getting the most attention, while frontend, backend, infrastructure, platform, and other software documentation tend to lag behind or get zero attention.

Sometimes it feels like you versus the world!

Many organizations struggle with getting buy-in for documentation efforts from team members who may view it as time-consuming or unnecessary.

And … that’s just reality. 🤷‍♂️

Making Change

But, if you want to CHANGE your culture towards one of documentation, the good news is that yes, it’s totally possible to do it, and yes, it’s going to pay off when done right.

Great documentation helps great teams build great products.

Many teams with super smart people think the documentation doesn’t matter, and then they stagnate. They hit an invisible velocity wall that prevents them from LEAPING across sprints. More and more time shifts away from the development of code or infrastructure and towards meetings and alignment activities.

Thus, documentation should be handled in a way to address these issues:

  1. Content should be as publicly available as possible, with everyone being able to read and contribute to documentation (with some quality gates in place)
  2. Documentation work should be part of the release process
  3. Experience of finding simple, easy to search information across a single source of truth

There is a very finite window of time that exists between the moment you created something and the moment you flush it out of your memory to make way for new things. Inside of that window, it’s extremely easy to change what has been built, be it code, documentation, or tests. This is the sweet spot for iteration of documentation in small, easy bites.

Know that the investment into quality documentation is a worthy one that will have a massive impact on your organization. According to 2023 State of DevOps Report, investing in documentation accelerated improvements for trunk-based development teams by a factor of 12.8. I’ve seen it empower greater adoption AND enjoyment of both CI and CD for domain-driven teams. Good documentation is a game changer that must be experienced to be understood by the masses.

Documentation Pipeline

I like to find “sneaky” ways to make documentation just a standard part of the work required to change things.

Define Needs in User Stories

Documentation should become part of the Acceptance Criteria (AC). The team writing the story is responsible for scoping the documentation work that is needed. This team should know the “what” of the equation and make it clear to the engineer or developer what needs to be captured. Over time, these sorts of “do the documentation work for X” distills into part of the story template when selecting a feature or team.

Remember, if no one builds what you use, then what’s the point? Documentation fosters adoption of what you build, and adoption should always be a goal.

Build it. Then get it adopted. Repeat.

Service Index for Discovery

I personally like to let teams build out the documentation of their choice and then create a service index across them. Some teams like to use a linux tool to translate files or comments in their source code into markdown files. Other teams want to publish to something like Confluence or GitLab Pages. As long as the documentation can be easily maintained by EVERYONE on the team (and not just one person who takes it on as a hobby) AND can be found easily by the greater organization, I’m happy.

To some, this approach will sound chaotic. I suppose it works best with more mature teams that all understand the value proposition. You’re trading consistency across the documentation platform itself for the freedom to pick the right documentation flow for each domain / service / application / platform team. I highly value that freedom, copy the good stuff from my peers, and expect them to do the same from me. It removes dependencies that I don’t feel are necessary and allows me to precisely optimize the documentation flow that works best with my source code tooling and development flow.

The discovery of services and rich information spreads!

Other folks prefer to get everyone on the same documentation platform. And that works, too. I’ve seen Confluence rollouts where each domain team had their own Confluence space, along with a centralized Confluence space that acted as an index to all the different domain teams. With this model, you spend more energy on content outside of the development flow in exchange for consistency and/or potentially a better user experience for your non-technical stakeholders.

Either way, don’t tightly couple documentation if you can help it. There is no single platform that works in all use cases. There is no single tool that everyone will be happy with, and WOW do people have strong, “die on that hill” level feelings about their tools. ;)

Accounting for some level of documentation platform flexibility now will help you in the future.

Be Picky about Tools

I’ve mentioned a few tools above. Be picky about your tools. If it takes multiple tools working together to produce documentation, it’s unlikely to be the easiest or most efficient approach.

I find Python inspiring in how they write and maintain their documentation. It’s simple, lightweight, and easy to use. It has everything I want without feeling bloated.

Using markdown files that are published to a page tends to be the usual method for instrumenting change-driven documentation. There are a bajillion tools to do this, even some that grab code comments or use GenAI to summarize spans of code. The GitLab 16.9 release with GitLab Duo Chat, for example, calls out summarization of issues, artifact collection, and the ability to “generate code or content based on the information in these artifacts” resulting in documentation for the code.

My WordPress blog, for example, literally has a “AI” button next to the post summary box in case I could not be bothered to write the summary myself. It won’t be long before we’re just letting AI do most of the boring grunt work related to documentation without the need to build custom tooling for it. Nirvana or dystopian future? ;)

Have your pipeline do the heavy lifting for publishing docs using the same code and tools every run. If you’re feeling especially saucy, add a pre-commit hook to your repo to ensure the documentation is automatically regenerated upon each commit.

In short, documentation generation shouldn’t be hard or complex.

Align with Standard Documentation Contracts

It can be helpful to have a standard for what you document. Figure out what people need to know from one another, then agree to write down the answers to those things.

Common things I like to document:

  • Service configuration (consumption) details as a how-to or quickstart guide
  • Architecture deep dive on how the service works
  • Platform Compendium with deep dives on major/minor architecture patterns employed with supporting Architecture Decision Records (ADRs) to explain the why
  • Product Increment (PI) plans and related roadmaps
  • Bugs and their RCAs

Another fun 2023 State of DevOps Report stat: quality documentation had a “substantial increase” (the highest value) on team performance, organizational performance, and operational performance.

Responsibility

I’ll spare you the hand wavy “everyone owns it” nonsense. Because there must always be an owner of things to make decisions, otherwise no one owns it and no decisions will be made. Make sure a piece of documentation has an owner looking after it or the content is likely going to getting squirrely.

Quality Gates

If you’re using Quality Engineering practices in your sprint rhythm, making sure documentation is completed becomes a part of accepting the merge request. It’s ultimately on the service, domain, platform, or application team to feed the beast and then take care of maintaining documentation.

Quality documentation is valuable documentation!

Don’t let garbage documentation into your system. Make it part of your quality gate to accept a merge request. Hold the line the same way you would with a feature. You don’t need a Picasso painting, you just need documentation basics. You can always iterate on the basics later if that part of the documentation ends up getting more time and attention, or feedback loops come back saying it needs improvement. And, again, GenAI is going to make this a lazy person’s paradise.

Sprint Demos

Make documentation useful in Sprint Demos, add links that are shared with the stakeholders and allow them to go deeper than the demo. The more your Product Owners and stakeholders can see the value of documentation, especially if you’re keeping a log of new features and roadmap items aligned to your PI plan, the better off and more informed your community will be.

I often put my screenshots and other demo materials directly into a piece of documentation and make that part of my overall set of documentation. Memorializing events and decisions is a high value activity.

Key Metrics

Keep an eye on these metrics to help understand today, tomorrow, and growth made.

How well does your documentation provide coverage for your product / service / application?

  • Coverage measured against aligned standards
  • Questions that can’t be answered with documentation

What measures do you have in place to keep documentation quality high, and what are those tools telling you?

  • Readability scores
  • Spell checker tooling
  • Completeness

Are your stakeholders (Product Owners, Solution Owners, customers, etc.) using the documentation?

  • Analytics
  • Surveys
  • Feedback

Continuous Improvement

This is not the end. The work continues every day. Regularly evaluate and refine your documentation practices. Relentlessly seek friction, toil, and slack in the documentation flow. Prioritize what you find and continuously seek to squash it with updated tools and processes.

Regular reviews and feedback sessions can also help identify areas for improvement and ensure adherence to established alignment of documentation standards.

Summary

Don’t feel bad if your organization is disorganized and information is fragmented. Things can change, and it only requires small changes in a small team to light the 🔥 fire that ignites a movement.

Some final advice:

  • Focus on figuring out what everyone wants.
  • Align on that outcome and start inserting documentation work into your user stories.
  • Demo the results and be as public with your documentation as possible.
  • Link to your documentation when asked questions, don’t answer them by hand!
  • Automate away the toil.

Good luck

✌💙