Milecia McGregor - Unit Testing Your Docs

Why Documentation Matters

• Helps external developers figure out how to work with an application that was developed by someone else
• Helps internal developers figure out why or how things have changed over time
• Gives the decision makers the inspiration to try your tool
• Helps marketing to show how to use the product in different projects

“When you can explain to others how your tool works, it makes you look more authoritative: people know you know what you are talking about because they know what you are talking about.”

A good detailed documentation:

• Walks users through how to use your tool
• Shows why it works the way it does
• Silently supports the company

Code Snippets in Docs

• Show how to use your tool and how that code works
• Show how to integrate with your code and how easy that is

What developers are looking for:

• Copy-paste ready code snippets
• Guide them through how this copy-paste code snippet works

“Working code snippets are a big driver of the adaption of a tool into the developer community.”

Example Doc Application

How to include unit tests for your code?

• Replace the inline code pieces of your MD (markdown) file with embedded code files (e.g., PY, JSON) using a plugin (e.g., Gatsby’s plugin)
• Store your documentation in MD and the code in a separate code file

You avoid:

• Code snippets that are normally in the MD file get skipped by the unit test because the MD file is seen as a simple text file
• Translating code snippets to MD
• Checking every code snippet and then updating the ones that don't work

It allows you:

• To write unit tests to check a particular code file every time our documentation is built
• To target the files that are typically associated with MD and your unit test
• Update only the files that fail the test

Writing Unit Tests for Snippets

Writing the code in a separate file and import it into the MD:

• Your code snippets don’t have to live in the same directory as your MD files
• You can call the unit test form the MD directory
• Clear division between document writers and the code that they use in that document

Make configurable code highlighted:

• To make users know exactly what to change to make the snippet easily work with no problem
• Reduce friction

Benefits:

• Able to narrow down the problem areas that need to be updated
• It’s the same as unit testing your application code
• You know the exact snippets that need to be updated

Including Doc Tests in CI/CD

• Include the test in your build process
• Run it in parallel so it doesn’t stop the main run
• Fix the bugs when you see them
• Use this automation to make docs easier to maintain

Key Takeaways

• Code snippets in docs represent your application code - if the snippets look bad, ppl assume the code looks bad
• No one can use your APIs without good documentation -give them everything they need
• It’s not too much extra work - a few more lines to the unit tests
• You’ll stop so much confusion if the docs are good from the beginning