Michael Meng - Optimizing Documentation – Some Guidelines & Effects

The work reported in this talk was a part of a project that explored ways to make API docs more useful and effective in collaboration with Stephanie Steinhardt, Andreas Schubert. The talk presents:

• Some guidelines developed to make API docs better fit information needs and expectations of developers
• The results of the empirical tests conducted to demonstrate that these guidelines indeed improve developer performance

The research

Focuses on two main decision that technical writers have to make in their work:

1. Which information should be presented for the user: what is the information-need
2. How should this information be presented in order to support the audience best

API docs differ greatly regarding the information and the way of how that information is represented:

• Different decisions can make a difference in cost, but does it make a difference from a user’s perspective?
• Would a conceptual description, a screenshot, and what they see on the screen make developers more successful?

Applied research: the kind of research we need to address these questions:

• Halfway through Basic research (purely theory-driven) and Usability testing (remains at the individual product level):
• Studies effect of design variables on user behaviour
• Develops evidence-based principles of information design (if details like screenshots does make us more successful)
• Principles that are grounded in theory can be derived and can guide the work of technical writers, but are also generalizable and validated by empirical evidence

The research is called for to find out whether differences like:

• minimalist description,
• text and code examples presented in a single column,
• content organized by topics,
• left side navigation
  matter and if evidence can be developed based on principles to drive the work of technical writers.

What makes learning an API hard?

Results of two studies

Obstacles are related to: (study 2011)

• Learning resources (e.g., documentation, code examples)
• Technical environment
• API structure
• Developer background (e.g., prior knowledge, professional training)
• Process (e.g., lack of time, interruptions).

Mostly selected answer: (study 2018)

• Wrong/incomplete documentation
• Incomprehensible documentation

Meeting the information needs of developers

Do guidelines lead to better documentation?

Research findings that motivated the guidelines based on the empirical study:

• Developers adapt different strategies from learning and coding, like opportunistic, systematic, pragmatic approaches
• Conceptual knowledge on the domain covered by the API facilities learning, but developers tend to ignore concept documents
• Developers want to solve a problem not to learn an API as a whole: documentation is accessed in a task-oriented manner
• Code examples are used for learning and as a starting point for new solutions
• Mapping business objects to API elements and identifying entry points into an API constitute main challenges

From these findings, various guidelines for optimizing docs have been derived:

Enable efficient access to relevant content:

• Organize content by tasks and use cases
• Present important concepts integrated with relevant use cases
• Provide transparent and consistent navigation
• Structure content consistently

Facilitate initial entry into the API (help starting with the API):

• Provide working code examples
• Provide relevant background knowledge
• Support mapping of business objects to API elements
• Provide a concise API overview

Support different learning strategies:

• Enable selective access to code
• Signal text-to-code relations
• Present important concepts redundantly
• Enable fast and productive use of the API

See non-optimized and optimized versions of the documentation of an existing API in the presentation.

Testing the guidelines showed:

• Higher accuracy on tasks for group working with optimized documentation (solved more tasks correctly)
• Small but nonsignificant tendency to solve tasks faster for group working with optimized docs
• Developers spent less time outside the documentation (like outside resources as Postman client) when working with optimized docs: actual task execution was faster
• Time spent in the documentation was slightly increased, but the difference did not reach statistical significance

For full research details check the ACM SIGDOC 20 conference paper. The content becomes available after registration.