In several cases, companies emphasize their need for a developer portal, but only mature organizations have clear ideas about what kind of ecosystem they want to build. It is uncommon among them to think or talk about complexity, about how this will change them as an organization.
This article is based on the presentation by Kristof Van Tomme, Pronovix Developer Portals co-founder, at APIDays Paris in December 2021. In it we will discuss complex adaptivity, what the Stacey matrix and the internal variety is and what does all that have to do with APIs?
When we produce APIs, we are creating boundary systems that enable complex adaptivity. Complexity arises when a set of agents have the right level of:
Like in the case of coral reefs, every component evolves together, and if something tears apart the system, it will lose its essential properties.
Complexity is not just about bad code, it is not something that you’re trying to eliminate. In fact, there is a form of complexity that you might want to deliberately try to achieve and that you might want to consciously try to cultivate and improve, because it’s actually useful.
How can we define complexity?
A considerable body of academic research exists about adaptive systems: how they behave and what are the parallels between different systems and different layers of emergence. These parallels can be used both for biological and for social technical systems that we interact with.
Following Scotty Page’s theory, there are four parameters—the above-mentioned interconnectedness, interdependence, adaptiveness, diversity—that are important to create the complex adaptive behavior of a system.
When you have a system of connected agents that are not fully interconnected, there are still some missing edges in the network. They are interdependent but are not fully dependent on each other, so there is still some semi-autonomous adaptive behavior that is not just unthinkingly following each other. There is adaption going on, but it is not instantaneous and complete when you have diversity. When you have all of these four together, an exciting behavior happens in the system that creates the unpredictable adaptiveness that is so characteristic of living systems and ecosystems all around us.
The Stacey matrix and internal variety
The Stacey Matrix was developed by Ralph Douglas Stacey. “It is designed to help understand the factors that contribute to complexity and choose the best management actions to address different degrees of complexity.” - from praxisframework.org.
The Stacey matrix divides problems into four domains:
- Simple domain (number 1),
- Complicated domain (number 2 and 3),
- Complex domain (number 4),
- Chaotic domain (number 5).
The simple system has a optimisation curve that looks like a mountain with a single peak, as long as you go up you will get to the top. It’s easy to maximize or optimize, because you just keep going in one direction.
In a complicated system it helps if you have a guide that can lead you to the right hill in the rugged landscape before you start climbing.
Chaotic systems can be dangerous, because nothing is predictable.
In a complex system the interaction or the actions of each of the players that are participating has an influence on the rest of the players and they need to adapt. As a result you get this infinite play of interaction that’s going on. Pretty much every natural environment is complex, at least when there’s life present there.
Adapting to a complex environment requires internal variety: according to Ross Ashby’s law of requisite variety, if you are in a system in an environment that is changing a lot or where there is a lot of different states of the environment, you also have to have a similar number of states on the inside. You cannot adapt to a really complex environment if you do not have a lot of variety on the inside.
Living systems—and this is the connection to APIs—use signal boundary systems to create pockets of resources, energy, and information in a far from equilibrium state that allows them to differentiate unique behaviour that makes them useful participants in the ecosystems they inhabit. They use semi-permeable membrane systems: boundary systems that allow them to interact and coordinate with the rest of the world. For example in your gut an adaptive membrane layer provides an interface between the microbiome and your immune system and rest of your body, that constantly evolves to help you to digest a wide variety of foods.
What does complexity have to do with APIs?
When you put a living being into a system that system itself also becomes complex. This is the case for all machines and devices that we work with: it is humans that are interacting with them, maintaining, developing them. Everything around us has this complex property.
With APIs we are creating interfaces between different subsystems to help to address the complexity. APIs behave as membranes: the API puts itself between your organization and the people who are using some of your services.
In a previous article we applied social practice theory and the importance of shared meaning to developer portals.
We are moving towards machines that are interconnections between complex subsystems: customers and teams, teams and the rest of the company.
Since all machines require humans for maintenance, and most machines are operated by humans, that means that our machine world for the most part behaves as a complex system. The complexity of human systems is especially salient for us, because dealing with the infinite game of complex interactions is what most of our lives are involved in. Similar signal boundary systems pop up also in social and organisational contexts, however for the most part of economic history, these interfaces were mediated through humans.
But APIs change this game: they allow developers to evolve new interfaces, and applications through which users can interact with organisations without other people intermediating. We assume that on a global scale, something even larger is happening, something that in complexity science is known as dynamic decoupling: when a layer of interactions emerges that shows new unprecedented behaviour.
We are shifting from human-mediated interconnection only, to machine-made interconnection also. People can play the interconnection role and this adaption to complexity. But as we are moving towards programmed algorithms as the interconnection between complex subsystems, between the customers and teams, between teams and the rest of the company, we need to understand complexity. If we don’t, we risk creating systems that are not made for people.
What we’re really doing with API programs is that we’re creating evolving complex adaptive systems. That is why it is important that we not mindlessly push this further, that we not unwittingly optimize for efficiency only but also think about adaptivity. APIs can balance both of these.
When you can find a way to harness deliberate complexity it can become a source of value. If you look at areas that are complex, you will see parties evolving that help to mediate and guide other people through.
Some of the best business models are built around lasting sources of complexity that are stabilized and reinforced by an organization that evolved to be an interface for it.
How can API programs benefit from complexity awareness?
Do you want to know more about how complexity science can help us to build better social and technical systems? We are planning a conference series dedicated to the topic.
All Pronovix publications are the fruit of a team effort, enabled by the research and collective knowledge of the entire Pronovix team. Our ideas and experiences are greatly shaped by our clients and the communities we participate in.
Kristof van Tomme
Kristof Van Tomme is an open source strategist and architect. He is the CEO and co-founder of Pronovix. He’s got a degree in bioengineering and is a regular speaker at conferences in the API, developer relations, and technical writing communities. He is the host of the Developer Success & the Business of APIs and the API Resilience podcasts.