Devportals & Digital Transformation

Developer portals & digital transformation: APIs in the complex adaptive enterprise

What is digital transformation?

Depending on the person you ask this question, they will all tell you a different answer. If you ask somebody in the IT industry, their answer will probably be centered around their own services.

For example, for some people, digital transformation is about the digitalization of documents, ERPs, CRMs, APIs, AI,… For others, it is about becoming an Agile organization, about DevOps, Open Source, InnerSourcing, Industry 4.0,…

All these buzzwords are tactical, they are patterns of behavior and technology sets that you can use to achieve digital transformation. For a couple of years now I’ve been researching systems thinking and complexity theory. Now I believe that I have found a macro-level understanding of what digital transformation really is in complexity theory.

I think digital transformation is the process through which organizations learn to accept and leverage their internal complexity: becoming a deliberate CAS (complex adaptive system), embracing complex adaptive behavior as the only way to deal with the increased complexity of our digital environment. Doing this through the usage of digital technology, this transforms the way businesses are structured from hierarchical command chains to lean swarms of cooperating adaptive agents.

In the past couple of months, I’ve started presenting a synthesis of this research (first at the Nordic APIs event in Stockholm) and I now feel that the ideas are sufficiently condensed to share in a written format.

My presentation on this topic at the API the Docs track at the APIdays in Paris (2019), I introduced 5 objectives that organizations can pursue to implement the organizational transformation based on my interpretation of complexity theory. Below is the video recording of this presentation (CC-BY-ND Pronovix) and its transcript as an article.

In the coming months, I will publish in-depth articles on the different aspects of this model, and what it means for our and your business.

The role of developer portals in digital transformation (transcript)

“Formal rules and procedures don’t have a predetermined effect on people’s behavior. Rather, people actively interpret rules and use them as a resource to fulfill their goals. What matters are not the rules, but the ways people use them.” Morieux, Yves. Six Simple Rules, Harvard Business Review Press

When you create rules, procedures in an organization, that doesn’t mean that people will actually follow those. People will use those rules and procedures as their environment wherein of which they act. Whatever rules you set up, don’t ever expect people to follow: you always have to look for the unintended consequences.

The goal of this article is to make you curious about:

1. Systems thinking
2. Complex Adaptive Systems
3. The role of APIs in Digital Transformation
4. The role of devportals in Digital Transformation

What is digital transformation?

Digital transformation has become a buzzword and so it is getting hollowed out. But I want to go back to the basics and look at this from a systems perspective. Why is it important? For a lot more than just being competitive. I think the core of digital transformation is that today most enterprise companies are like trees, adaptive as they grow, but becoming rigid as they mature.

To survive we need our companies to become more adaptive, more like ant hills. That is a hard thing to do.
Another way to see what we are trying to do is to consider an enterprise that is very hierarchical as an oil tanker —where all the decisions are taken at the top, where it can take a year to change course, iand a year to buy something— and to turn them into something that is much more adaptive. Something that has much more surface with the world where it can learn and adapt.

From tightly integrated value flows, companies need to become value networks, rerouting around bottlenecks towards constantly evolving value sources. The different components have to interact with each other in a non-designed way: it is more bottom-up and more evolutionary.

One goal, many transformations:

• Agile transformations
• DevOps
• Self-organizing teams
• InnerSourcing
• APIs

It can seem hard to grasp that these movements are all digital transformations. It is a lot of things happening at the same time.

What triggered this need for change?

The environment has fundamentally changed. Yes, digital technology changes everything and we have to adapt, but the change is more fundamental. It changed the architecture of how business works.

Digitaltechnology has increased our interconnection & interdependence, resulting in two major shifts: value space singularity and the increasing complexity of the environment.

The value space singularity

Proximity is being replaced by experience (Convenience, Familiarity, & Immediacy) as the new dominant value dimension. It is no longer that important how close you are to your customer, it became much more important how fast you can service them and what the customer experience is like.

Increasing the complexity of our environment

Interconnection and Interdependence are two out of the four variables that drive complex behavior. In his book ‘Understanding Complexity’, Scott E. Page writes that when a set of agents have the right (not extreme) levels on all four variables, you get complex behavior. That is a magical, emergent behavior that makes things so robust in ecosystems, in nature.

Complicated or complex

The way we used to build companies is like a watch, as machines where we would stake people, we would tell them exactly what to do, what exactly their job is. They would be part of a defined flow of value that is efficient as long as the environment didn’t change. Because machines and complicated systems —that can be understood by experts—, are real good and doing what they are made to do as long as you do not change their environment.
Complex systems on the other hand, like ecosystems, like humans, are much more resilient but they are impossible to understand because you cannot predict their emergent behavior just by looking at the individuals. There are surprises all the time. But they are also a lot more resilient: if you change the environment, complex systems can adjust. You can take away a tenth of the birds in a flock and the whole group will still show the same behavior. This has implications on everything: on global warming, on our society, on politics, etc. It impacts our businesses.

Rising complexity means that industrial age companies organized as complicated systems no longer work.

The Cynefin framework is a framework for thinking about problems because you cannot try to solve every problem the same way.
Simple problems are very easy to solve, there is one peak where you keep going up and you get to the top. Complicated problems are like a hilly landscape: you go to the peak and you notice, ‘Oh, there’s bigger peak there’ and you can go down and go up again there. It is hard find to your maximum but an expert knows where the peaks are and can get you there.
In complex problems you have a dancing landscape: every time somebody makes a decision the whole landscape changes, like when low-cost airlines entered the market everything changed and everybody had to adjust. To be able to keep up with all these changes you cannot be a complicated system any more: complex environments require complex agents.

Complex environments necessitate complex agents

Complex Adaptive Systems are a group ofinteracting agents, that adapted to complexity in an environment with complex adaptive emergent behavior. For example, a bee-hive shows behavior that you wouldn’t expect if you were looking at a single bee.

My hypothesis is that your developer portal can be a central resource that allows you to manage complexity:

“Through your developer community a devportal can help you tune your company for complex adaptive behavior, to adapt to the world outside.”

Patterns for complex adaptivity

How to adjust to our new environment? I think these five properties are what need to be tuned. It is something I am thinking about and researching now, and I would love to hear your feedback on it if you have concrete examples. For some we have heard examples from our customers, this is what we are now learning about and exploring.

1. Cooperate to remove friction
   Appropriate intermediate levels of the complexity parameters:
2. Interconnectedness
3. Interdependence
4. Diversity
5. Adaptivity

5 objectives to become a Complex Adaptive System (CAS)

Improve DX to improve customer experience

Remove developer friction through an improved developer experience. Why does this matter? We are living through the time of the value singularity, where proximity is gone and it is now all about experience and flow. One of the best ways to improve flow is to have APIs that allow developers to create automations and speed up interactions. You can remove a lot of friction from your customer interactions with your business. One of the best ways to make that possible is by creating APIs with a good developer experience, to be more efficient instead of having to explain every time to every single developer how to use your APIs. How do you do that practically?

Developer eXperience is achieved through dialogue, not through top down rules & governance. Too much enforcing does not necessarily result in a good experience. The API producers and the API consumers have to co-operate on the design of the APIs to make them better.

Developer eXperience can’t be reached without adjustment costs: to really collaborate fully, respectful conflict is essential. That is not coercion and not aggression.

Increase interconnection through APIs

Build as few devportals as possible. How many do you need? Do you need a separate portal for internal and for external use cases? It makes sense to create separate portals because you need a different experience. But not all your developer teams need to make their own devportal: you need interconnection. Strongly encourage reuse, but remain curious about exceptions. This is a debatable point. Complex systems need to grow. I think there is a way of providing a scaffold to facilitate the growth. However, I think we need to be careful with providing a scaffold and mandating the use of APIs - it may not necessarily result in better interconnection. This is a point I would like to hear your feedback about.

Use APIs to engage with more diversity

Most enterprise companies that use APIs already have a lot of diversity from a cultural perspective. As most enterprises are located on various parts of the country or the world, that already brings a natural cultural diversity. What you can do through an API program is to bring into contact different parts of your organization with each other.

An API partner platform business model will allow you to benefit from what is outside of your organization. Don’t try to standardize everything but embrace healthy diversity.

Create loosely coupled interdependence through APIs

This is one of the most important aspects. Ideally APIs are constrained in such a way that they don’t allow you to fail, like REST APIs: you can give the parameters but you cannot dictate everything and you do not have to define everything. But they are open enough to allow for surprising applications. An API that doesn’t surprise you, and if people do not create surprising things with it, then you probably created an application rather than an API. API consumers should be able to influence API design and capabilities, but not dictate them. This is an exciting but hard exercise.

Use APIs as a boundary behind which teams can adapt

APIs allow for adaptability, because they are constrained in such a way that they are not fully open and not fully coupled, what is inside the API boundary is able to evolve and to adjust. That is important for creating the adaptivity we did not have in the SOAP-era. What does this mean practically for devportals and API design?

1. Be careful with API versioning, but allow for change. Clearly communicate about API versions on your devportal.
2. Integrate documentation into the development process: keep docs in synch with changes to the API. Do docs-as-code, continuous documentation from your development into your production environment so that it is always up-to-date, so that your consumers can adapt to the changes that happen inside.
    

CAS through APIs and developer portals:

• Improve DX to improve customer experience.
• Increase interconnection through APIs.
• Use APIs to engage more diversity.
• Create loosely coupled interdependence through APIs.
• Use APIs as a boundary behind which teams can adapt.

Reference books

• Incomplete Nature: How Mind Emerged from Matter by Terrence W. Deacon
• Neither Ghost nor Machine: The Emergence and Nature of Selves by Jeremy Sherman
• A Simpler Way by Margaret J. Wheatley and Myron Kellner-Rogers
• 7 Rules for Positive, Productive Change: Micro Shifts, Macro Results by Esther Derby
• Leadership and the New Science: Discovering Order in a Chaotic World by Margaret J. Wheatley
• Six Simple Rules: How to Manage Complexity without Getting Complicated by Yves Morieux ad Peter Tollman
• Understanding complexity by Professor Scott E. Page

Cover image by Jonathan Wilkins CC BY-SA 4.0