Alvin Cho - Success metrics when launching your first developer portal

This talk was presented at API The Docs Virtual 2023 Feedback, Metrics and Analytics event series on 8 February. We are glad to present the video recording, slide deck, talk summary, and the panel discussion below. Enjoy!

Visit the talk listing page for an overview of all presentations!

Alvin Cho

Vice President of Product and Engineering (Digital Innovation & Transformation) at Fiserv

The Fiserv Developer Studio journey

Recognizing the needs of customers and product teams:

• Customers need a unified and consistent developer experience, easy discovery, self-service integration.
• Product teams need automation and governance, and self-service for content and workflow management.

Defining the success criteria

Define the Northstar as a guide from which to draft OKRs later.

“At least the Northstar for us was, you know, radically change the developer experience to provide untold levels of enjoyable productivity."

Prioritize the developer experience first -> the cornerstone of the developer relations ecosystem.

Define OKRs:

• Build the world’s best developer portal experience
• Seamless & quick onboarding experience for product teams
• Become the go-to place for building fintech applications

Solution

Make a developer portal which is like a luxury apartment complex:

• It provides governance and standardization for the product teams (the “tenants”).
• The portal provides the building structure.
• It’s a consistent experience for the developers.
• The product teams (the “tenants”) load and manage their own content (they “decorate their apartments”).
• Guidelines and validation provide consistency for loading and consuming documentation.

“Search is like the pool: it’s the same for everybody. You come here, you get it, and it's a consistent experience for the developer.”

Minimum viable experience was built in 9 months.

Success metrics

User research metrics - user testing & interviews aimed at:

• measuring the layout and interactions
• collecting microcopy feedback (“Workspaces” vs “Dashboard”)
• collecting qualitative feedback (what do users like or don’t like, “Concept A” vs “Concept B”)

Usage metrics - measuring the engagement:

• number of created accounts, businesses, and API keys
• number and type of search queries
• most popular pages or interactions (code snippets, run API, split screen)

“Our instincts provided us with the foundation of what to do, and then the data allowed us to change direction as we needed.”

• As part of the definition of done, think about the analytics that need to be tracked.
• The analytics provide the tangible data that proves you’re successful.
• Share the data with the product teams, so they know what is important.
• Queries that are popular can help product teams (e.g., to create a how-to guide).

Takeaways

1. Listen to your instincts to start
2. Set your Northstar & waypoints
3. Priority is to solve the core problem
4. Build, test, iterate
5. Show results quickly

At this event, the presentations were followed by a panel discussion, where the speakers shared further thoughts and insights.

Panel discussion

Participants: Zdenek Nemec, Alvin Cho, Laura Vass.