For a more recent interview with 2021 DevPortal Awards Jury, See What goes into an award winning devportal in 2021!
Interview with the 3 Jury Members who judged 32 developer portal nominations over 9 award categories for this year’s Devportal Awards. The 10th award was awarded to the most popular devportal as determined by community votes.
- Ellis Pratt, Director at Cherryleaf
- Emmelyn Wang, Global Business Development Leader for AWS Marketplace
- Bob Watson, PhD, Senior Technical Writer at Amazon Web Services
Ellis Pratt
Director at Cherryleaf (@ellispratt)
Ellis reviewed nominations for the best accessible devportal, and best community outreach categories.
Q: For the award category of best accessible devportal: What advice would you give future nominees to excel in these areas? For example, what do you think a developer portal should have by default? Is there a bar?
Thank you, all the entries are very good. In fact, I think for most developer websites, most people have got this side of things sorted pretty well.
From a baseline perspective, check your website in a web accessibility tool like wave or kick it out.
Other things I did when looking at the entrants was to check if the site was mobile friendly. If you resized the site, would it work? If you were only using a keyboard, could you use it that way? Was there a clear learning path?
To excel it has to be a little bit extra, the winner gave the impression that they started with accessibility from the beginning, from when they were designing the whole site, and that showed through.
And I think in the future, we’ll need to consider anxiety: the unnecessary stressing out of people with popups, or creating time limits, or limited situations - when those limits are false. So keeping it clear and simple for those with an extra range of disabilities over and above the ones which we normally think about like sight and hearing.
Q: For the award category of Best community outreach. What was the most impressive feature or service that you witnessed?
Well again there was a huge amount of effort that went into these and the way that I ranked the scores was on a score of 1 to 5 with 5 being the best. Lots and lots of the entrants were 4s or 5s. There was a great deal there. The guidance from Pronovix was to consider blogs, events, community pages, links to third party communities, and forums.
Again, in terms of the organization that’s won, they really considered community from the beginning. It was something that was very easy to get to. There were links to things like StackOverflow.
They had a large community and they spent a lot of time in the navigation categorizing information. Putting popular topics that people might ask towards the top to help sort out the wayfinding fees and it was a very clear and clean layout that they provided.
Rather than saying we need the community so let’s create a blog or let’s create a link to a forum, it was instead let’s think about the user journey or developer journey. From the start, how can we integrate community in that side of things.
Q: What do you think the future might hold?
We were talking about this when we had to get together as judges. One of the challenges we all faced was, there were some very large organizations that had a large install base for their community. They had lots of API’s and the challenge of managing complexity. Then you had startups which were starting with maybe one API, a clean sheet, and a smaller user base.
We were talking about how you judge the small with the big, perhaps for next year, maybe split it and recognize both groups and the different challenges that they face.
Emmelyn Wang
Global Business Development Leader for AWS Marketplace (@lifewingmate)
Emmelyn reviewed best API business model, best design, and best developer dashboard categories
Q: What category was most challenging for you and why?
So for me the most challenging category was developer dashboard and I’ll explain why. Today, there is the low and no code sort of platform approach where because of the shortage of very talented technical talent and developers, companies are asking for users to contribute to app development and developer dashboards.
To judge this category it is interesting to note the type of developer that you’re trying to reach. Some developers want to be able to customize and roll their own metrics dashboard. Other developer portals have a UI where you can quickly see metrics for the app or the API’s that you’re either building and/or consuming.
I used an objective framework for logging friction. Basically, you want to log in and see how much time it takes to get to your first API call. What was interesting is some of the API’s had either a spec, or perhaps a postman collection, or a swagger hub where you would be able to have the machine- ready piece of it. Whereas other ones offered SDKs or recommended that you use your own IDE, to be able to consume the API’s in that developer portal. I really wanted to weigh that very properly in the sense that you may have some developers that have certain preferences and it’s really having the empathy and anticipating how the developer would want to use that particular developer portal and its features. I didn’t want to take away points if they didn’t have a metrics UI versus on the other end where you have to build your own where it was very robust and easy to launch metrics the way that you would want to see it, the way that your organization wants to see it, as well as adding it and making it a part of your own technology stack.
Moderator: That sounds incredibly challenging. It comes right back to that maturity model of where your company is and to whom you are directing your audience to and how you are trying to provide this technology.
Q: Was there a feature or functionality that you felt all devportals should have by default?
I thought it was very interesting that some dev portals will section off their personas. They will say, ‘this is for the business user, this is for the architect and the developers’ and so on. I recommend a book, API Economy 101, that is vendor agnostic and available in many languages for our global community. It talks about the API economy. You want to have empathy for the stakeholders that you’re trying to reach. I’ll give you an example.
Developers, architects, and product leaders, have to be able to justify their time to senior leadership. And so what I recommend for the portals is that they offer each information for each persona that is a first class citizen. As judges we talk about what that means and anticipate what the user is looking for to help them make the justification so that they can spend time on the technology that’s offered in your portal.
Bob Watson, PhD
Senior Technical Writer at Amazon Web Services (@bobwatsonphd)
Bob reviewed best onboarding, best API reference documentation & support, and best new DX innovation categories
Q: What key aspects did you take into account when reviewing your categories?
I use the same (friction framework) approach but you know obviously with different areas of focus for each category. As an API consumer and documentation producer, I am most interested in ‘Does it help you get the job done?’ It’s a usability centered approach to ask ‘is this working for me as a consumer to do what I’m looking to do on your website?’
It all boils down to ‘does your site help me make a decision to use it?’ or ‘does your site help me make a decision to fix it? If I’m using it already. Does it meet me on my customer journey?
Q: You’re actually a returning juror because you were in the jury last year as well. Did you see a shift in the caliber of these developer portals that were nominated from last year to this year?
Oh, yeah. They just keep getting better and better. We are going to have to move the scale up to 11 for the next year’s because you know the one to five or even one to 10 is not going to cover it and that’s a good thing. It’s good to see and I hope that events like this keep bringing this into focus. What’s good today becomes normal tomorrow. You have to reach even higher. Between last year and this year I’ve seen that happening. I’m looking forward to next year just to see what people come up with.
Q: Would you say there is a standard or bar that needs to be met with regards to best onboarding?
There are definitely things that you need to make the onboarding experience easy for your potential consumer. The one thing that I saw a couple (of devportals) had missed is just the value proposition, it should be clear on your site. What’s it going to do for you?
It’s easy to forget that when you’re all caught up in all the details. Remember the person coming to your site who hasn’t seen it at all. If you can’t express what you can do for them, something that is instantly recognizable in five words or less. You have made it harder for them right out of the gate. On some sites I looked and I couldn’t find it. If you just put five words on your website, (my experience) would have changed.
It is important not to lose focus of the whole experience.
Final Thoughts
All jurors voted in the best overall category.
Emmelyn
I completely agree with everything that Ellis and Bob are saying. What’s really fascinating is not starting with the solution, so not starting with the API or the dev portal. But starting from, what is the key problem that we’re trying to solve.
I think that was really important and Bob and I had a lot of fun because he was talking about the dev portal consumer and saying, ‘Well, you know how much do we want to give to entice someone to decide whether to invest further time or resources’.
And then I was coming at it from the angle of I’m the product manager or, the leader who is responsible for the success of the devportal. My concerns also need to be met’.
If someone doesn’t register or sign up, can we nurture that relationship?
If they aren’t invested, say we’re a small company, we also have to be careful of the resources that we use to support people that are just looking around.
I saw several devportals actually address that right, balancing Ellis’s idea of anxiety. When you sign up, some portals were very good at knowing what questions to ask. They would say ‘let’s ask you just four simple questions to understand how to serve you when right’ and ‘do you want more interaction with us or do you want less so you can discover more’.
This was all very interesting because I think more and more of all the folks on this call and all the nominees, you’re getting better at serving who you’re trying to reach, as well as making it easier for the people who are creating the devportals to have success with them.
Ellis
There were some sites where they assumed that the reader knew what they wanted. Then there are others which provided a benefit or an answer to the question ‘why should I look at this bit or why should I use this API?’ And again in the meeting we had to talk about who would be the winner — Emmelyn talked about using friction as a way of judging a site. How much friction was there in the process of signing up or using it? Some sites had more friction than others and that was another way of judging.
Bob
You know when the site is a lead generator. They want you to come to the site and sign up to get hooked up into their sales funnel. I looked at it from a different perspective as more of a lead filter. If you make your site productive so that I as the potential consumer can explore it and your site explains to me what it can do for me then I can be engaged and start exploring and say ‘yeah this is gonna work’.
Then when I contact you. You know it’s a much higher quality lead. You filtered out the tire kickers and you’ve got someone who’s ready to engage. As opposed to having to go through the 10 or 15 to 1 of potentials, find the one who’s actually there.
This works for the sales team trying to find people, the actual customers, and for the consumers to find the solutions for products by having a high quality onboarding experience, and a good developer portal.
2020 Devportal Awards DevPortal Awards brings together the API community to recognize, celebrate and learn from the world’s greatest developer portals and their API documentation.
Pronovix and the organizers of API the Docs would like to thank our three Jurors for their time and energy dedicated to selecting the winners of the 2020 Devportal Awards. The interview was moderated by Diliny Corlosquet, Senior Business Product Manager at Pronovix and was recorded during the virtual 2020 Devportal Awards Gala Event, 25 November 2020.
