Interview with the 11 Jury Members of the 2021 DevPortal Awards. The jury interview took place in two parts. The first part interviewed seven jurors. The second part with the remaining four jurors will be added to this article after the 2nd DevPortal Awards Gala on December 15th.
The 2021 DevPortal Awards jurors are independent of the DPA organization, they are experts in their fields, and conduct their activity as jurors in a vendor- and gateway neutral way.
The jurors tackled 49 developer portals nominations over 15 categories. To achieve this, they were split into three working groups with the responsibility of 5 categories each and returned to collectively select the final overall winners.
This year’s jurors have expertise that spans business development, UX, technical writing, design, development, marketing and API strategy. Our jurors are senior level managers, directors, speakers, writers, consultants, and strategists.
Table of contents for Transcript
- Jury Group One | Best International and Localized DevPortal [00:03:17] | Best Innovation [00:06:30] | Best Accessible DevPortal [00:09:24] | Best Community Spotlight & Outreach [00:13:09] | Best Design [00:17:36]
- Jury Group Two [00:21:36] | Best Onboarding developer portal [00:25:01] | Best API Reference Documentation & Support [00:27:56] | Best Developer Dashboard [00:31:37] | Best Editorial Experience in a Devportal [00:33:36] | Best Internal DevPortal [00:35:33]
- Learn more about our jurors!
This transcript has been edited for legibility from a computer generated script, our apologies for any mistakes or misinterpretations. Please refer to the attached video for reference content.
Jury Group One
Our first group of jurors were tasked with evaluating developer portals for design, developer experience, community pages, and innovations.
Diliny: We have Sophie Rutard here who has been working as an API strategist for Euler Hermes of the Allianz Group. Sophie, which nominees stood out for you?
Sophie: There were a number of very, very nice portals, but if I have to choose, it's actually Twitter and I can't really explain why, but it kept me somehow. There was something very natural too, to look at it. And to simply like.
Diliny: We also have Michael Meng, our resident academic juror. He's a Professor of Applied Linguistics at Merseburg University of Applied Science. Michael, which category gave you the most challenge?
Michael: That's really a difficult one, but I think, the one that was really a challenge, DX innovation of course developer experience is something that we all know is very important.
It's a concept with multiple connections to other important areas. But the innovation part costs. Important challenge to me. So what is an innovation? What is an area where we need improvement? What is the next big thing that can be tomorrow-standard? And yeah, this category kind of forced me, forced all of us to think about our understanding of innovation and about things that need to be addressed by (the) developer portal.
Diliny: We have Katalin Nagygyorgy who is a director of UX at platformOS. Which nominees stood out for you?
Katalin: So it's really a hard question, but I try to think it through, and I would like to mention the Plaid developer portal from a design point of view. They revamped the documentation site in a really nice and consistent way, and their micro animations are making the site more friendly.
However, they kept the focus on the functional.
Best International and Localized DevPortal
One of the awards that you were judging was the Best International and Localized DevPortal for which we had three nominations. What were some of the criteria that was used to determine the winner?
Michael: First and foremost, I think this is one of the categories that really deserve more attention.
And one of the issues that deserves more attention, because on the one hand, of course, API business is typically a global business. Right. And so you really have to think about a global audience. On the other hand, of course you, don't only, you not only have to serve the needs of a global audience, but you also have to think about the business that your audience is doing, which is also global, of course.
And, therefore one of the first things we considered was business-wise. the API offering the API portal would support users in kind of filtering and selecting, features and functionality in terms of whether that applies to a certain region. I mean, this is the most obvious for. Companies for example, who may offer features that Volvo works in one region of the world, but not in others.
Apart from that, of course, it's about serving your global audience, which first and foremost means. You have to offer content in different languages. You have to support your audience in selecting the local. So language and region they are working in and there's other things that are important and that perhaps we're getting more potent than the future, like, will your devportal be able to automatically detect the region that your user is currently in and adjust your offering kind of automatically.
And, there are other dimensions to localization and international offerings that have to do with the fact that. English is still the language that is used most often, of course. And you can expect that your audience has a certain level of proficiency, but the level can vary a lot. Right. And you really have to think about how you can address your audience, and how to serve the needs, at those very different levels.
They, they have, So that means that you have to think about easy language, plain language and, support, of language through, for example, appropriate visualizations. Of course, this kind of relates to other dimensions and other categories like accessibility, but, still, that was kind of a very important criteria for this category localization and international developer portal.
For the Best innovation Award in 2021 we had 7 nominees, what did you define as the standard for innovation in 2021? How did applicants consider themselves innovative?
Sophie: And I think Michael mentioned it. Michael had mentioned it earlier on,i t was difficult to select a candidate in this category because innovation is moving on and what is innovative today will possibly be standards tomorrow. So we had a lot of things which had very good filtering options to make it really easy to find the API APIs that you need for your use case.
And yet we didn't feel that. As really being innovations because we saw them widely spread. And so we then finally said to ourselves, the winner in innovation should be the one really surprising. And we had these things, on the onboarding journey, for example, where the onboarding journey automatically recognized our company and address details, with the background check, obviously very nicely done.
then we had E-learnings but, but really greatly done. E-learnings integrated in that developer portal. And these are really things that we had not seen in others and, which were really standing out for us then.
Diliny: I can see the real joy when you're describing these experiences. So it must've been very fun as a jury to be discovering these.
Michael: Yeah, I think that was one of those challenges or challenges, like interesting. The information needs, the different goals that your audience brings to the problem. I mean, this is one issue, but. New and innovative ways of solving that problem, which I think really is a problem. Right. That was an area where we said, well, we saw very cool solutions to that, and that kind of helped us finally, also with the decision.
Best Accessible DevPortal
Our Best Accessible DevPortal Award had 12 nominees. What set the finalists apart from the rest of the nominees? What advice would you give next year’s nominees?
Katalin: Yes. So as you mentioned, a total of 12 grade developer portals competed with each other in this category.
So the following three guidelines helped us to define the rank. First we checked how the applicants define their site related to accessibility. Why did they applied in the first place? It was important to see if they are in line with the category definition. In this case, it was of course, the definition of web accessibility, quickly I share these definitions
so, yeah, everybody knows about this. Web accessibility means that websites, tools and technologies are designed and developed in a way that people with disabilities can use that. This approach also benefits people without disabilities. For example, people using devices with different screen sizes or different input modes.
also temporary disabilities, goes into this category such as broken arm or loss grasses, people with situational limitations, such as bright sunlight or in an environment where they cannot listen to an audio. For example, people using a slow internet connection also fall into this category.
So we can tell that finalists understood the accessibility and fit into the category descriptions. second accessibility, checkers showed informative and comparable numbers about the accessibility scores of the developer porters. We used Google lighthouse rave and eCheck, or, for this purpose, all of the troll bytes, explanations next to the scores and highlight the accessibility errors and problems of the sites.
All finalists actually had outstanding scores in this site can tell. And, as a third step, we compare the XL extra accessibility to related features of the outstanding developer porters, like decorating structure, consistency, terminology, accessible language, gender, communications and so on.
So, yeah. There were a lot of items, but we were able to make or group smaller and smaller based on this and highlight. There'll be. related to the advice, what we can tell for next year's nominees. First, I believe, check the criteria of accessibility and explain in the devportal, of course, a questionnaire how your developer portal helps your users to access the content in every situation.
Second, that's your developer portal with accessibility checker. If you find any problems, you can still fix them. So that's the good part. And also it will be good for your users and also it will be good for your. So I believe that's it.
Diliny: That was a long list of testing and, you know, lots of things to verify there.
So sounds like you guys were incredibly thorough and, not an easy task, but, again, from the smiles on your faces, I'm glad that,
Best Community Spotlight & Outreach
Our next award had five nominees for Best Community Spotlight & Outreach.What would you say is the main beneficial take home of what you felt was necessary for community spotlight and outreach for developer portals?
Sophie: We saw a lot of things and linked a bit to the innovation.
You'll see how this develops over time. So in the past, for your community management, you would probably have done, I dunno, an FAQ or a forum or a blog, but this is very far away from what we have seen. So we really see that the runners are investing heavily in being in touch with their communities.
We're seeing, even a possibility for contributors to the portal to improve the content, to be then rewarded with sort of credit points that you can get. So you could get a rebate on the product, for example, to, to really, incentivize your contributions. We have also seen portals, which
really, did very segmented communities, depending on the most common use cases or the type of users of the API that one could have. So, for example, it could be, private persons or companies or non-profit organizations, and then the partner would accordingly propose dedicated communities.
Twitter even allows you to create your own community, which is fancy. So you, you would have localized communities, in, let's say Berlin or Paris, and you can then really, link the developers with each other. That's really beautiful to see, and it's very inspiring too. To learn from that and try to put this in our own pulseless.
Also, it's really very, very sophisticated.
Diliny: Does anyone want to add to Sophie's answer?
Michael: Only that I think this category for me was most fun to judge and to talk about, to discuss it was really, really amazing to see what ideas, you know, people are out there generate in order to.
Interact with their audience in order to get people attracted, in order to keep people in the portal and also to have the audience participate in kind of shaping the roadmap. That was also very impressive to see right. How to compete back, how to involve them, in your thinking about, what to do next and which direction your product should.
Diliny: just one additional question because from your answers, did you see a difference between when there was a small community versus a large community? Cause when you mentioned Twitter, they probably have like huge, massive community. whereas other portals probably have a lot smaller developer community.
Michael: So you could see like a difference in terms of how this was. yes. I think there are definitely differences also in the budgets that one can have to actually create all these features because it's. Yeah, I hear they don't come alone. but you don't have to be Twitter to have great community works.
We have seen this on many other portals, very nicely done, and really also in an innovative way, which is really inviting people to contribute and continuously improve the site for the benefit of.
Diliny: We will be announcing the winner for this particular category in the second gala, actually.
The award for Best Design had 21 nominations. How did you create your short list from 21 nominations? Do larger companies have an advantage over smaller companies or do you see both holding their own?
Michael: Well, first foremost, there was also a very, very challenging category, similar to the next innovation because elective of the experience design is a very broad concept.
And so. First, we had to kind of clarify our understanding of that concept. And, quickly found out that we agree on this kind of two types of meaning of narrow one and a broader one. And everyone related to the visual appearance of the site, consistently. The look and feel, is it modern? Is it intuitive?
Is it appealing? Right. Things like that. But then at the bottom it also relates to functional aspects. So does it serve the purpose appropriately? Does it make it easier for you to get the information you need? Does it support usability, things like that? What was 20 more than 20 nominees to can kind of just, Go on and, and kind of do a deep dig into every portal, although you would like to do that.
Right. So we did a two stage approach here. We first kind of checked every developer portal, nominated for that category and focused on our first impression. And we kind of used questions that customers would perhaps also typically use, like, can I find out what. API is offering, what are the main benefits I can draw from it, how to get started, things like that.
And then kind of use that to do some interaction with a portal and check your intuition about the look and feel, the visual appearance and so on. And this letter. Yeah. Some would say, well, really cool. And all those, we would say, well, I'm a bat, but perhaps in that category, not among the front runners and we then arrived at a smaller.
I dunno, five or six candidates, will we all agreed on that? They are pretty good with this kind of first pass check and then did a more thorough check and went deeper and checked, using other questions. And finally discussed for quite a while to, determine the one that was really a close race and, showing the high showing of course also the different dimensions that you have to take into account your, but, yeah, like all the other categories, it was a big, big learning effort learning, gain also here in that one.
regarding the second part of the question. Similar to what Sophie already mentioned. I don't think there's kind of a correlation between company size and quality of the developer portal, or even say design. Right? In fact, it could be that big companies say, well, people out there have to use my API anyways, so why care?
Right. So offer them something. And for small companies, it could be like the main assets. So they will really think hard about. How to make it a success. but of course we only have good examples. It's actually very often we had only a very rough understanding, whether it's a small company, a big company, I mean, Twitter, you have some sort of, knowledge that you can bring into that, but many other, nominees where, where you had to do some background check first and see, Independent within belong to a lot of companies, things like that.
But as I said, there's no, yeah, no correlation that I would emphasis here.
Diliny: Thank you very much for joining me today for this interview.
Jury Group Two
Our second group of jurors today had the challenge of tackling nominations, not only for developer experience, but for the under the hood operations and experience of technical writers, managing content within developers.
I'd like to introduce Lukas Rosenstock, is a software developer, technical writer, API consultant, and entrepreneur. Lukas, Which category gave you the most challenge and Why?
Lukas: So, as you already mentioned, we had two categories, which we'll talk about later, about internal portals and processes and those are a bit challenging to judge because, since it's internal, we couldn't directly observe it, but we had to rely on the information, and the self description of the material that they provided us.
But which I still think is a good proxy of how, how well, their internal process in how well they could present it to us.
Diliny: Bob Watson is a Senior Technical Writer at Amazon Web Services. Bob, Which category gave you the most challenge and Why?
Bob: Well, for me, it was the reference docs. partly because they're so detailed and generally pretty extensive.
And in this case this year had so many entries. So we have the challenge of going deep and going wide and, in, in all the variations of, and quantity of reference docs to evaluate, but it was a great experience. I mean, Those, those are the kinds of problems to have too many. Good, good entries.
Diliny: I'd like to introduce Ellis Pratt, the Director at Cherryleaf, a technical and UX writing services company based in the UK. A different question for you, which nominee stood out?
Ellis: what stood out. I had the privilege along with Bob and some others who had jury members this year to be a judge last year.
And there was a difference this year and there were some, some new features with some new, with some sites, which we hadn't seen last year. And that was that, Taking into consideration onboarding and the developer journey. And in some ways we're treating the API or the portal, like it was not vacation.
And there was much clearer process of what to do, where to start that I didn't notice last year. And, it made a number of some, a few sites stand out as being really good for having those clear developer journey.
Diliny: Thank you.
Tom Johnson is a senior technical writer working for Google. He is behind I’dratherbewriting.com. Tom, Which nominee stood out for you?
Tom: Well, a lot of, a lot of the nominees were great. I really zeroed in on a lot of the getting started stuff with onboarding and there were some excellent tutorials that really onboarded in different ways. some provided more of a conceptual overview of the entire process.
[00:25:01] Others were breaking the, getting started into like a, more of a part one in part two. And I thought a lot of these approaches were innovative and working.
Best Onboarding developer portal
For the award of Best Onboarding developer portal, there were 19 developer portals nominated.
This is a huge number, unprecedented, in devportal award history. Tom, would you like to answer first, what advice would you give to the many nominees who had great onboarding experiences, but fell short?
Tom: I was putting together a few thoughts on this. I think first of all, use standard terminology to refer to onboarding doc types. I think the most common name is like getting started. And when people would link to the getting started directly from their doc homepage or product overview, it made it really easy to find and get started.
And if there was some kind of registration required, hopefully, make that painless, some registration processes as too much information required signing of agreements or, you know, just too much, too much, of a hurdle to get going. But the, the biggest takeaway that I realized in looking at these, getting started tutorials is that a good tutorial needs to strike a balance between giving you enough information to become familiar with the product, to actually learn something about the product and how you set it up and how you work with it while also being concise and.
Having this endless tutorial that just duplicates the documentation. Some getting started tutorials were too skimpy. They were just like, here's how you authorize a call and make a call. Here's your response done? And it's like, well, what am I doing? What's the product about what is this larger context and workflow kind of missed out on that.
And then on the flip side, some sites just had too, too long of an onboarding, like it was multiple courses and so on. Like that's not quite the angle. So, find a balance, give enough detail, but don't overwhelm the user. And then finally, this is your first impression with a user you want to, you want to build confidence and inspire confidence in the doc. [00:27:33] So if you're getting started tutorial. Isn't accurate or doesn't help the user and they're not successful. What's the rest of their experience going to be like in the docs? Probably not great. Right? So that first 15 minutes or hour or whatever that they're spending with their docs can make a huge difference in their attitudes and perceptions as they go through the rest of your documentation.
Best API Reference Documentation & Support
The Best API Reference Documentation & Support award received an even larger number of nominees at 26. What was the process this year for narrowing down the nominations for this category? Were there reasons a nominee would rank higher or lower?
Bob: Yeah, no, this was quite an impressive turnout for that category. And, like I said, having so many entries is a challenge, but it's the right kind of challenge. Cause it's great to see all the creativity and especially the improvement in some of the, repeat, entrance. but, how we, how we manage that was to, agree on, on a rubric of customer experience and.
categories on how we would evaluate the different topics. And they, you know, we centered on things like navigation, appearance, the utility, how easy it was to use and the completeness. And so with that, we re-looked at all of the entries submitted at a high level to sort them into, finalists and not finalists.
And then we looked at where I can look at the finalists in detail with, you know, more, a critical eye with each of those categories, you know? And then we came up with a winner, one of the things that we saw in the high scoring entries, which made it a bit of a challenge. find a clear winner, was in the usability and utility of different sites.
And the winners were consistently easy to use. They served their audience well, and they had an engaging design. It was really a challenge. but, it was good to see, you know, something like reference topics, getting that much attention by the companies that have submitted their entries.
And so, you know, I'm looking forward to next year's entrance cause they just keep getting better.
Diliny: Is there any advice you'd give for planning next year's 2022 awards?
Bob: I'd encourage them to look at the, you know, the winners and see some of the techniques they applied, you know, how did they, how, how was their design, how did they approach the navigation?
you know, it's all about making it easy to use and useful for the audience and some of the, you know, the winners and the, the, the, runners up, you know, really took a creative approach. So, you know, learn what you can and then, you know, just keep, you know, getting better each time. W I wanted to add onto a point that Bob mentioned about reference stocks, getting, getting more attention.
And I really think this is a noteworthy call-out because, in the API world, 20 years ago, most of the outputs were just Java docs. Doxygen standard documentation generators. And it's really, I mean, it would be hard to judge entries and so on. With them all being Java doc output or something. But, yeah, with this web API landscape, the approaches are so much more varied and innovative and lend themselves to more creativity.
it's really kind of an interesting space to observe, to see, oh, this person is. Doing this. And that's kind of an interesting technique, you know, how they're making it less seamless and maybe another person is grouping things in a different way and they've got their own design. So there's a lot more variety now in this reference landscape.
Best Developer Dashboard Award
There were two nominees for the Best Developer Dashboard Award. Was it an easy decision for Best developer dashboard this year? What in your opinion identified success?
Ellis: This is probably one where there were two candidates that stood out much more than the other. And so in some ways this was an easier one to find the winner and the runner up compared to some of the other categories. The challenge with a dashboard is do you create a blank canvas and. Get users to start from that point, or do you give them lots of choices?
And for us as judges, we were essentially the type of user that had no previous experience at that particular API. So we were coming in without necessarily knowing a great deal about what the product did or any buzzwords. And so the ones that stood out. Again, go back to what we were talking about with onboarding and they treated the dashboard more like an application.
There was a clear starting point. There was a path that we could say, okay, there are five steps that we need to do to do this, to get a result or to create a product. And they have that segmented away from the analytics and the configuration element. Some of the other sites, which weren't so successful, that the registration could be quite complicated.
You'd have just a blank screen and no guidance on what to do or too many choices of where to go next, or they assumed you knew certain passwords that we as judges, because we weren't in those, in that industry and weren't familiar with, for the ones that stood out there. Consider the beginner. They have that clear onboarding path again, and that's why they were the winner and the runner up.
Diliny: Moving on to the...
Best Editorial Experience in a devportal
There were seven nominees in this category. What challenges did the jury have when judging the Best Editorial Experience in a Devportal category? What was the ultimate ‘value’ built into the process that ranked higher for portals running in this category?
Lukas: The difficulty here was that we were not able to judge the process because we weren't in those teams that we can't see their editorial process on a day-to-day basis, but we had to rely on how they described it. And we would also have to judge. Okay. Is that how authentic is what they say?
Do they really do what they say they do? if they follow the process. And so something that we also considered a value that, that, that we used in, in ranking, these portals was, Like, what kind of technical assistance do they have? Do they have some kind of automation which enforces the process?
and well, if it's just a, is it just humans talking and I'm not trying to discount the importance of people having conversation and bringing everybody together and deciding, how to, create your docs and how to create your develop experience. If, if it's just okay, we all get together and, and, and talk about it then it's, it's difficult to see.
Okay. Are all stakeholders involved? So, the automation that could be, something like Docs as Code where you have some tooling and continuous integration to bring all the content together, to validate the content and to publish it. And also, another thing that we saw was to use a lot of structured data to really enforce inside a content management system, how to, yeah.
Yeah. How to structure the data in, in, in ways that the process guides, the content is.
Best Internal Devportal
Diliny: The Best Internal DevPortal was the final award that the four of you were looking at. They were seven nominations. In addition to this, I believe we had to send them a questionnaire because internal developer portals are proprietary. We can't look at them. Five of those nominees answered the required questionnaire. The Best Internal Devportal was probably our most challenging to rank new category this year and yet one of the most interesting. What helped make the final decision for selecting a winner and runner up?
Lukas: Yeah. I mean, again, this is a category where we had to judge without first hand experience. And I don't think that that can be avoided here. one thing that we found is that there's a difference between an internal and an external deaf portal. And I think that's, that's something, something we, we looked at because, If you just have one developer portal, where you publish your internal APIs and your external APIs, then you have something like raw role-based access control or X measuring.
We're just, okay. We show those APIs. And I don't think for an internal devportal that's enough because I think in an, in an organization, especially a larger organization that has a lot of internal APIs. you need to. To manage the, manage the access. You need maybe some approval processes. So you need specific toolings to support your internal developers.
And that's something we looked at, but it was sometimes difficult to determine. Okay. How does the internal API differ from the external APIs. And, the part that we found out is when they specifically had a good process to manage API access internally, and maybe that's something I would like to see next year.
Ask the companies, how does their internal devportal differ from the external and how do they make these distinctions? And I think that would be really helpful both in, in the developer portal strategy in general, as well as in the future jury, being able to judge this, this really good as an internal developer portal are great.
Diliny: Thank you so much. Thank you again Michael, Katalin, Sophie, Tom, Ellis, Lukas, and Bob for taking the time for both being on the jury and for meeting with me today. Also, a thank you in advance to the jurors I will interview at the next Gala: Emmelyn Wang, Katrien Van Gijsel, Flavio Geraldes, and Liz Couto.
Last year we interviewed the 2020 DevPortals Award Jury to find out What goes into an award winning developer portal.
Want to learn more about the 2021 DevPortal Awards Jury? Check out their current works or follow them on twitter!
- Docs By Design - Bob Watson
- Medium - Emmelyn Wang
- Katrien Van Gijsel
- Sophie Rutard
- Cloud Objects - Lukas Rosenstock
- Cherryleaf Podcast - Ellis Pratt
- ResearchGate - Michael Meng
- Idratherbewriting.com - Tom Johnson
- Flavio Geraldes
- Liz Couto
- Katalin Nagygyörgy
For more information about the current DevPortal Awards jurors, please see the Jury page on DevPortalAwards.org.
All Pronovix publications are the fruit of a team effort, enabled by the research and collective knowledge of the entire Pronovix team. Our ideas and experiences are greatly shaped by our clients and the communities we participate in.