This talk was presented at the AI The Docs 2025 online conference. We are thrilled to share the recording and the summary with you.
Visit the talk summary page to see all of the presentations from the conference.
presentation
Jorge de Antonio (Principal Data Scientist at Amadeus) and Anna Tsolakou (Developer Advocate at Amadeus)
Summary
What are the primary technical challenges faced when using LLMs for generating or modifying large API specifications, particularly regarding context length and the nature of the content?
In their presentation, Jorge de Antonio (Principal Data Scientist) and Anna Tsolakou (Developer Advocate) from Amadeus shared their joint project on how LLMs can support building and maintaining API specifications.
The team’s initial chatbot prototype worked for small APIs but failed with larger ones, exposing the context length problem: LLMs like GPT-4 have maximum output token length of 4000 tokens (some of Amadeus's API specifications [Swagger files] were much larger, often exceeding 100,000 tokens). Standard approaches such as random chunking or RAG also broke down because API specs are deeply hierarchical, full of internal references, and more like code than text.
Their solution was to restructure the problem. By converting JSON specs into a tree, grouping nodes into token-bounded chunks, and applying chain summarization, they created compact summaries that preserved internal references. Instead of generating fully updated Swagger files, the LLM now produces step-by-step modification instructions.
This approach proved cheaper, faster, easier to validate, and more robust than direct file generation. Humans can quickly test the suggested steps, and small errors no longer cascade into unusable files.
Key takeaways:
- Starting GenAI projects is easy, but complex cases require data science expertise.
- Flexibility is essential: newer models with larger context windows may soon simplify today’s workarounds.
Sign up to our Developer Portal Newsletter so that you never miss out on the latest API The Docs recaps and our devportal, API documentation and Developer Experience research publications.
