Real-Time Syncing of API Documentation to ReadMe.io

01.08.2024

For an API to be successful, it needs to meet certain quality criteria, such as:

  • intuitiveness, developer friendly, focused towards easy adoption
  • stability and backward compatibility
  • security
  • very well documented with up to date documentation
 

Focusing on the last one raises a few questions:

  • How do we document?
  • How do we make the documentation public?
  • How can we keep the API and the documentation in sync in real-time with minimal maintenance effort?
 

How do we document?

The sky is the limit here, but on the other hand there is also a clear standard to describe APIs which is OpenAPI Specifications.

For our services, the spec is automatically generated from the C# XML doc comments.

This will output a JSON or YAML standardised specification that can be interpreted by any other tool higher up the stack that supports the same standard. The ubiquitous swagger UI is the most well known example.

That is great for local development and internal access, but once the API goes live we need something publicly accessible by any consumer or by our partners.

How do we make it public?

This is usually done by uploading the specs in some kind of developer portal that supports OpenAPI Specs and offers support features like partner login, public access, high availability, user friendly UI, playground to try the API etc.

While there are a few examples and solutions, we will focus on ReadMe.io which basically takes as input an OpenAPI spec and layers a nice UI on top of it.

Keeping it all in sync

The goal is, once a developer makes a change to the API, as soon as the code is deployed to production and ready to be consumed, the documentation magically updates at the same time without additional work. No copying and pasting of text, no manual editing, no updating of wiki pages etc.

The immediate go-to solution is to automate this process in the CI/CD pipeline.

What we need:

  • tooling to push the file in readme.io – this is provided by readme
  • an API spec file (json/yaml) – as input to the readme CLI
  • automation in the CD Azure pipeline – to call the CLI whenever we have a code update
 

Here are the Azure pipeline tasks that do the trick:

				
					- task: CmdLine@2
  displayName: 'Install readme.io CLI'
  inputs:
    script: 'npm install rdme@latest -g'

- task: CmdLine@2
  displayName: 'Update OpenApi spec in readme.io'
  inputs:
    script: 'rdme openapi https://$(PUBLICLY_ACCESSIBLE_HOST)/swagger/v1/swagger.json \
        --key=$(RDME_KEY) --id=$(RDME_SPEC_ID)'
				
			

 

There are a few important points here that can be solved in other ways depending on your setup.

The CLI is run from the context of an Azure pipeline, so the pipeline needs a way to reach the swagger.json file. Either you somehow:

  • put it in an DevOps artifact and use it as a local file or
  • just take it from a deployment location that is publicly accessible from the CI/CD pipeline
 

The last two arguments are the access key to readme.io and the API specification ID in readme.io, they can be easily obtained from the readme.io admin panel. If you have multiple APIs, you will have multiple different IDs. They are stored as locked variables in the Azure DevOps library.

Private infrastructure

You might run into the case where your API deploys inside some infrastructure where Azure DevOps pipelines do not have access. This was our case:

We built our custom API gateway using YARP (which isn’t very compatible with swagger).

You can work around such a problem by exposing the swagger endpoint in the API gateway.

If you have multiple services, you expose them on different endpoints in the API gateway. YARP requires unique URLs for different endpoints. 

For example:

  • [https://$](<https://$/>)(API_GATEWAY_CUSTOM_DOMAIN_HOSTNAME)/swagger/myAPI1/v1/swagger.json
  • [https://$](<https://$/>)(API_GATEWAY_CUSTOM_DOMAIN_HOSTNAME)/swagger/myAPI2/v1/swagger.json
 

Then implement a URL transformer in YARP to rewrite the URL to /swagger/v1/swagger.json just before forwarding the requests to the private services.

This is enough for most scenarios, but there is yet another special case that can occur on top of everything else.

Special scenario

In case you have multiple APIs, you might keep the input/output models of that API in a separate nuget package that is referenced by all APIs.

In this scenario, the generated OpenAPI Spec will miss all the XML documentation that is part of the external nuget.

All the underlined lines will be missing by default!

The problem is that there is a bug in the .NET project system that prevents from copying the existing XML doc file located in the nuget package in the output folder of the API.

 

A few seconds later….

How to fix it

  1. A bit of manual intervention is required inside the .csproj file:

				
					  <Target Name="CopyReferenceFiles" BeforeTargets="Build">
    <ItemGroup>
      <XmlReferenceFiles Condition="Exists('$(OutputPath)%(Filename).dll')" 
					      Include="%(Reference.RelativeDir)%(Reference.Filename).xml" />
    </ItemGroup>
    
    <Message Text="Found XML doc for copy: %(XmlReferenceFiles.FullPath)" 
					   Importance="Highest" />
					   
    <Copy SourceFiles="@(XmlReferenceFiles)" 
			    DestinationFolder="$(OutputPath)" 
			    Condition="Exists('%(RootDir)%(Directory)%(Filename)%(Extension)') AND
			     $([System.String]::Copy('%(Filename)').StartsWith('Connect.Api.Models'))" />
  </Target>
				
			

This will iterate over all dll files and will try to copy the respective XML files to the output directory, if the file exists and if it matches the name of our models’ nuget. Otherwise a lot more XML will be copied over and will needlessly increase the deployment size.

This will work beautifully locally, but it will not work in a docker environment where the image is built by Azure pipelines.

  1. To make it work from the pipelines, where you will likely use dotnet publish command, a new line is to be added after the initial copy operation.

				
					<Copy SourceFiles="..." DestinationFolder="$(PublishDir)" Condition="..." />
				
			

The only difference will be the destination folder, instead of the output path it will be the publish directory.

  1. The last touch is to add the NUGET_XMLDOC_MODE environment variable to instruct the restore operation to unpack the XML docs from the nuget packages together with the DLLs. This is set by default to skip to shave a few seconds off the pipeline run time.

				
					ENV NUGET_XMLDOC_MODE=none

RUN dotnet restore "myApiProject.csproj"

RUN dotnet build "myApiProject.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "myApiProject.csproj" -c Release -o /app/publish
				
			

 

Wrap up

To ensure the success of your API, don’t neglect the documentation side of it. Never rely on manual updates of wiki pages, Confluence pages or even using the UI editors that some tools and services put at your disposal.

Always strive to automate and correlate the API code deployment with updated docs deployment.

Depending on each particular case, the system and technology stack used and the tactics used, the hoops you might need to jump through to achieve this can vary – but the strategic goal should be the same.

Explore more

API

What is an API?

API hub What is an API? The term API is an acronym, and it stands for “Application Programming Interface.” An API is a vital building block in any digital transformation strategy, and one of the most valuable in achieving scale, reach and innovation. Behind every mobile app and online experience,

Read More »

API

API

A guide to monetising APIs

API hub A guide to monetising APIs In this guide learn about monetising energy APIs, and their commercial value as a new channel for monetising existing digital assets and data. APIs aren’t new. In fact, they’ve been around for quite a while, embraced fully by industries from all new digital

Read More »

API

APIs in energy

API hub APIs in energy Digitalisation in the energy sector. Unlike other industries where digitisation is the norm, the energy sector is a child by comparison.  In many countries, electricity is still purchased via a sales representative using a paper contract. Many energy retailers, (renewable) energy producers or grid operators

Read More »

API

API

re.alto Talks Part II

API hub re.alto Talks, Part I: Realising the energy transition in times of change This webinar is Part One of a three-part series on “Realising the energy transition in times of change”. https://youtu.be/YBdnui2y904 Explore more

Read More »

API

API

re.alto Talks Part III

API hub re.alto Talks, Part III: The benefits of an API marketplace in energy This webinar is Part Three of a three-part series on “Realising the energy transition in times of change”. https://youtu.be/C2IRj699eWg Explore more

Read More »

API

API

re.alto API overview

API hub re.alto API overview re.alto energy – Technical Setup for Existing APIs On the re.alto platform, individual Users can search for, and subscribe to a Provider’s API products. These subscriptions are monitored, tracked, and (if monetised) billed and settled individually by the re.alto platform.   Each subscription made by

Read More »

API

Frequently asked questions

API hub re.alto Marketplace FAQ What is the re.alto API marketplace? The re.alto marketplace is a marketplace for digital energy products and services, delivered via APIs. You can register as a provider or a consumer/user. As a provider, your digital products via APIs are uploaded to the re.alto marketplace, where

Read More »

API

Energy Quantified and re.alto case study

API hub Energy Quantified and re.alto The API-led approach to digital scale and industry growth As decentralisation of the energy market drives the rise of a host of smaller industry players, easy access to digital products at volume is now an essential factor for the rapid scalability desired by those

Read More »

API

APIs are everywhere – short animation

API hub APIs are everywhere – short video animation We are in living in an increasingly API-centric world. APIs are everywhere you look – and you might not even realise it. Need evidence? Gartner considers API management tools an essential component of the unrealized hybrid integration platform (HIP), currently an

Read More »

API

Adopting an API as a product mindset with APIs

API hub Adopting an ‘API as a product’ mindset with APIs APIs have enormous potential to open companies up to new revenue streams, unlock new markets and extract value from existing assets. To fully realise this potential however, APIs need to be lifted out of the sole domain of the

Read More »

API

Three things you may not know about APIs

API hub Three things you may not know about APIs API. Application Programming Interface. It is the communications channel between two web-based applications, allows the exchange of data without any connecting physical infrastructure. APIs lie at the very heart of digital transformation. According to the 2020 State of the API

Read More »

API

Alternative APIs for Dark sky

API hub Alternative APIs for Dark sky and the strategic value of weather forecasting data in energy In this article you’ll be introduced to weather data use cases and the importance of weather data within the renewable energy and digital landscape.  We also do a deeper dive into alternative APIs

Read More »

API

Dev

Dev

Scaling with Azure Container Apps and Apache Kafka

API hub Scaling with Azure Container Apps & Apache Kafka 11.06.2024 This article, written by re.alto’s development team, explains how to scale with Azure Container Apps and Apache Kafka. While such documentation already exists for use with Microsoft products, our development team did not find any similar documentation on how

Read More »