01.08.2024
For an API to be successful, it needs to meet certain quality criteria, such as:
Focusing on the last one raises a few questions:
The sky is the limit here, but on the other hand there is also a clear standard to describe APIs which is OpenAPI Specifications.
For our services, the spec is automatically generated from the C# XML doc comments.
This will output a JSON or YAML standardised specification that can be interpreted by any other tool higher up the stack that supports the same standard. The ubiquitous swagger UI is the most well known example.
That is great for local development and internal access, but once the API goes live we need something publicly accessible by any consumer or by our partners.
This is usually done by uploading the specs in some kind of developer portal that supports OpenAPI Specs and offers support features like partner login, public access, high availability, user friendly UI, playground to try the API etc.
While there are a few examples and solutions, we will focus on ReadMe.io which basically takes as input an OpenAPI spec and layers a nice UI on top of it.
The goal is, once a developer makes a change to the API, as soon as the code is deployed to production and ready to be consumed, the documentation magically updates at the same time without additional work. No copying and pasting of text, no manual editing, no updating of wiki pages etc.
The immediate go-to solution is to automate this process in the CI/CD pipeline.
What we need:
Here are the Azure pipeline tasks that do the trick:
- task: CmdLine@2
displayName: 'Install readme.io CLI'
inputs:
script: 'npm install rdme@latest -g'
- task: CmdLine@2
displayName: 'Update OpenApi spec in readme.io'
inputs:
script: 'rdme openapi https://$(PUBLICLY_ACCESSIBLE_HOST)/swagger/v1/swagger.json \
--key=$(RDME_KEY) --id=$(RDME_SPEC_ID)'
There are a few important points here that can be solved in other ways depending on your setup.
The CLI is run from the context of an Azure pipeline, so the pipeline needs a way to reach the swagger.json file. Either you somehow:
The last two arguments are the access key to readme.io and the API specification ID in readme.io, they can be easily obtained from the readme.io admin panel. If you have multiple APIs, you will have multiple different IDs. They are stored as locked variables in the Azure DevOps library.
You might run into the case where your API deploys inside some infrastructure where Azure DevOps pipelines do not have access. This was our case:
We built our custom API gateway using YARP (which isn’t very compatible with swagger).
You can work around such a problem by exposing the swagger endpoint in the API gateway.
If you have multiple services, you expose them on different endpoints in the API gateway. YARP requires unique URLs for different endpoints.
For example:
https://$](<https://$/>)(API_GATEWAY_CUSTOM_DOMAIN_HOSTNAME)/swagger/myAPI1/v1/swagger.jsonhttps://$](<https://$/>)(API_GATEWAY_CUSTOM_DOMAIN_HOSTNAME)/swagger/myAPI2/v1/swagger.jsonThen implement a URL transformer in YARP to rewrite the URL to /swagger/v1/swagger.json just before forwarding the requests to the private services.
This is enough for most scenarios, but there is yet another special case that can occur on top of everything else.
In case you have multiple APIs, you might keep the input/output models of that API in a separate nuget package that is referenced by all APIs.
In this scenario, the generated OpenAPI Spec will miss all the XML documentation that is part of the external nuget.
All the underlined lines will be missing by default!
The problem is that there is a bug in the .NET project system that prevents from copying the existing XML doc file located in the nuget package in the output folder of the API.
A few seconds later….
<Target Name="CopyReferenceFiles" BeforeTargets="Build">
<ItemGroup>
<XmlReferenceFiles Condition="Exists('$(OutputPath)%(Filename).dll')"
Include="%(Reference.RelativeDir)%(Reference.Filename).xml" />
</ItemGroup>
<Message Text="Found XML doc for copy: %(XmlReferenceFiles.FullPath)"
Importance="Highest" />
<Copy SourceFiles="@(XmlReferenceFiles)"
DestinationFolder="$(OutputPath)"
Condition="Exists('%(RootDir)%(Directory)%(Filename)%(Extension)') AND
$([System.String]::Copy('%(Filename)').StartsWith('Connect.Api.Models'))" />
</Target>
This will iterate over all dll files and will try to copy the respective XML files to the output directory, if the file exists and if it matches the name of our models’ nuget. Otherwise a lot more XML will be copied over and will needlessly increase the deployment size.
This will work beautifully locally, but it will not work in a docker environment where the image is built by Azure pipelines.
dotnet publish command, a new line is to be added after the initial copy operation.
<Copy SourceFiles="..." DestinationFolder="$(PublishDir)" Condition="..." />
The only difference will be the destination folder, instead of the output path it will be the publish directory.
NUGET_XMLDOC_MODE environment variable to instruct the restore operation to unpack the XML docs from the nuget packages together with the DLLs. This is set by default to skip to shave a few seconds off the pipeline run time.
ENV NUGET_XMLDOC_MODE=none
RUN dotnet restore "myApiProject.csproj"
RUN dotnet build "myApiProject.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "myApiProject.csproj" -c Release -o /app/publish
To ensure the success of your API, don’t neglect the documentation side of it. Never rely on manual updates of wiki pages, Confluence pages or even using the UI editors that some tools and services put at your disposal.
Always strive to automate and correlate the API code deployment with updated docs deployment.
Depending on each particular case, the system and technology stack used and the tactics used, the hoops you might need to jump through to achieve this can vary – but the strategic goal should be the same.
API hub What is an API? The term API is an acronym, and it stands for “Application Programming Interface.” An API is a vital building block in any digital transformation strategy, and one of the most valuable in achieving scale, reach and innovation. Behind every mobile app and online experience,
API hub First open data European list of energy suppliers The most comprehensive open data list of European energy suppliers, for European energy data, ever compiled in API format is now available for free. For the first time, users will be able to integrate and browse a full database of
API hub A guide to monetising APIs In this guide learn about monetising energy APIs, and their commercial value as a new channel for monetising existing digital assets and data. APIs aren’t new. In fact, they’ve been around for quite a while, embraced fully by industries from all new digital
API hub APIs in energy Digitalisation in the energy sector. Unlike other industries where digitisation is the norm, the energy sector is a child by comparison. In many countries, electricity is still purchased via a sales representative using a paper contract. Many energy retailers, (renewable) energy producers or grid operators
API hub API Marketplace approach How APIs drive the digital transformation of energy. In this article, you will learn more about the power of APIs in the energy sector and how adopting an energy API Marketplace approach is fundamental to connecting businesses to energy data. Share on facebook Share on
API hub re.alto Talks, Part I: Realising the energy transition in times of change This webinar is Part One of a three-part series on “Realising the energy transition in times of change”. https://youtu.be/YBdnui2y904 Explore more
API hub re.alto Talks, Part III: The benefits of an API marketplace in energy This webinar is Part Three of a three-part series on “Realising the energy transition in times of change”. https://youtu.be/QjwjNNw5PSo Explore more
API hub re.alto Talks, Part III: The benefits of an API marketplace in energy This webinar is Part Three of a three-part series on “Realising the energy transition in times of change”. https://youtu.be/C2IRj699eWg Explore more
API hub Tutorial: Visualising solar forecast data using Python and web APIs Web APIs are becoming an increasingly popular standard for communicating data. The days of CSV downloads, file transfers, and even web-scraping are numbered, and for good reason. Before APIs can be used and integrated into any system or
API hub re.alto API overview re.alto energy – Technical Setup for Existing APIs On the re.alto platform, individual Users can search for, and subscribe to a Provider’s API products. These subscriptions are monitored, tracked, and (if monetised) billed and settled individually by the re.alto platform. Each subscription made by
API hub re.alto Marketplace FAQ What is the re.alto API marketplace? The re.alto marketplace is a marketplace for digital energy products and services, delivered via APIs. You can register as a provider or a consumer/user. As a provider, your digital products via APIs are uploaded to the re.alto marketplace, where
API hub Energy Quantified and re.alto The API-led approach to digital scale and industry growth As decentralisation of the energy market drives the rise of a host of smaller industry players, easy access to digital products at volume is now an essential factor for the rapid scalability desired by those
API hub APIs are everywhere – short video animation We are in living in an increasingly API-centric world. APIs are everywhere you look – and you might not even realise it. Need evidence? Gartner considers API management tools an essential component of the unrealized hybrid integration platform (HIP), currently an
API hub Adopting an ‘API as a product’ mindset with APIs APIs have enormous potential to open companies up to new revenue streams, unlock new markets and extract value from existing assets. To fully realise this potential however, APIs need to be lifted out of the sole domain of the
API hub Three things you may not know about APIs API. Application Programming Interface. It is the communications channel between two web-based applications, allows the exchange of data without any connecting physical infrastructure. APIs lie at the very heart of digital transformation. According to the 2020 State of the API
API hub Alternative APIs for Dark sky and the strategic value of weather forecasting data in energy In this article you’ll be introduced to weather data use cases and the importance of weather data within the renewable energy and digital landscape. We also do a deeper dive into alternative APIs
API hub Tutorial – building an energy data API from scratch in under an hour Web APIs are becoming increasingly widespread in the energy sector. Historically they have been a popular technology for communicating market data (e.g. wholesale electricity prices) and grid information. Now we are seeing more and more
API hub Containerisation: Introduction & Use Case 11.04.2024 An Introduction to Technologies Used in Containerisation & a re.alto Use Case Example In this article, we want to focus on the more technical side of re.alto and are going to highlight some of the technologies currently used by our developers when
API hub Scaling with Azure Container Apps & Apache Kafka 11.06.2024 This article, written by re.alto’s development team, explains how to scale with Azure Container Apps and Apache Kafka. While such documentation already exists for use with Microsoft products, our development team did not find any similar documentation on how
Winner: European Energy Data Access pilot
