The energy industry is in flux, with existing change accelerated by the dual influences of global Covid-19 disruption and a heightened sense of urgency in the fight against climate change.

The upheaval to the status quo presents a unique opportunity for innovation and transformation; for digitalisation to expedite the development of new service led business models and solutions which enable long-term low carbon transition and industry growth in these challenging times.

As this whitepaper will show, the energy industry however remains relatively digitally immature, lacking the understanding and technical capability to fully realise the digital opportunity. In this paper, we explore the existing operational barriers to digital change that are deeply embedded within the industry. In particular, we examine the obstacles preventing the smooth exchange of energy data, a critical element of digitalisation, including the high cost of data acquisition and the challenges presented by legacy IT infrastructure.

At re.alto, our ambition is to accelerate digital transformation and decarbonisation through the standardised exchange of energy data and digital services. We believe that a collaborative approach through API connectivity is the key to business transformation, sidestepping the legacy infrastructure of yesterday. In this paper, we delve into use cases around e-mobility, energy communities and data-driven profiling for utilities to demonstrate how a data-led approach enables operational streamlining and unleashes entirely new revenue streams. The purpose of this whitepaper is to demonstrate the added value of an API marketplace in facilitating this data exchange and driving the creation of a digital ecosystem which ultimately enables the energy transition

Within energy, digitalisation has been a disruptive force for years, but progress has remained painfully slow.

Amongst the talk now of a ‘new normal’, the pandemic will almost certainly accelerate the momentum towards digital transformation as those with more traditional business models struggle to bounce back and find themselves unable to compete at an expedited pace of change. As recovery now becomes the priority, the pan-European infrastructure upgrade initiatives promised to stimulate the sector offer a unique opportunity to accelerate decarbonisation, enabled by digitalisation and an API-led connectivity approach. Post-Covid, building a digital eco-system and forging new collaborative industry partnerships to support next-generation business models will be more important than ever to move towards a dynamic and sustainable energy future.

I. The role of COVID-19 in accelerating change

The energy industry has felt the impact of the pandemic in key areas such as demand reduction and the supply chain thanks to lockdown. Rather than raise new issues however, the disruption from Covid-19 has instead exacerbated existing challenges and is accelerating change that had already been well underway. It has, in fact, given us a fascinating glimpse into the future of a power market where renewable sources (RES) outstrip non-renewables in system share and demonstrated the opportunities for innovation.

At the height of the first wave in April, electricity demand dropped by nearly 20% across Europe as business and transport networks slashed their usage during global lockdown. Despite short-term recovery as restrictions on movement were eased, the longer-term impact of Covid-19 on demand continues to ripple through the energy market. In the last week of July, electricity demand was still 5% below 2019 in all EU countries bar Italy, while global energy demand Is expected to contract by 6% in 2020, the largest drop in more than 70 years.

A combination of lower lockdown demand and surging RES generation has caused electricity prices across Europe to consistently dip into the red. At the same time, demand for non-renewable sources has significantly dampened. During this period, wind delivered 17% of Europe’s electricity, producing 241 terawatt-hours of electricity , while at one particular point in early April, renewables accounted for a record 70% of UK electricity demand. Coal consumption in the UK, on the other hand, fell from 50 Mtpa to 5Mtpa.

Covid-19 and  the disruption it has caused should not be seen as the prime cause of this shift in the power mix, but rather an important contributory factor in the acceleration of an ongoing industry transition. Social and political factors have too played their part, in particular, a decline in oil production in light of the Russia-OPEC price war and political volatility in the Middle East. There is no doubt that the energy industry is currently in flux and Covid-19 is helping to further shake up the status quo. While the major European RES generators are experiencing their highest profit margins in years, low wholesale electricity prices mean that energy suppliers continue to struggle to maintain profit margins with their single KW/h revenue stream. Grid operators are encountering greater technical challenges in balancing erratic RES generation, demanding more advanced demand-side management and the increased flexibility that comes from a decentralised system.

One particular benefit of the unique combination of decreased non-renewables production, the rise in RES generation and the worldwide restrictions on mass movement is a reduction in global carbon emissions, which have temporarily plunged by 8% to the lowest recorded levels since 2010. As promising as as that sounds, a rebound in emissions levels is inevitable in the long-term without sweeping intrinsic strategic change across the energy industry and beyond. Growing social and political pressure to reduce CO2 emissions has made decarbonisation a core objective for all energy companies, with an ever increasing sense of urgency as we fight to head off irreversible climate damage, caused in no small part by decades of fossil fuel burning and car exhausts.

The Covid-19 crisis has helped to create fertile ground for rapid innovation. The upheaval presents an opportunity to accelerate the development of new digital solutions which drive both long-term low carbon transition and industry growth in the face of economic stress.

II. The data-led opportunity

Digitalisation is the enabler of both growth and decarbonisation. Fuelled by data, it is the key to overcoming the dual economic and environmental crises of Covid-19 and climate change that we are now facing.  Energy data is the very foundation stone of digitalisation in the industry. As we have explored in our previous ebook ‘Understanding how APIs drive the digital transformation of energy’ data is already recognised and valued as a crucial building block in the creation of a smart energy system. It goes beyond simple consumption metrics, spanning the value chain from SCADA data to market pricing and weather forecasting.

Harnessing the power of this data should be a key objective in any digital strategy of an energy company looking to scale. Data connectivity enables innovation on a scale previous unachievable in energy as well as the creation of new customer-centric business models. It is a powerful driver of growth in the sustainable energy sector.

There is a recognition across multiple sectors that electrification is a clean route to a sustainable future, and there has already been a noticeable shift towards a decentralised renewables system with significant technological developments. But while a proliferation in open data combined with increasingly advanced algorithms and availability of real-time processing capability is revolutionising sectors such as telecoms and insurance, the energy industry is failing to keep up. As such we are not yet seeing the widespread evolution of commercial models which adopt the required data-led digital collaborative approach to unleash innovation and take decentralisation and decarbonisation further still. The smooth exchange of data which we see in other industries, a critical element of digitalisation, is hindered in energy by two key barriers to change: firstly, the relative immaturity of the industry in realising the potential of available data. Secondly, the costs associated with data acquisition and, for traditional incumbents in particular, the issues involved in integrating data streams with existing legacy IT infrastructure.

Barrier 1: Realising the potential of data

The energy industry remains fairly rudimentary in its understanding of the value of data and in particular its role in developing new digital products and services quickly and at lower cost. Given this, it is hardly surprising that many organisations within energy still fail to therefore recognise the importance of web based APIs, a standard means of making an organisation’s data and services digitally available to external developers and partners.

The perception remains that APIs belong only within the IT domain, used for internal processes at best. As such, much data exchange nowadays still uses the approach of sending large CSV files to an FTP server, a remarkably outdated method of enabling the communication between two systems. Moreover, CSV files only allow for data in its simplest form while using an FTP server prevents any real-time response or error handling, which significantly restricts the possibility of any sophisticated data interaction between two parties.

Over the last decade, APIs have evolved and scaled in volume, and their worth as a technological interface to facilitate agile internal processes and improve operational efficiency is well recognised. But without a basic understanding of the value of the vast streams of data at its disposal, the energy industry simply will not be able to fully recognise the commercial value of an API as a revenue source in its own right.

Barrier 2: Data acquisition and integration (access and cost)

If understanding the value of data is the first step to digital connectivity, access to data is the crucial second step, or in the case of the energy industry, a hurdle. There is currently no central search engine for energy API discovery as most energy data remains siloed, and navigating the jungle of companies, services and products in search of data in web-based API format is challenging. As the industry further diversifies into new sectors such as smart mobility, data is likely to become even more fragmented. Moreover, a significant lack of transparency around pricing and availability within the market makes it difficult to even benchmark the monetary value of data.

With smart meter data in particular, there is added complexity around the handling and user consent to consider. Unlike data generated by smart phone or social media usage, data generated by smart meters legally belongs to the end user and requires explicit consent before it can be accessed. Data protection obligations may restrict what customer and systems data can be shared, and it requires an awareness of the implications of data security as well as potentially the development of specific IT architecture to support privacy requirements.

Beyond access, the costs associated with integration and ongoing maintenance can be an obstacle. For larger incumbent energy organisations, onboarding each individual API from a new supplier can bring higher than necessary costs around compliance, procurement and security – far outweighing the far smaller basic cost of the API licence – as well as administrative costs stemming from disparate contractual documentation, largely non-standardised and all from separate sources. Each API subsequently also requires significant resource in ongoing management. API providers will regularly update the APIs with new features, security fixes and product improvements, but unless monitored closely, these have the potential to cause backward compatibility issues – and a break in critical business processes – when third-party endpoints are removed or relocated. The more APIs that are integrated, the more prohibitive the level of resource and cost required for initial technical set up and ongoing maintenance. It is unsurprising therefore that data acquisition costs can easily soar, transforming what should be a cost effective and quick solution for rapid product development into a significant investment and budget drain.

Incumbents in particular face a second barrier to digital transformation beyond the cost of data acquisition. A lack of experience in cloud computing and real time processing technology often leaves them incapable of creating or accessing the technology platforms required to deliver big data processing capability. Complex and inflexible legacy IT architecture and a scarcity of technical skillsets ensure that the integration of multiple disparate data sources is prohibitively difficult. For new smaller industry players, the disruptors for whom digitalisation already lies at the very heart of their business model, the challenge lies less in legacy technology and more in the availability of resource and their own limited budgets. They may be forging the innovation path by maximising the value of their own data to offer digital products because they recognise their power to add value to customers. But their internal business processes and offline B2B sales activity remain remarkably analogue in nature and  operationally inefficient. These digitally minded companies encounter the same challenges as the incumbents in being unable to devote the required resource to discover, integrate and maintain multiple API integrations as part of a long-term API-led strategy. They simply struggle to fully exploit the value of third-party data internally as well as externally, and as a result cannot efficiently streamline their operations into a cost-effective end-to-end digital pathway.

Seen in a less comprehensive form in other industries as API management platforms but not translated to the energy sector until now, an API marketplace connects the providers and consumers of data in API format.

It facilitates the connection between parties, simplifying the exchange of data and digital services by amassing them in one place in a standardised form. As a broad high-level outcome, this API economy allows organisations to create and interact with a much broader ecosystem of service providers and consumers, increasing market visibility and opening up new revenue streams with agile product development.

In much the same way as the traditional energy business model of a single centralised supply and a passive consumer is becoming obsolete, so too is the market model of single party transactions – that between one buyer and one seller – which make scaling slow and cumbersome. In today’s connected world, this is being displaced by the more collaborative model of flexible multi partner ecosystems, leveraged through the channel of a digital marketplace and the exposure it brings.

The benefits of an API-led approach

Taking a deeper dive, the API marketplace offers operational and cost efficiencies on a technical level which make a key difference in streamlining day-to-day data-led business activity. In light of the stresses placed on the global economy by Covid-19 and downward trends in spending and investment, these benefits which deliver growth and revenue in times of crisis become increasingly attractive. Put simply, a digital marketplace for energy APIs is the nexus that the energy industry needs to unlock the unrestricted flow of information across the entire value chain in the most technologically sophisticated, cost-effective and operationally efficient manner possible.

The standardisation of data transfer technology in the form of web-based RESTful APIs facilitates integration and allows for faster cross-sector collaboration and digitalisation at scale. It is the modern replacement for the CSV/FTP approach. The importance of having structured data in a standardised format that consumers can easily identify and subscribe to has already been realised across other industries. We already know that there are enormous quantities of fragmented and diverse energy data across the industry; making it available to third parties in standardised API format with standardised contractual documentation improves the quality of the information exchange and reduces friction.

Centralised digital transaction management

As we’ve seen, a data-led approach raises the need for the structured management of a myriad of disparate APIs, which can be time and cost-heavy. Standard API management platforms, much like developer portals, tend to focus on the basic transactional nature of an API subscription – a means to solely find and subscribe to the API. Some enable evaluation of the utilisation of the product, any service disruptions, and the technical adequacy of a setup. This delivers data, yes, but does little to mitigate the hefty resource requirements for setup and ongoing maintenance. A digital marketplace however delivers a full API management package in one integrated system. It not only provides a discovery mechanism for a full range of API products (and not simply a vendor-specific set), but also offers an array of online tools which span discovery, centralised governance and standards, security, user consent management, contract management, centralised billing and settlement and usage analytics, all designed to maximise the value of the APIs.

Let’s take the example of a utility requiring the subscription of five separate APIs in the development of their own digital app aimed at improving customer experience through usage tracking and digital billing. At least one of the APIs will comprise smart meter data. Acquired separately, the utility would need to spend a significant amount of time sourcing the APIs, negotiating the contracts and obtaining all required documentation, managing the initial set-up and performing all ongoing monitoring and maintenance. Consider firstly that the entire initial B2B sales process of finding and subscribing to the APIs will likely take place offline as there are unlikely to be any digital processes in place with each of the five API providers. Secondly, consider the added complexity brought by the integration of smart meter data from residential end users – aggregated and anonymised, but still requiring user consent management and throwing up privacy and security considerations which cannot be ignored.

And finally, consider the time that will be required to monitor and maintain each of the five APIs in the long-term to ensure that any updates or fixes from the provider do not interrupt the functionality of the utility’s own new digital app.

Now, let’s take that example and translate it into the resource and cost savings of performing the same transactions and long-term management of the five APIs on one integrated API marketplace. All discovery, subscriptions, standardised documentation and user consents (where needed) can be taken care of in one place in a fully digitalised online sales journey, while all ongoing maintenance can be bundled together to leave the API consumer one update to perform rather than five. The increased efficiency of an outsourced fully end-to-end digital process on a digital marketplace is staggering.

USE CASE 2: ENERGY COMMUNITIES AND DISTRIBUTED GENERATION

Local energy communities (LECs), where generation is dispersed across smaller local plants, have previously attracted much support, largely thanks to how they empower consumers to play a far more active role in the green energy transition. Over recent years, LECs have become increasingly prevalent, particularly across Europe, as consumers now demand a more active role in their own supply. Pioneering digital technology is accelerating the move towards this decentralised system where renewables play the starring role. Virtual power plants (VPP), for example have been described as the ‘internet of energy’ and are showing promise in taking power aggregation to a new level and driving virtual generating capacity to take pressure off centralised assets.

Aligning grid balance at a local level through technology-enabled demand response solutions offers up far greater flexibility to flatten the load curve and integrate volatile sustainable sources. It has been forecast that digitally enabled demand response combined with increased storage could reduce the curtailment of PV and wind power from 7% to 1.6% by 2040 in the EU alone, cutting a possible 30 million tonnes of CO2 emissions[1]. Continued advances in storage solutions are likely to take this decentralisation trend even further. It is an evolution of a new collaborative business model, turning a previously centralised system on its head with clear cost benefits to the consumer – no longer a system designed on the economies of scale as with behemoth fossil fuel plants, distributed generation drives down costs instead through the economics of small-scale and volume. But in order to thrive, digitalisation is vital to coordinate the more granular nature of distributed generation and ensure that the power supply remains consistent, reliable and secure.

USE CASE 3: UTILITIES AND DATA-DRIVEN PROFILING

Energy data, such as that from smart meters and HEMS, provides utilities with two valuable insight streams: consumption metrics and behavioural patterns. Following in the enormously successful footsteps of Amazon and Netflix, utilities are now beginning to realise the value of the millions of human behavioural big data points – such as how and when customers interact with services – and understand how it underpins the ability to understand and engage on a more meaningful level with the customer. In particular, it offers the opportunity to add value by delivering personalised services and apps based entirely on data-driven preferences. Digital flexible billing and visibility over real-time variable tariff information are just two examples of such user-centric services which put the customer more in control and help to avoid annual bill shocks. For the utility, these services not only enhance customer loyalty but also could  in theory shorten the settlement cycle – delivering an accurate invoice directly to a customer over a digital channel is shown to speed up payment, cutting the risk of bad debt.