The solar market is booming. According to a report by industry association SolarPower Europe, 2019 saw a 100% increase in the market, the strongest growth year since 2010. As demand for clean energy increases and renewables push forward with greater market share, more new solar capacity (on both a commercial scale as well as residential PV systems) was installed across Europe than any other power generation technology last year.

The sun is very much shining on the solar market, it seems. But there is a lingering grey cloud causing headaches for network owners.

Much has already been written about the challenges involved in the integration of solar power into electricity grids, and in particular the delicate balancing act to stabilise the grid in the face of two-way distribution surges.

As residential PV systems boom, the amount of surplus solar energy exported back to the grid by prosumer households is flooding the network. The majority of utility systems however are still set up for traditional one-way distribution and are simply not designed to cope with a surge of incoming energy above a particular level generated at peak daylight, usually when demand is low.

Thanks to Covid-19, we’ve already seen how low demand leading to excess supply can destabilise electricity prices. But unless balanced correctly, low load and high generation can also lead to overvoltage and grid instability.

Network owners can mitigate the issue around voltage control with investment in grid updates and innovative storage solutions to absorb the excess supply, amongst other options. But what about the overvoltage effect on the thousands of prosumer households who are looking to maximise their solar production? When voltage reaches a certain threshold, inverters from solar panels tend to disconnect from the grid as per regulation requirements. They literally shut down to protect the grid if they detect a problem.

And ironically, the more neighbours with solar panels, the greater the supply so potentially the higher the voltage and the less the panels may ultimately generate as the local grid struggles to keep up. What’s worse however is that the homeowners are unlikely to even be fully aware that their solar generation has stopped, despite the fact that most inverters nowadays are connected directly to the internet, as grid operators don’t monitor it remotely or provide alerts.

So what’s the solution? Data. Specifically, data from smart meters coupled with a home energy management system (HEMS) application, usually on a smartphone, for real-time data analytics to detect anomalies and proactively push alerts to the homeowner. Smart meters and HEMS accelerate demand-side management capability, offering greater control in particular to the consumer over their own energy supply and demand in real-time, but the benefits cascade further down the energy value chain.

At one end of the chain, the end-users can be alerted to potential overvoltage issues and help by increasing their load during times of peak generation, usually the middle of a sunny day. They could schedule the charging of an electric car, for example. Currently, this type of demand-side management relies on proactive manual actions by the end-users, but technology is starting to emerge which will automate the process.

Further up the energy value chain, for the network owner, real-time analytics of smart meter data would work hand-in-hand with smart grid load balancing. For the energy trader forecasting the load of customers in general, consumption data offers the opportunity to adjust forecasts based on behavioural profiling.

So smart meter data and big data analytics certainly have the potential to significantly improve the dynamic management of energy supply and demand, drive innovation and enhance the decision-making process for parties across the industry. But there are two hurdles to clear.

Firstly, further steps in technology are required to make the leap towards true digital transformation in energy. There are still too many limitations around real-time analytics, for example, for utility owners to really maximise the opportunity that smart meter data currently presents.

The second obstacle focuses on the controversial issue of privacy. As expected, there are strict regulations around access to smart meter data, made tighter with recent GDPR regulation changes. To reduce the risk associated with personal data sharing – and energy consumption data from smart meters certainly falls into this category – smart meter data tends to be aggregated and anonymised to preserve privacy by masking consumption patterns by individual households amongst larger neighbourhood datasets or removing any localisation information. Customer-specific data is generally only used for billing purposes, while utilities use aggregated data sets for system planning.

Does aggregating and anonymising smart meter data reduce the benefits? Both have value as a solution to different use cases, but arguably, yes.

Let’s look specifically at the example of a homeowner with a solar panel installation who is experiencing the voltage problems we’ve outlined above. In his case, the issues come down to an individual gridline so is a problem for his street but may not be noticeable at all for those living in the next. But with his utility receiving aggregated data from smart meters across the neighbourhood, accurate analytics at such a granular level are impossible, leaving the issue unidentified and unsolved.

Aggregated data may work perfectly well for energy traders who just need to identify patterns in consumption to accurately forecast demand. But for the voltage optimisation required to counter solar panel disconnection or better balance the local grid, more detailed data at an individual consumption level is required for both consumer and utility. The better the granular detail, the better the insight. Working within data protection guidelines, predominantly through opt-in customer consent, this data collected by utilities could be provided to strictly regulated third parties across a standardised data exchange platform and used to really transform the industry, from the end-user customer experience to those involved in generation, supply, transmission and distribution.