Digitalizing Hydropower: Challenges and Solutions in Data Integration

The first challenge hydropower operators face when starting a digitalization process is data collection and integration. The need for data integration in hydropower plant operations is twofold. Firstly, it is essential for achieving safer, easier, and automated operations. Secondly, the flexibility demanded by power markets, driven by advancements in solar and wind technology, necessitates digitalization of static and live data flows. Hydropower must be able to complement wind and solar, and this requires a digital transformation.

Traditional Systems & the Need for Digitalization

On the renewable energy stage, hydropower is certainly the most mature technology. The modern era of hydropower began in the late 19th century, and significant developments occurred throughout the 20th century. However, the bulk of the hydropower infrastructure worldwide was built before 2000. Naturally, these systems primarily rely on analog hardware, which, while robust, present several limitations in today's fast-paced energy market.

Analog systems in hydropower plants typically involve manual monitoring and control processes. Operators act as a bridge between the physical realities of the plant and the digital infrastructure used to track performance and make adjustments. This approach can be time-consuming and prone to human error. Additionally, the lack of real-time data and automated responses can hinder the plant's ability to quickly adapt to changing conditions.

Despite these limitations, analog technology doesn't have to hinder digital operations. The goal shouldn't be to completely modernize all systems in one go to upgrade to "smarthydro." This approach is neither the most sustainable nor the fastest way to achieve practical results in the market. Instead, operators can adapt existing smarthydro digital solutions to the current infrastructure, making only essential hardware investments that bring value to daily operations.

While the process comes with it’s unique set of challenges, the drivers for data integration and digitalization in hydropower plants are multifaceted:

Increased Safety: Digitalization enables 24/7 surveillance of key safety parameters and preemptive, automatically adjusted responses to risks such as spills, reservoir level issues, and power market demands. This continuous monitoring enhances the overall safety of plant operations.

Operator Time Savings: Operators play a crucial role in ensuring optimal dispatch and risk management. However, in analog systems, their time is often consumed by repetitive routine checks and tasks. Automation of these tasks through digitalization can free up operators' time for more strategic operations, improving overall efficiency.

Power Market Flexibility Needs: Liberalized power markets are trending towards increased granularity, with European markets recently switching to 15-minute intervals. This change is necessary for enhanced grid stability in an energy landscape increasingly dominated by wind and solar. However, traditional hydropower infrastructure wasn't built for this level of flexibility and often faces start-stop, ramping, and environmental constraints. Digitalization can help find the sweet spot that protects the plant while taking advantage of market movements.

Knowledge Transfer Between Multiple Teams: Digitalization facilitates faster and more efficient synchronization and reporting between teams and external stakeholders. By implementing digital tools and platforms, data can be shared in real-time, reducing the lag in communication and decision-making. This improved knowledge transfer ensures that all teams are on the same page, enhancing collaboration and operational coherence. Additionally, digital reporting tools can automate the generation of reports, making it easier to provide accurate and timely information to external stakeholders, such as regulatory bodies and market operators.

By addressing these drivers, digitalization can significantly enhance the operational efficiency, safety, and market responsiveness of hydropower plants, ensuring they remain competitive in the evolving energy landscape.

This paper will cover the key areas of data collection, aggregation, transmission, and usage in automation. It will explore the practical challenges, opportunities, and existing solutions for each of these areas, providing a comprehensive overview of how data integration can facilitate the digitalization of hydropower plant operations.

Initial Set-up of Integrated Data Systems

Data collection in hydropower plants involves two primary types of data: static data and telemetry. The set-up varies with each hydropower project, but these are some typical data sources you will have to bring together under the same digital umbrella:

Static Data

Static data includes historical records and digital archives. Many hydropower plants have crucial historical data stored on paper, such as handwritten logs, blueprints, and maintenance records. Some of this data may already be digitized but stored in disparate systems or formats, making integration challenging.

Telemetry

Telemetry encompasses real-time data collection from various sources. This includes sensors and instrumentation like flow meters, pressure sensors, temperature sensors, and vibration sensors, which provide continuous operational data. Supervisory Control and Data Acquisition (SCADA) systems collect and monitor this real-time data, enabling centralized control and analysis. IoT devices connect these sensors and systems for seamless data transmission. Additionally, drones equipped with cameras and sensors collect visual and thermal data for inspection and monitoring.

Challenges & Solutions

The quality of data fed into smarthydro software directly impacts the quality of optimization on all levels. This is why data accuracy issues should be addressed before or alongside data integration challenges for optimal results:

Initial Digitalization of Analog Systems

Converting static data stored on paper into digital formats is a significant upfront challenge. This process can be time-consuming and labor-intensive, especially when dealing with handwritten records and outdated documentation. However, it is a crucial step in the digitalization process.

Implementing optical character recognition (OCR) technology can help convert handwritten and printed paper records into digital formats. This data can then be stored in centralized digital archives for easy access and integration.

Data Quality

Data quality issues can arise from various sources, including sensor malfunctions, environmental interference, and human error during data entry. High-quality data is essential for effective monitoring, control, and optimization of hydropower plant operations.

Another dimension of data collection quality is the implementation of a unified measurement system: will you measure reservoir level relative to sea level (masl) or relative to the spillway level? Regardless of the choice, it should be implemented throughout the entire system and preferably, throughout the entire fleet for best results. Using standardized data formats and protocols can simplify data integration and ensure compatibility between different systems.

Sensor Reliability

Sensors can degrade over time, leading to faulty readings. Regular maintenance and calibration are necessary to ensure their reliability. Inconsistent sensor performance can result in inaccurate data, which can compromise the effectiveness of automated systems and predictive maintenance algorithms. Ensuring sensor reliability through routine checks and timely replacements is vital for maintaining data integrity.

Implementing a routine maintenance and calibration schedule for sensors and instrumentation can improve data quality and sensor reliability.

Environmental Noise

‘Noise’ caused by wind, finite resolution in the sensor, or inaccuracy of sensors, can affect reservoir level readings, leading to inaccurate data.

Besides these systematic errors, there are artifacts related to the interference of the control units with the sensors. One common example is sensor proximity to the point where a turbine feeds. Any change in turbine operation will create a curved reservoir level reading, which reduces accuracy of the telemetry reading. On turbine start, the water level in the proximity of the turbine will suddenly drop. Conversely, when the turbine is stopped the reservoir level suddenly increases. While these outliers are easy to predict and recognise using shallow water equations (SWEs), they are just one of many ways plant operation interferes with sensor readings.

To use telemetry data for automated dispatch planning, this noise has to be prevented or filtered. Utilizing automatic data smoothing techniques are one way to filter noise, as well as resolution and outage problems of sensors, ensuring more accurate and reliable data, which can then fed into optimization algorithms.

Here’s how automatic smoothing can look like:

The top chart shows direct telemetry feed of a reservoir level, with abrupt fluctuation, often including free surface flows negative feedback. In the smoothing process, all content change recorded at turbine or gate level (see second chart), as well as estimated water loss & gain through soil humidity, inflow and precipitation are accounted for. On the last chart you can see the smoothed reservoir level curve in dark purple, in comparison with the much noisier unfiltered reservoir level data in light purple.

Once implemented, background inflow curve smoothing creates a solid foundation for smart production planning and automated operations.

Temporary Outages

Temporary outages can disrupt the continuous flow of telemetry data, causing gaps in data collection and analysis. These outages can be caused by power failures, network issues, or equipment malfunctions.

Implementing robust backup systems and redundant data pathways can help minimize the impact of temporary outages and ensure continuous data flow. Deploying edge computing devices can also reduce latency by processing data closer to the source, enabling faster decision-making.

Data Integration

Data integration challenges include ensuring compatibility between various data sources, maintaining data consistency, and enabling seamless data flow between systems. Effective data integration is essential for creating a comprehensive view of plant operations and enabling advanced analytics and optimization.

Using standardized data formats and protocols can simplify data integration and ensure compatibility between different systems. Implementing secure communication protocols, such as encryption and authentication, can protect data from cyber threats and ensure its integrity.

Data Aggregation and Management

Data integration involves combining data from multiple sources to create a unified view of plant operations. This process includes collecting, transforming, and loading data into a central repository where it can be analyzed and used for decision-making. Effective data integration ensures that all relevant information is accessible, accurate, and timely, enabling better operational insights and optimization.

Challenges & Solutions

The most common pitfall of hydro digitalization at this stage is software sprawl.

Software sprawl occurs when multiple software tools are adopted independently by different teams or departments without a cohesive strategy, resulting in overlapping functionalities, underutilized licenses, and integration challenges. To mitigate software sprawl, hydropower operators should adopt a strategic approach to digitalization. This involves selecting a unified digital platform that can integrate various functionalities, streamline data collection and analysis, and provide a cohesive user experience.

How can it be determined that the digitalization process is affected by software sprawl? A unified platform should be considered if one or more of the following challenges are posed by operations:

Redundant Applications: Various software tools are used by different teams for similar tasks, such as data analysis, monitoring, and reporting, leading to confusion and inefficiencies. Different software is used for interconnected processes such as production planning, maintenance planning, and energy trading, which could be more efficiently managed with a single tool.

Integration Issues: Data is moved manually from one system to another by operators, often requiring processing to make it compatible with another system. As a consequence, critical information is not easily accessible or shared across the organization, hindering effective decision-making.

Increased Maintenance Costs: Multiple software applications are managed and maintained either at a team level or a central level. The updates, licenses, and support of these applications are implemented separately, adding to the operational burden.

Training and Usability: Training is required for operators and staff on each of the software tools before performing a task. Switching between software is counter-intuitive due to design, usability, and language discrepancies.

Adopting a unified digital platform that integrates various functionalities, such as data collection, monitoring, analysis, and reporting, can streamline operations and reduce software sprawl. Beyond these benefits, a unified digital platform can be a much more powerful tool because its algorithms can draw from multiple data streams across the organisation. Such solutions can offer a dispatch plan that already accounts for all restrictions, planned maintenance work, mid-, and long-term production planning, market prices and inflow forecats. This comprehensive approach ensures that all operational aspects are considered, leading to more efficient and optimized plant performance.

Data Transmission

For most sites, the production planning and trading processes have been developed and implemented successfully decades ago, with minimal alterations over the years. The industry standard file format remains Excel .xls and the transmission channel, especially with external stakeholders, remains e-mail. Here is one example of daily data flow in a hydropower plant, where each node is an operator handling data, adapting file formats and measure units before moving the information further.

Challenges & Solutions

While this legacy process has worked well for decades, it exposes plant operations to two risks: one is human error and the second is untimely reaction.

This modus operandi makes trading on more dynamic power markets impossible due to its inherent inefficiencies and delays. The reliance on manual data handling increases the risk of human error, which can lead to inaccurate data being used for critical trading decisions. Additionally, the process's dependency on the responsiveness of various stakeholders means that any delay, whether from internal approvals or external suppliers, can result in missed trading opportunities. In dynamic power markets, where real-time data and swift decision-making are crucial, these delays and inaccuracies can prevent the plant from capitalizing on market fluctuations, ultimately hindering its competitiveness and profitability.

IT/OT Convergence

The integration of Information Technology (IT) and Operational Technology (OT), known as IT/OT convergence, is becoming increasingly important for digital transformation.

Here is one example of data integration and IT/OT automated transmission using an Electronic Document Management (EDM) system and an API scprit, complemented by a cloud connection via a Rest API interface:

‍

In this case, the user posts power (gross active) time series for the whole plant to the RESTfull API, then automatically picks up the planned power ahead of the market bid close. All information is processed and visualised on an interactive online platform.

During the digitalization process, a standardized digital twin model of the plant unifying the static data and the real-time telemetry was created. On this basis real-time generalized optimization tasks, visualizations and user facing control mechanisms are created. On this user interface, operators can send overrides, change operational parameters, retrieve inflow forecast and its impact on reservoir level and production planning.

Digitalized transmission systems also come with the added value of automatic repository protocols and back-ups, which help asset managers navigate different versions of their data, report, troubleshoot and backtrack if necessary. In this case, historical data is also accessible via the user interface, making it easier to access and use.

These new process infrastructures also come with a new set of challenges such as latency and security, which are typically approached using both hardware and software solutions:

Latency
Real-time monitoring and control systems are particularly latency sensitive and can benefit from the integration of edge computing. By decentralizing data processing and bringing computation closer to the data source, edge computing minimizes transmission delays and improves system responsiveness.

Security
Asset managers of any kind should take active steps to increase security when implementing digital automated data integration platforms. A first step is to vet suppliers against recognized security standards, such as ISO 27001 and SOC 2, to ensure they adhere to best practices in data protection. Additionally, choosing API connections over direct linking can enhance security, as APIs provide a controlled interface for data exchange, reducing the risk of unauthorized access and data breaches.

Data Utilization for Automation - Opportunities and Future Trends

The full potential of integrated data management can be reached by using all of it to streamline plant operations interdependently, and breaking silos between departments as follows:

• Constraint Management: real-time data can be benchmarked against fixed and flexible regulations, as well as best efficiency parameters to adjust water flow at gate and turbine level.
• Production Planning: A central data hub used for production planning can bring together all variables from market price to turbine efficiency, inflow forecasts, environmental restrictions and planned outages. Given that all parameters are weighted to account for conflict, an optimal production plan can be generated, using AI and emerging smarthydro algorithms.
• Strategic Energy Trading: Storage flexibility can be used to automate price driven dispatch using water value calculations. Moreover, once all data flows reliably feed into the same system, more demanding power markets such as primary and secondary reserve can be entered and ancillary operations can be automated.
• Medium-, and Long-term Planning: Historical and daily telemetry data can be used to forecast seasonal weather changes for your particular hydrological basin, allowing you to strategically plan production and maintenance based on opportunity cost calculations.
• Investment Decision Simulation: using digital twin technology and your data, you can assess hardware investment - such as additional turbines or pump storage retrofitting, different weather scenarios, different production strategies, and more.

HYDROGRID Insight is a digital hydropower operations platform that uses hydropower plant data to aid operators with all of the above and additional features.  

Conclusion

Integrated data management offers hydropower operators significant advantages, including improved efficiency, faster response times, and greater market adaptability. Digital tools enable automation, better decision-making, and streamlined operations without requiring a full-scale hardware overhaul. Instead, targeted upgrades and smarter integration of existing systems can deliver substantial benefits.

To ensure a smooth transition, the following is recommended:

• Objectives and KPIs: It is essential to establish clear goals, such as reducing human error and improving response times, while implementing KPIs to track progress.
• Advanced Technology: AI, machine learning, and IoT should be utilized to optimize production and support energy trading in a dynamic market.
• Process and Change Management: Stakeholders need to be engaged from the outset to develop and implement new processes, ensuring the smooth adoption of digital tools.
• Collaboration: Encouraging knowledge sharing across teams and with external partners can help drive the successful implementation of digital initiatives.

‍