Performance Analysis

Removing Model Bias from Your Guarantee Metric

Our solar prediction tools continue to improve as they incorporate computational advances in modeling irradiance, clouds, shade, PV physics, array heat transfer, inverters, and other dynamic situations. Regardless of sophistication, however, there will always be a systematic error that shows up somehow and somewhere. Most often we will not know how this bias will manifest as we calculate performance KPIs. It is highly important to avoid a false positive or false negative – especially when applied to contractual agreements. The purpose of this article is to present a method that mitigates this issue. An added benefit is that this method allows performance managers to become model agnostic.

Important Definitions:

  • Model: This is the solar power prediction model that is configured with a project’s design parameters and loss factors. Included are characterizations for the modules, arrays, inverters, electrical power collection systems and anything else between the sun and delivery point.
  • Predicted: This is a dataset that contains energy generation calculated by a project’s Model with TMY [1] weather data as input. This TMY file is representative of the site location, and the simulated PV generation results are all a part of the Predicted data that are stored at hourly or smaller resolutions for a nominal year. (This is often referred to as an 8760hr file). [2]
  • Expected: This dataset contains energy generation calculated from the same Model but using the measured weather data collected for the plant. The measured MET data, and simulated PV generation results are all a part of the Expected data. Note that “the same model” are underlined above, and this is very important to this method. More on this later. [2]
  • Measured: The measured electrical generation logged for the plant that coincides with the measured MET data used to calculate the Expected Performance.
  • Plan: This dataset contains the planned generation throughout the project’s operating life.  This is typically stored in a pro-forma and has been used to determine the project’s sales price or financing terms. Often the TMY is stored along with the Plan generation.

Our facility is now operating, and performance managers need to understand how well things are going when compared to Plan. This is the purpose of performance KPIs. However, the actual weather most likely deviates from TMY (caused by differences in clouds, wind, ambient temperature, and snow).

Let’s say we totalize the energy production from the last month and wish to compare this to Plan. And, let’s say the month was extra cloudy and rainy. How do we do this? The common approach is to run the project’s Model with the Measured MET data, and then compare the Model’s result directly with the measured generation. This is a judicious approach – almost.

Here are just a few situations that can cause the model to no longer reproduce the original Plan with the same TMY:

  • The original Model configuration has been lost
  • The modeling software has been revised (or discontinued)
  • The original Model method is manual and highly time consuming, and there is a desire to automate with a different platform

Again, whether very simple or incredibly complex, all models are not perfect estimators, and the Model builders do not know how results may be biased. It may be that the Model has increased error at high temperatures, or it does not evaluate low irradiance well, or the inverter’s efficiency curve is inaccurate, and so on. Sometimes the bias is seasonal – over predicting during winter months possibly.

To control for model bias, do not compare Model results directly with Plan. Instead, apply a Weather Correction that is a ratio of the Predicted and Expected generation. Of utmost importance is that the Model for Predicted and Expected is identical with the only difference being the input metrological data. Here is the equation that should be used to correct Plan generation to be compared to Measured generation:

This method mitigates the bias errors because they cancel-out in the ratio:

Important, is the assumption that the bias characteristics are similar for the giving assessment period.
Here is the full equation to correct Plan so that it may be compared to Measured month energy:

To correct Measured to be compared with Plan, one just inverts the correction ratio:

And we have an unbiased assessment!

This method has been tested by applying several different modeling platforms to the same situation – all providing highly similar results. As desired, this frees us from being potentially confounded by model bias. This also allows one to become model agnostic. Please do conduct careful testing before switching modeling methods that may affect financial obligations.

Even though this article addresses solar power systems, the concept is applicable to almost all power plants. In fact, this author developed these methods during the 1980s when combined cycle power plants were a novelty and ASME PTC 46 was non-existent.

Side note: There is a very important corner-case that should be used to test any correction method. Let’s say that the measured MET data is identical to TMY. If this occurs, then there should be no need for a weather correction at all. We just compare the electrical meter readings directly with Plan. That is it! This method satisfies this test because Predicted [MWh] = Expected [MWh] and the correction factor is 1.0000000.

Second side note: Using back of module temperature measurements for performance modeling will fail this corner-case test. This is because the array heat transfer model WILL be different than the actual array heat transfer. Predicted [MWh] will not be the same number as Expected [MWh] even when Measured MET is the same as TMY. Ergo, this method is discouraged.

[1] TMY refers to a Typical Meteorological Year and represents many years of collected data for a given location providing an expected value for items such as irradiance, ambient temperature, wind, humidity, and precipitation. This is the nominal available energy for solar cycle modeling. Actual years vary of course. However, TMY values should represent the central tendency.

[2] Analysis of Photovoltaic System Energy Performance Evaluation, November 1, 2013, National Renewable Energy Laboratory. Also IEC 61724 – Part 3.

Ready to get started?