PV simulators comparison: Detailed results and analysis

Overview

Conclusion reports the final result and names the stages that drive it. This article shows the chain itself, node by node, for each site and configuration. It is intended for readers who need the numerical backing: energy yield analysts, independent engineers, and technical due diligence teams.

PVsyst carries most of the analysis, because it is the tool most readers will be comparing against. pvlib and SAM appear where their behavior isolates a cause that PVsyst alone cannot show. For the comparison setup (sites, system configurations, version numbers, and bias and RMSE definitions), see Setup and test methodology.

How to read the propagation figures

Each figure has one panel per test site and four traces per panel, one per system configuration. A trace shows the cumulative bias, or RMSE, at each node of the simulation chain, expressed as a percentage of the Solargis Evaluate value at that node. A negative value in the bias figures means the simulator reports less energy than Solargis Evaluate at that point.

The slope of a segment carries more information than its height. A rising segment means that stage moved the simulator upward relative to Solargis Evaluate, a falling segment means the opposite, and a flat segment means the two agree on that stage. The height at the far right is the final result reported in Conclusion.

Key insights

1. User-specified angular loss (IAM) profiles tend toward optimism. Switching PVsyst from the PAN-file incidence angle profile to the standard ASHRAE model, with the sites and everything else held constant, moves its angular loss by 0.79 to 1.95 percentage points, in the same direction in all twelve site and mounting pairs, and brings it to agreement with Solargis Evaluate within ±0.23%. pvlib and SAM, both on standard models, sit between the two.

2. Horizon treatment matters in proportion to the terrain. The three simulators that expose horizon losses separately disagree by under 0.8 percentage points at sites without terrain and by more than 4 percentage points at a mountainous site, with losses calculated by pvlib low, PVsyst high, and Solargis Evaluate between them. Screening the horizon before selecting a tool is therefore worth doing, and the direction of the disagreement depends on the mounting type.

3. Rear-side irradiance is the least settled model in the field. The three tools disagree with Solargis Evaluate on rear-side irradiance by 9% to 20% at the median and by up to 74% in individual cases, against 1% to 2% on front-side irradiance. The consequence for yield is bounded by the low rear-to-front ratio at the test sites, around 1 percentage point, but it scales with that ratio. High-albedo and elevated-mounting projects are where this uncertainty becomes material.

4. Effort is best spent upstream. DC conversion moves the final result by 1.5 to 3.0 percentage points across the three simulators. No single stage between the DC array output and the grid moves it by more than 0.25 points, and the whole span moves it by 0.6 points at most. Refining module, thermal, and angular inputs changes the answer; refining electrical side inputs mainly changes the attribution.

5. A small final bias is not evidence of agreement. All three simulators arrive to bias within about 1.1% of Solargis Evaluate at the median (results for individual test cases being frequently higher), and all three do so by partly cancelling deviations of 1.5 to 3 percentage points. pvlib shows this most plainly: it drifts to bias +1.98% through the optical chain, loses 3.01 points at DC conversion, and ends with bias of −0.93%. SAM combines the smallest median bias of the three simulators with the largest deviations in individual test cases. A comparison made only on final median yield detects neither.

PVsyst results

PVsyst follows the same trajectory as the other two simulators: an upward drift through the optical chain to a median of +1.00% at the end of it, then a single downward step at DC conversion to −1.66%, and almost no movement after that. RMSE rises steadily across the same path, from 2.07% at front-side irradiance to 2.99% after the optical chain and 3.85% at the grid connection point.

What distinguishes PVsyst is that this trajectory splits sharply by module type. Bifacial configurations end at a median of +0.18% and fall within 1% of Solargis Evaluate in 7 of 12 cases. Monofacial configurations end at −2.95%, read lower than Solargis Evaluate in all 12 cases, and fall within 1% in only 2. Figures 1 and 2 show this as two clearly separated pairs of traces in every panel.

Figure 1: Cumulative bias for PVsyst against Solargis Evaluate at each node of the simulation chain, as a percentage of the Solargis Evaluate value at that node, by mounting configuration for each of the six test sites.

Figure 2: Cumulative RMSE for PVsyst against Solargis Evaluate at each node of the simulation chain, as a percentage of the Solargis Evaluate value at that node, by mounting configuration for each of the six test sites.

Source of the module-type gap

Table 1 follows the two module types separately along the simulation chain. The two module types are identical through transposition and horizon shading, separate by 0.17 percentage points at near shading, then by 1.54 points after the angular step and 3.16 points after DC conversion. The remaining five stages together add 0.03 percentage points to the separation. Two model differences therefore account for the whole separation, and the next two sections take each in turn.

Table 1: Bias for PVsyst against Solargis Evaluate along the simulation chain, in percent, as a median across the six test sites, for the bifacial and monofacial configurations separately. The change column gives the median contribution of that stage; the cumulative column gives the median bias carried at the end of it. The two columns are computed independently, so the change values do not sum to the cumulative value.

Angular losses: PAN file and ASHRAE

The monofacial configurations used the ASHRAE angular model. The bifacial configurations used the module-specific ("user-specified") IAM profile supplied in the PAN file. Comparing the four model settings at this single stage separates the two approaches.

Table 2: Angular reflection loss bias against Solargis Evaluate, in percent of the front-side GTI entering the step, as a median across the six test sites. Negative values indicate a smaller angular loss than Solargis Evaluate.

Every setting except ASHRAE computes a smaller angular loss than Solargis Evaluate, and within that optimistic group the settings are not far apart. On the bifacial tracker, PVsyst with the PAN-file profile and pvlib sit 0.14 percentage points apart, closer to each other than pvlib and SAM are. The PAN-file profile is therefore not an outlier by distance.

What distinguishes it is direction and consistency. It is the most optimistic setting at the median in both bifacial configurations and in 10 of the 12 bifacial cases, and switching the same tool to ASHRAE on the same sites moves it to agreement with Solargis Evaluate within ±0.23%. That swing runs from 0.79 to 1.95 percentage points and points the same way in all twelve site and mounting pairs. Because only the angular model changes between the two sets of runs, this is the one controlled model comparison in the dataset: every other contrast varies the simulator and the model together.

Figure 3: Angular reflection loss bias for pvlib, PVsyst, and SAM against Solargis Evaluate, as a percentage of the front-side GTI entering the step, by mounting configuration across the six test sites.

DC conversion stage

DC conversion subtracts 2.52 percentage points from PVsyst's cumulative bias at the median, which makes it the largest single contributor to the final result. The stage combines four effects: the rear-side irradiance reaching the back of the module, the spectral correction, the cell temperature model, and the module conversion model. Comparing the rear-side irradiance and the conversion efficiency separately accounts for most of it.

Table 3: Contribution of each component to the DC conversion step, in percentage points of the bias against Solargis Evaluate, as a median across the 24 site and configuration cases. Rear-side irradiance is weighted by the bifaciality factor of 0.78 and applies to the bifacial configurations only. The unresolved remainder covers the cell temperature model, the spectral correction, and differences in the exact optical endpoint at which each simulator reports irradiance.

Module conversion efficiency dominates for all three simulators, and PVsyst's step is almost fully accounted for: 2.29 of its 2.52 percentage points come from conversion efficiency and rear-side irradiance together, leaving 0.22 points unresolved. The larger remainders for pvlib and SAM are consistent with their reporting conventions rather than with a different physical cause, because pvlib carries the largest spectral correction of the three and SAM reports irradiance at a different optical endpoint.

For the module-type difference, the decomposition changes the reading of Table 1. Conversion efficiency separates the monofacial and bifacial module types by 2.24 percentage points at this stage, and rear-side irradiance pulls the bifacial side back by 1.13 points. The visible separation of about 1 point therefore understates the conversion difference by roughly half. PVsyst's monofacial conversion efficiency reads 2.42% to 4.13% below Solargis Evaluate at every site, against 0.09% to 2.17% for bifacial, and the gap holds in all 12 site and mounting pairs. Because the two configurations use different module types, the most likely location is the parameterization of the specific monofacial module rather than the conversion model itself.

Stages downstream of the DC array

PVsyst bias moves very little from the DC array output onward. DC-side losses add 0.11 percentage points at the median, inverter conversion subtracts 0.02, and the entire AC side adds 0.23. The AC side shifts PVsyst's result more than it shifts pvlib's or SAM's, which follows from PVsyst modeling AC cable and transformer losses across three voltage stages where the others apply a single aggregate factor. No single stage after the DC array output moves the result by more than 0.23 percentage points, and the whole span moves it by 0.57.

The practical consequence is that effort spent refining electrical side inputs changes the reported yield far less than effort spent on the module, thermal, and angular models. The electrical side matters for understanding where energy is lost, not for the size of the total.

Horizon shading and site selection

Horizon shading produces the most systematic disagreement in the comparison, and the size of the disagreement is predictable from the site. Solargis Evaluate, pvlib, and PVsyst all expose a separate horizon loss, so the three can be compared directly. SAM bundles horizon with near shading and is excluded from this stage.

The direction is fixed across the whole test matrix. pvlib computes a smaller horizon loss than Solargis Evaluate in 23 of 24 cases, because it applies no separate treatment of the diffuse irradiance blocked by the horizon. PVsyst computes a larger loss in 22 of 24 cases, a consequence of the linear approximation it applies to the horizon profile. Solargis Evaluate sits between the two at every site.

Figure 4: Horizon shading loss bias for pvlib and PVsyst against Solargis Evaluate, as a percentage of the front-side GTI entering the step, breakdown by mounting configuration and the six test sites.

The spread between the two simulators scales with how much horizon the site has. At the four sites with negligible terrain, pvlib and PVsyst disagree by 0.04 to 0.75 percentage points. At Las Vegas, which carries a modest horizon, they disagree by 1.00 point on fixed tilt and 1.11 on trackers. At Sulov, in mountainous terrain, the disagreement reaches 4.36 and 4.51 points, and the horizon stage alone carries an RMSE above 10% for both tools against under 2% at every other site.

Which of the two simulators diverges more depends on the mounting type. At Sulov on fixed tilt the disagreement is almost entirely pvlib's, at +3.10 points against PVsyst's −1.26. On trackers it reverses: pvlib returns to +0.16 while PVsyst falls to −4.35. A tracker sweeps a far wider range of sun azimuths, so it encounters more of the horizon profile, which is where a linear approximation to that profile loses accuracy. A fixed array at a single orientation sees a narrower slice, where the diffuse component that pvlib omits carries proportionally more weight.

Rear-side irradiance and bifacial projects

Rear-side irradiance, at the input to the DC conversion stage with all optical losses applied, produces the largest relative disagreements anywhere in this comparison. Median bias against Solargis Evaluate is +8.9% for pvlib, −16.8% for PVsyst, and −20.4% for SAM, and individual cases run from −60% to +74%. RMSE reaches 75% for pvlib, 41% for SAM, and 24% for PVsyst. Front-side irradiance, measured the same way over the same cases, stays within 1.0% to 1.6% bias and about 3.5% RMSE, so the divergence is real rather than an artifact of how the series are aligned.

The direction of the bias differs by tool. PVsyst computes less rear-side irradiance than Solargis Evaluate in every case, in a narrow band of 10% to 22%. SAM also computes less in every case, but the shortfall widens sharply on trackers, reaching 44% at the median and 60% at Las Vegas. pvlib swings both ways, from 38% below at Dharan to 74% above at Las Vegas, which is the widest spread of any quantity in the comparison.

What keeps this from dominating the final result is the weight the rear side carries. At these six sites the rear-side irradiance averages about 7% of front-side irradiance, and the bifaciality factor of 0.78 reduces its share of the effective irradiance reaching the cell to about 5.7%. A 20% rear-side disagreement therefore moves the effective irradiance by roughly 1 percentage point. Across the twelve bifacial cases the resulting shift runs from −1.4 to −0.7 points for PVsyst, −3.3 to −0.1 for SAM, and −2.1 to +4.0 for pvlib.

Note: The impact of rear-side model disagreement scales in proportion to the rear-to-front irradiance ratio. At these test sites that ratio is about 7%, and the models differ by around 1 percentage point of yield. On a project with high ground albedo, high mounting, or wide row spacing, where the ratio can be several times larger, the same relative disagreement would produce a correspondingly larger spread.

Note that these figures also carry a definitional component. The three tools report rear-side irradiance at slightly different points in the optical chain: Solargis Evaluate after the spectral correction, SAM after the angular correction, and PVsyst as global rear-side irradiance. That difference contributes to the measured spread alongside the genuine view-factor and albedo modeling differences, and it cannot be separated from them here.

pvlib and SAM results

pvlib produces the largest optical drift and the largest DC reversal of the three tools. It computes the smallest angular loss and the smallest horizon loss of the field, which carries its cumulative bias to +1.98% by the end of the optical chain, the highest peak any simulator reaches. DC conversion then subtracts 3.01 percentage points, landing it at a median of −0.93%.

That final figure is the most misleading number in the comparison. It is the residue of two large deviations of opposite sign, not evidence of close agreement. pvlib's RMSE confirms as much: it rises at every step in all 24 cases, from 2.00% to 4.29%, the largest accumulation of the three. This is because pvlib is a library rather than a configured application, so several effects that the other simulators apply by default are simply absent unless the user programs them. A pvlib result is therefore only as complete as the model chain behind it, and the small final bias observed here should not be read as a property of the library. Its rear-side model is the clearest example: pvlib is the only tool that reads both above and below Solargis Evaluate on rear-side irradiance, from 38% below to 74% above, and carries a 75% RMSE at that stage.

SAM produces the smallest median bias of the three at −0.45%, and also the largest individual deviations in the entire comparison: −7.66% bias at Dharan on the bifacial tracker and 24.11% RMSE at Sulov on the monofacial tracker. Both extremes trace to the DC conversion stage, where SAM's contribution ranges from −6.60 to +3.09 percentage points, a spread of 9.7 points against 4.6 for PVsyst.

SAM therefore agrees well on aggregate and poorly on individual cases, which is the pattern most likely to mislead a user comparing a single project. Two structural limitations compound the problem:

1. SAM cannot separate horizon from near shading. On a site with terrain a SAM user has no way to check whether the horizon is handled reasonably, because the loss arrives already combined.

2. SAM's rear-side shortfall widens to 44% at the median on trackers against 9% on fixed tilt, which is the largest configuration dependence in the rear-side comparison and the reason its worst individual cases are bifacial trackers.

Further reading

Solargis knowledge base

• "PV simulators comparison: Overview": Solargis

• "Setup and test methodology": Solargis

• "Irradiance modeling": Solargis

• "Optical losses": Solargis

• "Electrical modeling of the DC side": Solargis

• "Electrical modeling of the AC side": Solargis

• "Conclusion": Solargis