---
title: "PV simulators comparison: Setup and test methodology"
slug: "comparison-setup-and-test-methodology"
description: "Compare six PV simulation tools using consistent data and metrics to understand their methodologies and outputs across diverse climates and configurations."
updated: 2026-08-10T14:43:27Z
published: 2026-08-10T14:43:27Z
canonical: "kb.solargis.com/comparison-setup-and-test-methodology"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://kb.solargis.com/llms.txt
> Use this file to discover all available pages before exploring further.

# PV simulators comparison: Setup and test methodology

**In this document**

This article describes the test setup and methodology used to compare Solargis Evaluate, Solargis Prospect, PVsyst, System Advisor Model (SAM), pvlib, and SolarFarmer across the full photovoltaic (PV) simulation chain. It defines the test sites, system configurations, simulator versions, and statistical metrics used in all subsequent comparison articles in this series.

### Overview

This series compares six PV simulation tools by applying the same input data, the same system designs, and the same evaluation metrics to every simulator. The objective is to characterize differences in simulation methodology and outputs, not to validate any simulator against measured plant data.

#### Key principles of the methodology

- The same six test sites, four system configurations, and Solargis typical meteorological year (TMY) P50 input data are used across all simulators.
- Comparisons are presented relative to Solargis Evaluate – see [note below](/v1/docs/comparison-setup-and-test-methodology#comparison-metrics).
- Results are summarized using [bias and root mean square error (RMSE)](/v1/docs/understanding-accuracy-in-solar-software#validation-statistics) at simulator-to-simulator level, aggregated across sites and configurations.
- Climate diversity of the test sites is checked using the Köppen-Geiger classification, covering tropical, arid, temperate, and continental zones.

This setup follows the practice of cross-simulator benchmarking established in published studies, including Driesse and Patel ("Cross-validation of PV Simulation Software") and the NREL technical report on flat-plate PV modeling (NREL/TP-6A20-61497).

### Test sites

Six sites are used across the comparison series, chosen for climate diversity. Climate diversity matters because solar resource characteristics, ambient temperature, soiling regimes, and atmospheric clarity differ significantly across climate zones, and each of these affects how individual simulation models perform.

The selected sites cover tropical rainforest (Af), arid hot desert (BWh), humid subtropical (Cwa/Cwb), and warm-summer humid continental (Dfb) climates. This range exercises each simulator across a representative range of conditions encountered in commercial PV projects.

| Site | Country | Latitude | Longitude | Elevation [m a.s.l.] | Köppen-Geiger zone |
| --- | --- | --- | --- | --- | --- |
| Dharan | Nepal | 26.794171° | 87.291505° | 315 | Cwa |
| Kadhdhoo | Maldives | 1.858333° | 73.519720° | 2 | Af |
| Las Vegas | Nevada, USA | 36.102767° | -115.152796° | 615 | BWh |
| Pretoria | South Africa | -25.757827° | 28.227997° | 1,410 | Cwa/Cwb |
| Sulov | Slovakia | 49.162646° | 18.582428° | 393 | Dfb |
| Yaren | Nauru | -0.543409° | 166.931995° | 28 | Af |

***Table 1****: Test sites with coordinates, elevation, and Köppen-Geiger climate classification.*

![](https://cdn.document360.io/ae2d502f-6c0d-4865-a68e-43ad8da61149/Images/Documentation/mapa-MartinO.png)

***Figure 1****: Test sites on a world map.*

### System configurations

Four PV system configurations are applied at each site, combining two tracking geometries and two PV module types. The DC and AC capacities, inverter selection, transformer, and grid connection are identical across configurations to isolate the effect of mounting geometry and module technology.

Common parameters across all configurations:

- **Inverters**: Sungrow SG350HX-20 string inverters, 2 units
- **Transformer**: generic 800 kVA MV transformer
- **AC capacity**: 704 kW
- **Grid**: no active power limit, cos phi = 1
- **String layout**: 25 modules per string, portrait orientation for tracker, landscape for fixed-tilt

Bifacial configurations use a bifaciality factor of 0.78. Ground albedo is set to the Solargis long-term average albedo (12 monthly values) at all sites.

Single-axis tracker (SAT) configurations share the same tracker geometry: Ground coverage ratio (GCR) 40%, ±60° rotation range, 1 m installation height. Fixed-tilt configurations use site-specific tilt and GCR, set according to local optimization.

| Configuration | Mounting | PV modules | DC capacity [kWp] | DC/AC ratio |
| --- | --- | --- | --- | --- |
| Config 1 | Single-axis tracker | Bifacial JAM66D45-605/LB 1,500 units | 907.5 | 1.29 |
| Config 2 | Single-axis tracker | Monofacial TSM-345DD14A(II) 2,400 units | 828.0 | 1.18 |
| Config 3 | Fixed-tilt | Bifacial JAM66D45-605/LB 1,500 units | 907.5 | 1.29 |
| Config 4 | Fixed-tilt | Monofacial TSM-345DD14A(II) 2,400 units | 828.0 | 1.18 |

***Table 2****: System configurations applied at every test site.*

| Site | Azimuth | Tilt | GCR |
| --- | --- | --- | --- |
| Dharan | 180° | 24° | 46% |
| Kadhdhoo | 180° | 4° | 46% |
| Las Vegas | 180° | 34° | 56% |
| Pretoria | 0° | 28° | 45% |
| Sulov | 180° | 37° | 37% |
| Yaren | 0° | 4° | 46% |

***Table 3****: Per-site fixed-tilt parameters.*

### Input solar and meteorological data

All simulations are run using the same input data – Solargis typical meteorological year (TMY) P50 dataset for the particular location, in 15-minute time resolution, generated from Solargis Evaluate. Solargis Prospect is an exception, as it uses its own internal input format: twelve hourly profiles, one per month, each representing an average day for that month.

### Simulator versions

Where possible, each simulator is configured to use the same input data, the same model selections, and the same system parameters, so that observed differences reflect methodology rather than configuration choices. Where a simulator does not support a given input (for example, sub-hourly data or a specific sky model), the closest available option is used and noted in the corresponding article.

Table 4 lists the six simulators compared in this series, along with the versions used at the time of testing:

| Simulator | Version | Type |
| --- | --- | --- |
| Solargis Evaluate | v2.5 | Commercial, cloud-based |
| Solargis Prospect | v2.5 | Commercial, cloud-based |
| PVsyst | v8.1.4 | Commercial, desktop |
| System Advisor Model (SAM) | v2025.4.16 | Free, desktop |
| pvlib | v0.13.1 | Open-source Python library |
| SolarFarmer | v1.6 | Commercial, desktop and cloud |

***Table 4****: Simulators compared in this series.*

> [!TIP]
> **Note:**
> 
> - **SolarFarmer** results throughout this series are theoretical only. No SolarFarmer simulations were executed for this comparison due to licensing constraints. Where SolarFarmer appears in the articles, the comparison reflects its documented methodology and default settings, not simulated output, and no bias or RMSE figures are reported for it.
> - **Solargis Prospect** is likewise excluded from the quantitative comparison. Its output is limited to twelve representative daily profiles (24 hourly values per month) rather than a full annual time series, and it does not expose per-stage loss values, so it cannot be aligned with the methodology used for the other simulators. Where Solargis Prospect appears in the series, the comparison is limited to its documented methodology.

### Simulation chain and variables

Each simulator uses its own naming and structuring conventions for the variables that pass through the simulation chain, from sun geometry through optical irradiance loss stages to electrical output. The variables seldom map one-to-one between simulators. Most cover comparable physical quantities, but some have no direct equivalent in other tools. For example, the front-side global tilted irradiance without losses is called `GTI_FRONT_NOSHD` in Solargis Evaluate, `gti_front_theoretical` in Solargis Prospect, `poa_global` in pvlib, `GlobInc` in PVsyst, and `I` or `poa_nom` in SAM. In SolarFarmer, the front-side GTI is decomposed into three parameters: `POA Beam`, `POA Diffuse`, and `POA Reflected`.

This makes numerical comparison at some stages difficult, as it is not possible to map equivalent variables that can be compared. Therefore, in the subsequent articles in the series, some numerical comparisons are performed on a reduced set of simulators. In the [AC side modeling](/v1/docs/clone-pv-simulators-comparison-electrical-modeling-of-the-ac-side#system-unavailability1), three simulation stages are grouped into one for the numerical comparison.

### Comparison metrics

Results across the simulator series are reported using two statistical indicators applied to pairwise simulator outputs: bias and RMSE. Both are reported in relative (percentage) form to allow comparison across sites with different absolute output levels.

- **Bias** quantifies the systematic difference between two simulators for a given site and configuration. A bias close to zero indicates that two simulators produce, on average, the same result. The sign indicates which simulator produces the higher value.
- **RMSE** (root mean square error) quantifies the spread of differences between two simulators across time steps. RMSE is higher than the absolute bias whenever differences vary in sign or magnitude over time. It is computed at 15-minute resolution and reported as a percentage of the average reference value.

Bias is computed as the difference between the simulator mean and the Solargis Evaluate mean, expressed as a percentage of the Solargis Evaluate mean. A positive bias therefore means the simulator computes a higher value than Solargis Evaluate. All statistics are computed on daytime time steps only, defined as those where the Solargis Evaluate sun elevation is above zero.

The formulas for calculation of both metrics are detailed in [Understanding accuracy in solar software](/v1/docs/understanding-accuracy-in-solar-software#validation-statistics) (RMSE is referred to there as RMSD).

Results are presented as graphs of bias and RMSE: a breakdown view with one panel per site, and a summary view collapsing all six sites into the p10/p50/p90 (bias) or p50/p90 (RMSE) distribution per configuration and simulator. p50 is the median of the cross-site distribution, and p10/p90 its lower and upper bounds. RMSE (always non-negative) is summarized with p50/p90 only, while bias uses the full p10/p50/p90 range since it can run positive or negative. This is unrelated to the TMY P50 input dataset described above – here p50 describes how much simulators disagree across sites, not an input data product.

> [!TIP]
> **Note**: Solargis Evaluate is used as the mathematical reference series for bias and RMSE calculations across this comparison series. These statistics require a chosen reference, and Solargis Evaluate is one suitable choice. This convention does not imply Solargis Evaluate is more accurate than the other simulators. Differences indicate where simulators diverge in methodology, not where any one is correct.

#### Value versus loss metrics

Two categories of quantity are compared across the series:

- **Value stage** metrics compare the quantity itself at a point in the simulation chain (for example, GTI after transposition, or AC power at the grid connection). Bias and RMSE are computed directly between each simulator's value and the corresponding Solargis Evaluate value, both expressed as a percentage of the mean Solargis Evaluate value.
- **Loss stage** metrics compare an individual loss step (for example, soiling loss or inverter conversion loss) computed per simulator as the difference between its own before- and after-step values. Because a loss can be near zero at some sites and configurations, loss bias and RMSE are normalized by the mean of the incoming signal at that step (the irradiance or power entering the step), not by the loss itself. This keeps the percentage stable and physically meaningful instead of spiking when the loss approaches zero. Because a loss is subtracted from the incoming signal, a negative loss bias means the simulator computes a *smaller* loss than Solargis Evaluate, and therefore passes more energy to the next stage.

The DC conversion step in [Electrical modeling of the DC side](/v1/docs/comparison-electrical-modeling-dc-side#dc-conversion) is an exception. It is quantified as module efficiency (DC power output divided by the effective irradiance reaching the cell). This approach enables comparison of the DC conversion step only, excluding the effects of the optical and shading differences.

### Further reading

- "[Understanding accuracy in solar software](/v1/docs/understanding-accuracy-in-solar-software)": Solargis
- "[Argus PV simulation chain](/v1/docs/argus-pv-simulation-chain)": Solargis
- "[PV energy yield simulation](/v1/docs/pv-energy-yield-simulation)": Solargis
- ["Cross-validation of PV Simulation Software"](https://www.researchgate.net/publication/337627001_Cross-validation_of_PV_Simulation_Software): Driesse, A., Patel, N.
- "[Validation of multiple tools for flat plate PV modeling](https://docs.nlr.gov/docs/fy14osti/61497.pdf)" (NREL/TP-6A20-61497): NREL