Meteo data QC

Meteo data quality check (QC) must be performed to verify the accuracy, consistency, and reliability of measured meteorological parameters such as temperature, wind speed, humidity, and air pressure. Without these checks, problems caused by sensor drift, malfunctions, calibration errors, or data gaps may go unnoticed, resulting in misleading inputs for solar modeling, forecasting, and performance evaluation.

Key points to remember:

• The meteo data QC may take some time depending on the size of the tested dataset.

• A flag column for each parameter will be added to the dataset upon this check to store the flags (if not already added).

• Flags are numbered, each representing different data issue, and these numbers are added to the flag columns when flagging the data.  

• Zooming the chart also considers the selected height as well. You can use this advantage when selecting data range with low values which would otherwise appear close to zero in the chart.

Meteo data quality check tool

You can open the meteo data check tool from the main menu via Quality/Meteo or by using the quick access icon located below the main menu. The window interface offers the following (after the check has been performed):

1. Run automatic checks: Execute the meteo data quality checks. The results will be displayed in the plot section, and suspicious data will be flagged.

2. Visualization and summary tabs: Toggle between different visualizations to examine the flagged data or view the quality control summary.

3. Data visualization: Provides visual insights into where the data inconsistencies happen and what the detected issue was. The flags legend is located above the charts.

4. Meteo parameter columns: All meteo parameters in the dataset are listed here. You can choose (checkbox) which parameters to check when running the quality control (step 1). All parameters are checked by default.

5. Plot settings and parameter selector: Select which parameter to display for examination and set how to visualize the data.

Executing the meteo data quality check

If you open the meteo data QC tool, its interface may be empty, containing no visualizations. It is likely due to no checks being executed and saved before. You must run the tests first to display the test results in the interface.

Before you run the tests, consider the following:

• Using reference dataset:  If you have a model dataset with accurate data for the location, you can use it as a reference to compare and validate the meteo data.

• Advanced tests settings: If you are an advanced user and want to configure the tests, you can do it before the test execution.

Using a reference dataset and configuring the tests

Reference dataset and test configuration are done via the meteo test settings:

1. Open the meteo data quality check settings using the icon.

2. If you want to compare the data against model dataset, select it here. (e.g., Solargis TS dataset).

3. If you want to select particular parameter tests, select the parameter here.

4. Select which tests to run for the selected parameter. Available tests are usually different depending on the parameter type.  

5. Confirm changes by clicking the “OK” button.

Running the meteo quality check

Once you configure your tests or upload your model dataset (if required), you can run the test using the “Run” button at the bottom-right. Use the checkboxes to include/exclude parameters from the QC check. All are selected by default.

Important: This test will add new flag columns to the dataset for each meteo parameter. These columns serve to add flags to particular values in the dataset.

Examining the meteo data QC results

You can examine the results and view the flags added to the dataset once the system finishes the checks. There are several visualizations at hand to help you analyze the datasets and flagged data, and you can navigate them using the top navigation menu:

1. Heatmap flag plot: View the flags in a timeline chart. Passed data are displayed in green, and flags are displayed in their respective color (see legend above the chart).

2. Time series flag plot: Lets you view the flags in two charts: time series and the monthly statistics.

3. Quality control summary: Provides a summary of the checks per parameter performed in the dataset.

Tip: Use the toolbar controls to work with the charts and examine the flags closely. You can learn how to work with them here.

Heatmap flag plot

The heatmap flag plot is the default tab you will see after the meteo data QC finishes. It gives you a first look at the data and flags that have been automatically added to the identified values. It can help you quickly evaluate the data quality.

1. Heatmap: Shows the color-coded data and flags in a timeline. The Y axis represents 24-hour cycle, and X axis represents the timeline of the dataset. One heatmap contains one year of data by default. Green color represents correct values, and various color points depict the flags in the timeline.

2. Identified flags: Flags that have been identified for the selected parameter, their respective color in the heatmap, and what they represent.

3. Parameter selector: Select the parameter you want to display in the heatmap flag plot.

Time series flag plot

The time series flag plot tab lets you examine the flags in two ways - in a time series chart, where you can zoom and see particular flags, and in a monthly statistics chart, where you can visually compare amounts of different flags.

1. Flaged data: Color-coded flagged data.

2. Flags list: A color-coded list of flags that were identified in the dataset for the selected parameter. The same applies to the second chart.

3. Monthly statistics visualization: Each color represents a single flag. The amount of each flag is depicted in percent. This can help you quickly spot how much good data (green) is available for each month.

4. Parameter selector: Select the parameter you want to display in the charts.

Quality control summary

The quality control summary tab provides a comprehensive breakdown of all meteo data quality checks that have been carried out.

Finishing meteo data check

Once you are done with the meteo data checks, use the “Save flags” button to apply the changes and add meteo data QC status to the dataset.