In this document
We will show you how to execute the post-filtering QC check and flag scattered data with the post-filtering flag.
Post-filtering quality control (QC) is the last check to detect extremely short consecutively passed values (usually under 10) positioned between otherwise flagged values. These values can not be used for any calculations or in simulations and need to be flagged, too.
Key points to remember:
Post-filtering is available for solar radiation data only.
Post-filtering QC check should be done last, after automatic and manual QC checks.
Post-filtering QC check adds post-filtering flags to scattered values based on the configuration.
Post-filtering QC tool
You can open the post-filtering QC check tool from the main menu via Quality/Post filtering or by using the quick access icon located below the main menu. The window interface offers the following (after the check has been performed):
Run automatic checks: Execute the post-filtering quality check. The results will be displayed in the plot section, and the post-filtered data will be flagged with the post-filtering flag (blue color by default).
Visualization and summary tabs: Toggle between different visualizations to examine the post-filtering flags or view the quality control summary.
Data visualization: Provides visual insights into where the post-filtering has been applied. The flags legend is located above the charts.
Zoomed-in view: Zoomed-in visualization chart to see better where the post-filtering flag has been added.
Available solar radiation parameters: Select which parameters to check in post-filtering.
Post-filtering configuration: Use to configure how to filter out values during this QC check.
Plot settings and parameter selector: Select which parameter to display for examination and set how to visualize the data.
Executing the post-filtering QC check
When you open the solar radiation data QC tool, its interface may be empty, containing no visualizations. You must run the tests first to display the test results in the interface.
Set the minimum required size of a continuous group of valid data that should remain valid, or use the defaults.
Set how much of the valid flags during a day (in percent) will still be considered a day with a sufficient amount of valid data. All consecutive records within this period will be flagged as post-filtering.
If preferred, run the check between certain dates by selecting the checkbox and setting the dates.
Click the “Run” button at the bottom-right of the window to execute the check.
Examining the post-filtering QC results
You can examine the results and view the post-filtering flags added to the dataset once the system finishes the checks. There are several visualizations at hand to help you analyze the datasets and flagged data, and you can navigate them using the top navigation menu:
Heatmap flag plot: View the flags in a timeline heatmap. Values flagged with the post-filtering flag are displayed in blue by default (see legend above the chart).
Time series flag plot: Lets you view the post-filtering flags in two charts: time series and the monthly statistics.
Quality control summary: Provides a summary of the checks per parameter performed in the dataset.
Tip: Use the toolbar controls to work with the charts and examine the flags closely. You can learn how to work with them here.
Heatmap flag plot
The heatmap flag plot is the default tab you will see after the post-filtering QC check finishes. It gives you a first look at the data and flags that have been automatically added to the identified values.
Heatmap: Shows the color-coded data and flags in a timeline. The Y axis represents 24-hour day, and X axis represents the timeline of the dataset. One heatmap contains one year of data by default. Green color represents correct values, and various color points depict the flags in the timeline. Post-filtering flags are displayed in blue by default.
Identified flags: Flags included in the dataset for the selected parameter, their respective color in the heatmap, and what they represent.
Parameter selector: Select the parameter you want to display in the heatmap flag plot.
Time series flag plot
The time series flag plot tab lets you examine the flags in two ways - in a time series chart, where you can zoom and see particular flags, and in a monthly statistics chart, where you can visually compare amounts of different flags.
Flagged data: Values flagged with a post-filtering flag.
Flags list: A color-coded list of flags included in the dataset for the selected parameter. The same applies to the second chart.
Monthly statistics visualization: Each color represents a single flag. The amount of each flag is depicted in percent. This can help you quickly spot how much good data (green) is available for each month.
Parameter selector: Select the parameter you want to display in the charts.
Quality control summary
The quality control summary tab provides a comprehensive breakdown of post-filtering QC quality checks for each solar parameter.
Finishing post-filtering QC check
Once you are done with the post-filtering QC check, use the “Save flags” button to apply the changes and add post-filtering QC status to the dataset.