This is a simple ASCII plugin handling output only. It outputs meteo data in a text format that can be read by the WISKI database environment.

A WISKI database is set atop an Oracle database and built for meteo data and scalability. Some very large collections of meteorological data are stored this way. Due to its intertwined setup it is quite complicated to interact with directly resulting in more or less durable custom solutions. What always works however is an ASCII import via its native ZRXP format ensuring that all necessary processing is initiated, and all necessary recalculations within the database take place.

The WISKI database has a rather extensive set of meteo-specific filters and processing elements itself, but the versatility of MeteoIO is on a different level and for a big enough setup this comes with a performance penalty that might not be neglectable. So, the following could also be a proper use case: Export CSV data from WISKI --> Use CSV plugin to read that into MeteoIO and perform the processing --> Output as .zrxp --> Import back to WISKI.

Format

The ZRXP format is a basic ASCII-text format where in contrast to table based formats different parameters are written below each other, allowing for an arbitrary number of associated flags and annotations per value. Each parameter is preceded by one or more header lines laying out the keywords that will be used.

Currently, MeteoIO fills the following keywords:

Keyword	Meaning	Value
#	Comment line 1	Station and parameter IDs, export timestamp
#	Comment line 2	Station coordinates and altitude
ZRXPVERSION	Used version of the file format	30 (meaning 3.0)
ZRXPCREATOR	Origin of the ZRXP file	MeteoIO's library version
REXCHANGE	Data exchange number	Per station and parameter: MIO_STATIONID_PARAMETER
RINVAL	Code for invalid data	MeteoIO's nodata (-999), or the value set for `ZRXP_RINVAL`
SANR	Station ID	Station ID
CNR	[Optional] Parameter ID	MeteoIO's internal index for the parameter
TZ	Time zone	Time zone of first output datapoint
CNAME	Name of parameter	MeteoIO's short name for the parameter (if available)
CUNIT	[Optional] Unit of parameter	MeteoIO's unit for the parameter (if available)
LAYOUT	Parameter layout	(timestamp,value,status) and optionally remark

REXCHANGE denotes the exchange number/ID that is used to import the dataset into WISKI. Imports with the same ID will be associated with one measurement parameter of a certain station. For example, the exchange ID MIO_SEEG2_TA could be reserved for a MeteoIO-filtered temperature timeline of station SEEG2 and will then always contribute to this graph.

Note: CUNIT is automatically set by MeteoIO, but you can make it available as a comment only by setting ZRXP_WRITE_UNITS (see below). This is because if a unit is present, then WISKI is picky about the exact choice (a new unit has to be created in the database). If it is omitted however, it can be set in the WISKI timeline.; WISKI demands Latin-1 as character encoding. Running on Windows, you can set ZRXP_CONVERT_LATIN to true to let the operating system encode the degrees symbol (or convert after the fact with e. g. iconv -f UTF-8 -t ISO-8859-1 "$file" -o "$file".out).

Keywords

This plugin uses the following keywords in the [Output] section (all of which are optional):

ZRXP_FILE_EXTENSION: The file extension to use for the output files (default: zrxp);
ZRXP_SEPARATOR: The delimiter used to separate the header fields (default: |);
ZRXP_WRITE_UNITS: Controls whether MeteoIO's stored units for the parameters are written out. Can be KEYWORD (default, respected by the WISKI import), COMMENT (not used in WISKI import), or OFF;
ZRXP_CONVERT_LATIN: Encode non-ASCII characters on the operating system running the program (default: false);
ZRXP_WRITE_CNR: Outputs MeteoIO's internal parameter index (default: true);
ZRXP_RINVAL: The value given to WISKI to interpret as missing data (default: MeteoIO's nodata, usually -999);
ZRXP_REMARK: A remark that is passed through to each output line (default: empty);
ZRXP_NODATA_PARAMS: Parameters that are nodata throughout will be output if at least one nodata is the result of processing (default: true);
ZRXP_STATUS_UNALTERED: Status to give for data that was checked and left unchanged (default: 41);
ZRXP_STATUS_RESAMPLED: Status for temporally interpolated data (default: 42);
ZRXP_STATUS_FILTERED: Status for filtered (changed) data (default: 43);
ZRXP_STATUS_GENERATED: Status for data originating from a MeteoIO generator (default: 44);
ZRXP_STATUS_NODATA: Status for nodata values (default: disabled, has priority over all others).
ZRXP_STATUS_UNALTERED_NODATA: Status for when nodata is already present in the input file and left untouched (default: 255).

The last five status parameters are used to transport data quality assurance flags to the database. ZRXP_STATUS_NODATA will be set only if the filtered value is found to be nodata (e. g. -999). If ZRXP_STATUS_NODATA is not given a value, then this check is omitted and timesteps filtered to nodata will get the flag "qa_filtered". Furthermore, if there is a nodata value in the input file that is not altered at all by MeteoIO, then ZRXP_STATUS_UNALTERED_NODATA is set as flag, and the value is set to ZRXP_RINVAL.

Note: In addition, you can separately set ZRXP_RINVAL. This is necessary for a use case where original and potentially newer data is merged with filtered data. WISKI would fill data gaps with original data and therefore WISKI's nodata value (RINVAL) must not match MeteoIO's. This way it's possible to create persistent gaps on import according to the status flag ZRXP_STATUS_NODATA.

For the file extension, sometimes .rxp is used. The separator should either be | or ; which will result in the header fields being separated by |*| or ;*; respectively. The remark offers a rudimentary way to transport any kind of information, albeit statically and not on a per-value basis. This means that for example you can put ZRXP_REMARK = mio to give each individual timestamp-value-pair this remark. Remarks are automatically enclosed by double quotes.

Note: The internal parameter index that is passed as CNR may change within MeteoIO for special parameters, hence you can turn it off aswell and make sure WISKI does not use it.

Examples

Putting ZRXP as the METEO output plugin will produce .(z)rxp-files:

[Output]
METEOPATH = ./output
METEO     = ZRXP
ZRXP_FILE_EXTENSION = rxp
ZRXP_WRITE_UNITS    = KEYWORD
ZRXP_REMARK         = processed by mio

Output

ZRXP output could look like this if a remark is set:

##Station: 101329, export at 2019-02-06T11:53:45 (TZ: 1)
##Station coordinates (Lat/Lon): 47.5069, 11.5912, altitude: 946
##Parameter: Air Temperature
#ZRXPVERSION30|*|ZRXPCREATORMeteoIO2.80|*|
#REXCHANGEMIO_101329_TA|*|RINVAL-999|*|
#SANR101329|*|
#TZUTC0|*|
#CNAMETA|*|CUNITK|*|
#LAYOUT(timestamp,value,status,remark)|*|
20190130203000 269.81 41 "processed by mio"
20190130204500 269.66 41 "processed by mio"
20190130210000 269.63 41 "processed by mio"
20190130211500 269.56 41 "processed by mio"
20190130213000 269.49 41 "processed by mio"
20190130214500   -999 44 "processed by mio"
20190130220000 269.28 41 "processed by mio"

An arbitrary number of such blocks may be present, so in this example after all values for TA were printed the next header starting with ##Station... could follow below. Here, more information about the ZRXP format specifications is available.