MeteoIODoc 20241221.207bde49
Data generators

New data can be generated based on some parametrizations at two very different stages:

  • in Input Data Editing, by calling a CREATE editing command;
  • when the requested data could not be provided as last resort, as data generator.

In the first case, the parametrizations are used just after reading the data with the input plugins and before any filtering and/or resampling has taken place. It is however necessary to provide a few additional arguments, as documented in Input Data Editing.

The second case takes place once the data has been read, filtered and resampled, if some data points are still missing. These are either a few isolated periods (a sensor was not functioning) that are too large for performing a statistical temporal interpolation or that a meteorological parameter was not even measured. In such a case, we generate data, generally relying on some parametrization using other meteorological parameters. In a few cases, even fully arbitrary data might be helpful (replacing missing value by a given constant so a model can run over the data gap).

Finally, it is possible to disable a given data generator / creator for specific stations, using the exclude or only options followed by a list of station IDs (see example below) as well as restrict the operation of a data generator / creator to specific time ranges using the when option followed by a comma delimited list of date intervals (represented by two ISO formatted dates seperated by ' - ').

NEW: It is also possible to restrict the operation of a data generator / creator to specific heights (with the {paramname}@{height_in_m} syntax) using the at_heights and exclude_heights options followed by a list of heights (Can be arbitrary numbers) or 'NO' for no height.

Note
it is generally not advised to use data generators in combination with spatial interpolations as this would potentially mix measured and generated values in the resulting grid. It is therefore advised to turn the data generators off and let the spatial interpolations algorithms adjust to the amount of measured data.
it is also possible to make a copy of a given parameter under a different name. This is explained in section Input Data Editing.

Data generators section

The data generators are defined per meteorological parameter. They are applied to all stations (if using multiple meteorological stations). If multiple data generators are specified for each parameter, they would be used in the order of declaration, meaning that only the data points that could not be generated by the first generator would be tentatively generated by the second generator, etc. Please also keep in mind that at this stage, all data must be in SI units!

[InputEditing]
#calling the parametrizations very early on, just after reading the raw data
*::edit1 = CREATE
*::arg1::algorithm = CST
*::arg1::param = TAU_CLD ;create a new TAU_CLD parameter set at constant 0.5
*::arg1::value = 0.5
[Generators]
#calling the parametrizations after everything else happened
#but before spatial interpolations
RH::generator1 = CST
RH::arg1::value = .7
P::generator1 = STD_PRESS
P::arg1::when = 2020-07-01 - 2020-07-10 , 2020-07-20T12:00 - 2020-08-01
ILWR::generator1 = AllSky_LW
ILWR::arg1::exclude = DAV3 DAV5
ILWR::arg1::at_heights = NO 100
ILWR::generator2 = ClearSky_LW
ILWR::arg2::only = *WFJ *DAV
ILWR::arg2::exclude_heights = 100

Available generators

The keywords defining the algorithms are the following:

  • NONE: does nothing, it allows overwriting an existing generator in an included ini file
  • STD_PRESS: standard atmospheric pressure as a function of the elevation of each station (see StandardPressureGenerator)
  • HUMIDITY: generate any of the humidity parameters from the others (see HumidityGenerator)
  • TS_OLWR: surface temperature from Outgoing Long Wave Radiation (see TsGenerator)
  • ISWR_ALBEDO: ISWR from RSWR or RSWR from ISWR with either a snow or a soil albedo, depending on HS (see IswrAlbedoGenerator)
  • CST: constant value as provided in argument (see ConstGenerator)
  • SIN: sinusoidal variation (see SinGenerator)
  • CLEARSKY_LW: use a clear sky model to generate ILWR from TA, RH (see ClearSkyLWGenerator)
  • ALLSKY_LW: use an all sky model to generate ILWR from TA, RH and cloudiness (see AllSkyLWGenerator)
  • CLEARSKY_SW: use a clear sky model to generate ISWR from TA, RH (see ClearSkySWGenerator)
  • ALLSKY_SW: generate the incoming short wave radiation from the potential radiation, corrected for cloudiness if possible (see AllSkySWGenerator)
  • TAU_CLD: generate the atmospheric transmissivity based on cloud cover fraction (see TauCLDGenerator)
  • ESOLIP: generate precipitation from snow height changes (see ESOLIPGenerator)
  • PRECSPLITTING: generate the precipitation phase and/or convert between amount / phase and split precipitation (see PrecSplitting)
  • RADCOMPONENTS: generate the global radiation ISWR from the direct and diffuse components (see RadiationComponents)
  • WINDCOMPONENTS: generate the wind velocity and/or wind direction from the U and V wind components (see WindComponents)
  • METEOINDEX: generate common meteorological indices (see MeteoIndex)

Bibliography

The data generators have been inspired by the following papers:

  • Brutsaert – "On a Derivable Formula for Long-Wave Radiation From Clear Skies", Journal of Water Resources Research, 11, No. 5, October 1975, pp 742-744.
  • Prata – "A new long-wave formula for estimating downward clear-sky radiation at the surface", Q. J. R. Meteorolo. Soc., 122, 1996, pp 1127-1151.
  • Dilley and O'Brien – "Estimating downward clear sky long-wave irradiance at the surface from screen temperature and precipitable water", Q. J. R. Meteorolo. Soc., Vol. 124, 1998, doi:10.1002/qj.49712454903
  • Clark & Allen – "The estimation of atmospheric radiation for clear and cloudy skies", Proceedings of the second national passive solar conference, 2, 1978, p 676.
  • Tang et al. – "Estimates of clear night sky emissivity in the Negev Highlands, Israel", Energy Conversion and Management, 45.11, 2004, pp 1831-1843.
  • Idso – "A set of equations for full spectrum and 8 to 14 um and 10.5 to 12.5 um thermal radiation from cloudless skies", Water Resources Research, 17, 1981, pp 295-304.
  • Kasten and Czeplak – "Solar and terrestrial radiation dependent on the amount and type of cloud", 1980, Solar energy, 24.2, pp 177-189
  • Omstedt – "A coupled one-dimensional sea ice-ocean model applied to a semi-enclosed basin", Tellus, 42 A, 568-582, 1990, DOI:10.1034/j.1600-0870.1990.t01-3-00007.
  • Konzelmann et al. – "Parameterization of global and longwave incoming radiation for the Greenland Ice Sheet." Global and Planetary change 9.1 (1994): 143-164.
  • Crawford and Duchon – "An Improved Parametrization for Estimating Effective Atmospheric Emissivity for Use in Calculating Daytime Downwelling Longwave Radiation", Journal of Applied Meteorology, 38, 1999, pp 474-480
  • Unsworth and Monteith – "Long-wave radiation at the ground", Q. J. R. Meteorolo. Soc., Vol. 101, 1975, pp 13-24
  • Meeus – "Astronomical Algorithms", second edition, 1998, Willmann-Bell, Inc., Richmond, VA, USA
  • Mair et al. – " ESOLIP–estimate of solid and liquid precipitation at sub-daily time resolution by combining snow height and rain gauge measurements", Hydrology and Earth System Sciences Discussions, 10(7), 8683-8714, 2013.