Skip to content
Code Releases Docs (2.10.0) Docs (Dev)

This is the changelog for MeteoIO release 2.5.0

Bug fixes

  • The radiation splitting was computed using TOA_beam when it should have used TOA_h (after checking the Reindl, 1990 paper)
  • DEMObject: need to call the << >> operator of the parent class Grid2DObject in own operator<< and operator>>
  • When caching interpolated grids, the info string was not cached, leading to empty info strings when reading from cache
  • Bugfix in a constructor (that was making an error on some systems)
  • When a plugin was throwing an exception, the mapPlugins could get corrupted in the (rare) case of the caller catching and moving forward (uncaught NULL pointer). This is now not anymore possible, by design.
  • The config object now is much more verbose when reporting an invalid key and does not write out invalid key/values (basically, key with empty values). This makes it possible for example in Snowpack to re-run a simulation using the backed up ini file (without any changes).
  • Fixed a stupid bug when using "." as default path
  • Fixed a very stupid bug in Undercatch_WMO when dealing with 5 arguments (bad copy/paste)
  • Trying to close some potential security flaws (using FlawFinder). All files opened for reading are now checked and special files (devices, etc) are refused. More fixes from FlawFinder: tighter checks on valid file names and a few extra checks otherwise.
  • Rounding azimuths to 0.1 and preventing azi=360, prefering azi=0 instead
  • plugins
    • The GSN_URL key is now mandatory and the GSN_USER and GSN_PASS are now optional (for publicly accessible sensors)
    • After talking with the GSN team, the GSN plugin has been fixed and works again. It has been simplified according to their recommendations (ie that the metadata is always delivered with the data)
    • The GSN plugin now has a "GSN_DEBUG" key to print out the full dialog with the server in case of errors. Some headers returned by GSN did not follow what we expected (multi-lines description) and this is now supported. New sensors names are also supported so the Stillberg data can be read.
    • The GSN plugin has a new key: GSN_TIMEOUT. A bug in error handling was fixed in the GSN plugin.
    • The ARC grids written with Alpine3D naming were still not properly named, they should now be directly recognized by the Alpine3D grid viewer (with the "A3D_VIEW" option)
    • IMISIO: When using Snowpack simulations in the database for computing precipitation, this was leading to lots of very small events. This should now produce less events but with more precipitation.
    • GRIB: The new parameters/tables definitions have been taken into account in order to solve the problem of radiations not being properly read any more
  • science algorithms
    • A bug has been fixed in the Winstral implementation that could lead to infinite loops for very small DEMs.
    • When using Winstral with precipitation AND exclusively positive temperatures, the deposition mass was zero and was leading to an arithmetic error
    • Updated the Stefan-Boltzmann constant according to http://physics.nist.gov/cuu/index.html
    • The nodata points are now always properly filtered out when computing a least square fit. The code has been a little reshaped in order to be closer to the reference document that has also been cited. The quadratic fit was wrong and has been fixed.
    • Fixed a bug in a (currently) unsued optmized method for nth_root of a number (negative numbers were not working)
    • The simpleWindInterpolate spatial interpolation was not properly handling nodata values
    • When using Ordinary Kriging, the nodata cells where still computed... This has been fixed, they are now properly skipped.
    • An innacurracy has been fixed in IOUtils: lat_epsilon and lon_epsilon were defined in rad when they should have been in deg
    • Added a range check on RH before computing the dew point temperature in the RH spatial interpolation method
    • The original PrecipSNOW spatial interpolation algorithm has been reimplemented since the version implemented by Rob is unreliable
    • Now the Winstral algorithm properly handles the case of only one station providing HNW with the default base algorithm (by switching to CST in this case). The availability of wind data is now part of the rating return by Winstral. The case of no precipitation (for Winstral) as well as no wind (for Liston and Ryan) are now also handled in the rating method.
    • The Winstral interpolation was not behacing properly when dealing with a DEM with holes, this has been fixed

new features, improvements

  • Better reporting for DATA_QA, including the data generators. Parameters that are filtered but have no value (nodata) are now reported.
  • The parameter exclusion feature has been documented and now supports both a space delimited external file or by keys such as WFJ2::exclude = TA RH HS
  • Similarly to the "::EXCLUDE" statement, it is now possible to use a "::KEEP" statement. This makes it possible to keep only a few parameters from given stations.
  • Add the possibility to delete keys based on string match (from the begining or anywhere in the key). This is useful for example to remove all stations declared from the user in the ini file.
  • Following a bug report from GeoTop, the add_to_cache() method that had been partially removed is now back, under a slightly different name: add_to_points_cache()
  • In order to make it more modular and allow more advanced processing (virtual stations, downscaling, multiple io sources) the IOManager is now a proper wrapper: the processing is fully delegated to either a GridsManager or a TimeSeriesManager. These can in turn be reused at some other places (like in the Meteo2DInterpolator) to offer full buffering and processing of "local" time series. The buffering is now done by these two new classes (therefore the BufferedIOHandler class has been removed) and it is planned that dataClasses will be implemented for proper ringbuffers (one for time series, one for grids and one for isolated points) and receive the generic buffering code later on.
  • A gridBuffer class has finally been implemented (basically, moving the old grid buffering code into it) and is now in use for the GridsManager. The GridBuffer class has been expanded and is now used by Meteo2Dinterpolator for interpolated grids
  • Multiple DEMS can be cached
  • A getHash() method has been added to the StationData. It is currently quite unsophisticated but should do the job.
  • Finally adressing issue 72 with second resolution in dates. The ISO strings now contain the seconds when written out and by reading them, the seconds can be provided (but this is not mandatory). In ARC, grids are now named with the seconds (except when enabling the Alpine3D naming scheme). The adresses a bug that was found when requesting" raw" processing level from a plugin that constructs its timestamps from a decomposed time: time-1s and time+1s were the same and therefore this could trigger problems in the plugin.
  • The CH1903+ coordinate system has been implemented
  • grids
    • a new seter method has been created for the grids: grid.set(grid, init)
    • The grids.isEmpty() method has been renamed as empty() in order to follow the STL convention
    • The basic operators have been implemented in Grid2DObject and Grid3DObject as well as DEMObject (with the proper update strategy)
    • In order to suppress the risk of out of date "ncols, nrows" and since these were redundant with Array2D<>.nx/ny, they have been suppressed and it is therefore now mandatory to rely on the getNx/getNy/getNz getters. This is much safer but impacted quite a lot of code...
  • dates
    • A new Date::getTime() method has been written
    • The Date class can now compute the dow of week as well as the ISO week number. It is now possible to write timestamps as ISO_WEEK (this is YYYY-Www-dd).
    • A new Date print format has been added (ISO_DAY). A new date format has been added: RFC868
  • plugins
    • NetCDFIO: checkEpsilonEquality for modified julian dates instead of test for double equality
    • In order to make it easier to support ECMWF and CF1 NetCDF grids, there are now two NetCDF plugins: one for the CNRM schema (focusing on the meteo time series) and one for grids (NetCDFIO)
    • The CNRM plugin's documentation has been updated and improved. and its documentation now matches the code. For consistency, the CNRM plugin now uses METEOPATH.
    • The NetCDF plugin can now read grids and DEMs using various schema/conventions
    • The getTanMaxSlope method used for Winstral or Ryan has been optimized and improved: now a pixel that would be just behind a ridge would be properly processed (ie it would see the uphill slope and not the downhill slope behind the ridge).
    • The PNG plugin can now also properly output a hillshade (as MeteoGrids::SHADE)
    • New PSQLIO plugin for PostgreSQL
    • A new plugin has been contributed to acces the SASE Mysql database (based on MySQL)
    • A new plugin has been created to read the data from the ALPUG stations (ascii format)
    • The PNG plugin creates a few more EXIF fields in order to make it very easy to integrate in google earth
  • generators
    • The POT_RADIATION generator has been renamed as ALLSKY_SW for consistency and its documentation updated. The ALLSKY_LW generator's documentation has also been updated.
    • A new generator has been implemented to generate ISWR from RSWR or the opposite
    • Renammed and documented the ESOLIP precipitation generator after adding the reference
    • A precipitation phase data generator has been implemented. Currently, either fixed threshold or range are provided but more could easily be added
  • Resampling, spatial interpolations and filters/processing
    • The daily_solar temporal interpolation can now handle daily sums provided at midnight (then assuming they belong to the day that just finished) and the documentation has been updated
    • A new temporal interpolation method has been introduced, to generate sinusoidal variations of a given mean and range (DAILY_AVG)
    • It is now possible to only remove a specified fraction of a data set with the SUPPR filter (at random)
    • A new "NOISE" filter has been added, to add noise to the inputs
    • Implementing an interpolation algorithm that generates distributed PSUM_PH by calling the parametrizations on each cell
  • Utils
    • A new IOUtils::getCWD() method has been created, a new Config::getConfigRootDir() has been created (to help resolve relative paths) and is now used for the ADD and MULT filters as well as for the EXCLUDE feature of PSQLIO and IOHandler
    • Function to convert bearing double to string
    • The begining of a generic handling of units also appeared in IOUtils. The units conversionm now work for 'simple' units, with prefix and exponents. 'Composite' units (such as 'K/m' or 'N/m2') are currently not supported.
    • A new readDirectory() method has been implemented that directly returns a list of strings (slightly more convenient to use)
    • The K_TO_C and C_TO_K macros have been replaced by static methods in the IOUtils namespace. They now properly check for nodata in order to avoid weird values and outputs...
    • A new fastPow(double, int) has been implemented for positive exponents.
  • Meteo parameters and meteorology
    • Adding a new meteo parameter: the cloud transmissivity TAU_CLD defined as iswr_meas / iswr_clear_sky (ie the potential radiation computed at ground level with a standard atmosphere, see Iqbal 1983). This parameter is already used by the AllSky_LW and PotRad generators and properly handled by the PNG plugin.
    • PINT is now properly supported in SMET.
    • The OLWR/TSS conversion must now be handled by a data generator (implemented as "TS_OLWR")
    • A first implementation of the wet globe temperature index has been contributed (a few coefficients still must be set for the convective heat transfer coefficient and this looks like a messy business)
    • The Black Globe Temperature index should now work
    • A new method has been implemented to merge all stations close to each other within a single vector. Practically: several data sets from the same location can then be read and merged on the fly by a call to MeteoData::merge(vec).
    • A few physical constants have been added
    • A long awaited (and MAJOR) change: HNW is gone, it is now PSUM. This name should be clearer and every occurences of "HNW" (even in temporary variables or algorithms) have been replaced by psum. A check is performed on keys containing "hnw" and an exception is thrown if some are found, urging the user to update the ini file. Please keep in mind that the calling codes have to now refer to MeteoData::PSUM and that some algorithms are now named differently!
    • Adding PSUM_PH to the MeteoData enum
  • scripts
    • New script to explore the data contained in smet files
    • The smet2agr script has been modernized (changing the syntax of a regular expression)
    • Two Matlab scripts to read and plot SMET files, making use of all fields and metadata have been contributed
    • A much improved smet_extract that processes one file at a time has been committed. It can now provide a list of available fields if no field has been specified
    • The updated smet_range script now supports Julian timestamps (and converts them to ISO), properly shows empty lines for stations that miss the requested parameter and properly converts K to C when necessary. It also prints a usage message when ran without arguments and can take as argument a path.
    • The new smet2kml script creates a kml file locating all the smet stations provided in a given directory. This kml can then be loaded on map.geo.admin, googlemaps, etc and shows markers with station ids.

documentation and error messages

  • documentation
    • Trying to make the documentation more user-friendly
    • documentation update in GeneratorAlgorithms
    • Some documentation bugs have been fixed (multiple links definitions for doxygen)
    • The config.dox file has been updated (in order to remove the warnings for obsolete keys)
    • better documentation in the filters template
    • Complements to the "filter dev" documentation
    • Better plugin template, as mentioned in issue 411
    • Add documentation on how to implement new coordinate systems support (issue 410)
    • Improved the "usage examples" section of the documentation (some of the examples were way too old), added some advices
    • A bug in the Coords documentation has been fixed
    • Better documentation for the ADD and MULT filters
    • Update the trend/residual approach in the documentation
    • clarified the documentation about PINT
    • Documentation on how to install libpng on Mac
    • Some new links into the GRIB documentation
  • error messages
    • A better error message has been implemented for ProcHNWDistribute
    • Better information message when forcing a null lapse rate on a linear regression (this happens when using "frac" on precipitation for all timesteps without precip)
    • A better error message has been found for the "frac" lapse rates that can lead to a zero lapse rate (this is the case when there is no precipitation, for example)
    • The ADD and MULT filters now offer a better parsing of their arguments in order to avoid silently failing

build system and automatic tests

  • Adding the cmake script to detect MySQL. This is currently only working on Linux (no proper paths for Windows)
  • proper retrieval of compilation date and version number that also works on Windows
  • Fixing the warning on new Cmake versions about RPATH on OSX
  • Small tweaks for Windows: no version information in path anymore but in the name as it appears in "add/remove software"
  • More robust finding of MeteoIO that properly works on WinXP
  • Cleaning up the CMake files from unecessary things, proper support for x86_64 (it needs to be named as "amd64" on Debian), a few extra options for the cpack generators
  • A few options have been added to gcc compilations, including the use of LTO for releases and SANITIZE by default in debug (and OFF in release)
  • The statistical tests now cover the least square fits and ad-hoc linear regressions

code cleanup

  • removed the popc specific code
  • delete obsolete and unused file
  • better file structure: all the classes purely focused on data storage (such as grids, dates, etc) have been moved into a "dataClasses" directory. All subdirectories are now named similarly to the classes and methods: the words after the first one are capitalized.
  • The plugins are now handled by an object factory, removing the need for the IOPlugin class
  • removing the unecessary IOInterface.cc
  • The LIDW algorithm has been restructured in order to look more similar to IDW
  • the grids initializations use some recently introduced API that makes the code easier to read
  • Some fixes to make MeteoIO compile with Visual C++ 2013. The linker still complains about exported symbols (this also generate lots of warnings), but all the other issues have been fixed.
  • More code cleanup for NetCDFIO (ie getting some methods out of the way by moving them into libncpp)