At the moment, RAVEN implements two data smoothing routines. One routine uses an averaging window that has a specified data-point width, and the other routine is context sensitive and averages over a specified data width.
The first algorithm is a Savitzky-Golay digital smoothing filter [1]. This is also called a least-squares filter [2], or a Digital Smoothing Polynomial filter [3]. Essentially the filter fits a polynomial of order n to all the data within a window of width w for each data point. Due to the formulation, w must be an odd number, and larger than n+2. At the two “ends” of the data, data is mirrored to satisfy the data window for the algorithm. Details of the algorithm can be found in Numerical Recipes [4]. The algorithm is quite fast as it assumes equally spaced data and uses linear algebra to perform the least-squares fitting.
| [1] | Savitzky, A., and Golay, M.J.E., 1964, “Smoothing and Differentiation of Data by Simplified Least Squares Procedures”, Analytical Chemistry, V. 36, pp. 1627-1639. |
| [2] | Hamming, R.W., 1983, Digital Filters, 2nd Ed., Prentice-Hall. |
| [3] | Ziegler, H., 1981, “Properties of Digital Smoothing Polynomial (DISPO) Filters”, Applied Spectroscopy, V. 35, pp. 88-92. |
| [4] | Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P, 2007, Numerial Recipies: The Art of Scientific Computing, 3rd Ed., Cambridge University Press |
The second algorithm operates in the same space as the data. This requires that the base data be monotonic: either increasing or decreasing with no duplicates. The base data can be any data column that satisfies this requirement. The smoothing window is then specified in terms of this data column. For example, if “Time” is specified as the base data, the window might be “10 s”. If temperature is the base data, the window might be “2 C”. The smoothing algorithm then takes the mean of the data across the data window for each data point. The algorithm preserves the number of data points in the data set. As such, the beginning and ends of the data are less averaged than the middle due to the averaging window being clipped. This algorithm is quite a bit slower than the Savitzky-Golay algorithm.
Thinning data is generally used on larger data sets, in order to improve plotting performance. There are three thinning methods currently available in RAVEN; Convert to Grid, Arbitrary Row Reduction, and Maximum Delta.
The Convert to Grid thinning method interpolates and extrapolates the data to obtain values at fixed step intervals over a given range. The data column used as the basis for the stepping must be monotonically increasing, and only valid columns are shown in the dialog. The data extents for the selected base column are shown and used as the default range (but can be adjusted if needed). The default step value will result in 1001 data points.
Arbitrary Row Reduction thinning does no interpolation or extrapolation, and simply extracts evenly spaced data rows over the full data range to obtain the total number of rows specified. For instance, if the original data is made of 1000 rows, and the specified target is 500 rows, the thinning data will contain every second data row.
The Maximum Delta thinning method is not currently implemented.