I.V. Bezmenov

FSUE “VNIIFTRI”, Mendeleevo, Moscow region, Russia;
bezmenov@vniiftri.ru

Al’manac of Modern Metrology № 2 (38) 2024, pages 106–139

Abstract. This article discusses the problem of finding an unknown trend in the time series of measurement data generated by technical devices. Solving this problem is closely related to the task of detecting coarse measurements (outliers), which have a negative impact on the accuracy of estimates of various physical quantities obtained in solving many applications in which the input data are measurement results. The article proposes a method for finding a trend, which is based on the condition of maximizing the amount of data cleared from outliers and used in further processing. The reference values used to build the trend are determined as a result of an absolutely converging iterative process, the core of which is the minimization set method developed earlier by the author. At each step of the iterative process, the trend is approximated by a function from a predefined functional class dependent on the physical task under consideration.
Test problems for trend search in three following functional classes has been considered: in power polynomial class, in class of trigonometric functions with a given set of frequencies, as well as in the class of harmonic functions with unknown frequencies, phases and amplitudes. In the first two cases, the parameters of the desired trend were fitted using the least squares method. In the latter case, the trend-forming functions are non-linearly dependent on the desired parameters. Their search was carried out by the conjugate gradient method, generalized to non-linear problems. The effectiveness of the method proposed in the article was demonstrated: the relative error of the frequencies of noisy harmonic signals which were found in the latter case did not exceed the value of  4 · 10^–4. The proposed method of trend detection in measurement data can be applied in problems of predicting physical processes, as well as in problems of detecting outliers in observation data. Data cleared from outliers can be used in information and measuring systems of various types, in systems with artificial intelligence, as well as in solving various scientific, applied management and other tasks using modern computer systems in order to obtain the most reliable final result.

Keywords: information and measuring systems, time series, data pre-processing, outliers, data cleaning from outliers, optimal solution, functional class, trend detection.

Full texts of articles are available only in Russian in printed issues of the magazine.

Previous article ……. Contents ……. Next article