BaHaMAs: a method for uncertainty quantification in geodetic time series and its application in short-term prediction of length of day

Abstract

Some of the important geodetic time series used in various Earth science disciplines are provided without uncertainty estimates. This can affect the validity of conclusions based on such data. However, an efficient uncertainty quantification algorithm to tackle this problem is currently not available. Here we present a methodology to approximate the aleatoric uncertainty in time series, called Bayesian Hamiltonian Monte Carlo Autoencoders (BaHaMAs). BaHaMAs is based on three elements: (1) self-supervised autoencoders that learn the underlying structure of the time series, (2) Bayesian machine learning that accurately quantifies the data uncertainty, and (3) Monte Carlo sampling that follows the Hamiltonian dynamics. The method can be applied in various fields in the Earth sciences. As an example, we focus on Atmospheric and Oceanic Angular Momentum time series (AAM and OAM, respectively), which are typically provided without uncertainty information. We apply our methodology to 3-hourly AAM and OAM time series and quantify the uncertainty in the data from 1976 up to the end of 2022. Furthermore, since Length of Day (LOD) is a geodetic time series that is closely connected to AAM and OAM and its short-term prediction is important for various space-geodetic applications, we show that the use of the derived uncertainties alongside the time series of AAM and OAM improves the prediction performance of LOD on average by 17% for different time spans. Finally, a comparison with alternative uncertainty quantification baseline methods, i.e., variational autoencoders and deep ensembles, reveals that BaHaMAs is more accurate in quantifying uncertainty.

Graphical Abstract

1 Introduction

Quantifying the uncertainty in geodetic time series is an essential task (Kiani Shahvandi and Soja 2022a), since it provides a measure of the reliability of the data (Hüllermeier and Waegeman 2021). This type of uncertainty is called aleatoric, which is related to the inherent characteristics of the phenomenon the time series represents. Geodetic time series and the associated uncertainty information can be subsequently used in various other applications that use these data as input. However, often in practice, information regarding the aleatoric uncertainty is not provided. For instance, climate variables such as zonal and meridional winds provided by European Centre for Medium-Range Weather Forecasts (see e.g., ECMWF; Uppala et al. Uppala et al. 2005) lack rigorous uncertainty estimates. As a result, products such as Atmospheric and Oceanic Angular Momentum functions (AAM and OAM, respectively) that rely on these data lack uncertainty information as well, such as those provided by the GFZ German Research Centre for Geosciences (GFZ; Dobslaw et al. 2010, Dobslaw and Dill 2018). When predicting these time series, it is possible to derive the so-called prediction uncertainty, which is a measure of prediction reliability. This can be achieved using methods such as deep ensembles (DEs; Lakshminarayanan et al. 2016), wherein various models with different parameters are fitted simultaneously to the same time series. Subsequently, prediction statistics are computed from the predictions of individual models. However, assigning uncertainty to the original time series requires a different approach, as no prediction is made. The so-called autoencoders (see e.g., Baldi 2012) can be considered as a viable approach for approximating the aleatoric uncertainty in the data by learning it from the time series itself. In autoencoders, the input to the mapping pipeline and its output are the same, and the algorithm tries to discover the hidden structure in the data and reconstruct the input. However, to avoid a simple linear mapping that fully recovers the input (in other words avoiding input memorization) a so-called bottleneck is designed in the algorithm whereby the degree of predictability of the time series is analyzed. In other words, if not for the bottleneck, the parameters of the algorithm would be derived such that the input X is passed through the identity function $f(X)=X$, fully recovering the input. The usual bottlenecks used in autoencoders are batch normalization (Loffe and Szegedy 2015), layer normalization (Ba et al. 2016), or instant normalization (Ulyanov et al. 2016). These normalize the output of the neural network in each layer, preventing the autoencoder from converging to the identity function.

A variety of autoencoder approaches have been proposed. One of the most important classes of these approaches are the so-called Variational AutoEncoders (VAEs; Kingma and Welling Kingma and Welling 2014), which are based on Bayesian models. In VAEs, a model approximates the posterior distribution, typically Gaussian, with unknown mean and standard deviation. By random sampling from a normal distribution and iteratively minimizing the loss function (which is based on the negative logarithm of posterior and prior likelihoods, both having a normal distribution) the mean and standard deviation of the approximating posterior are derived. Although widely used for deriving a distribution over the output of the autoencoders, Yang et al. (2021) have shown that Hamiltonian Monte Carlo sampling (HMC; Neal 2011) is more accurate than variational sampling, partly due to the fact that the samples follow Hamiltonian dynamics (Lowenstein 2012). HMC thereby accurately represents the geometry of the problem, for example by finding the best optimization path and thus facilitating the loss minimization. We therefore present a new autoencoder method that uses HMC sampling, named Bayesian HMC Autoencoders (BaHaMAs). BaHaMAs is a general method that can be used for various time series, especially in the field of Earth sciences, where knowledge of uncertainty plays a pivotal role in the analyses (Kiani Shahvandi and Soja 2022a). To present the applicability of the method to Earth science data, we focus on the atmosphere and ocean and their role in causing variations of the rotation of the Earth (Brzeziński et al. 2002). This is achieved by analyzing the net effect of global atmospheric winds and pressure variations, as well as oceanic currents and mass redistribution, resulting in the AAM and OAM time series (Gross 2015).

As mentioned, AAM and OAM time series are usually provided without uncertainty information. AAM and OAM are widely used in different studies for the prediction of Earth Orientation Parameters (see e.g., EOPs; Gross 2015), including Dill et al. (2019); Kiani Shahvandi et al. (2022b); Kiani Shahvandi and Soja (2022d); Gou et al. (2023). Notwithstanding the improvements in predicting EOPs when AAM and OAM are used, the prediction accuracy can reach a limit due to the unquantified uncertainties of AAM and OAM, obscuring the predictability of EOPs. In case these uncertainties are available, it would be possible to design and use algorithms that directly take into account the uncertainty in the data for prediction purposes (Kiani Shahvandi and Soja 2022a). Among EOPs, this is particularly important for the case of Length of Day (LOD, unit milliseconds (ms)), because it is directly proportional to the AAM and OAM variations from subdaily to interannual time scales (Gross 2015; Kiani Shahvandi et al. 2024d). Therefore, we primarily focus on improving the prediction of LOD by quantifying the uncertainties in AAM and OAM. Note that the net sum of Hydrological and Sea-Level Angular Momentum functions (HAM and SLAM, respectively) as provided by GFZ (see, for instance, Dill et al. 2019) is close to zero (due to mass conservation in the continent-ocean system), thus the contribution of HAM and SLAM to the LOD prediction are not analyzed. We mainly focus on the short-term prediction of LOD (up to 30 days into the future) using the data provided by the International Earth Rotation and Reference Systems Service (IERS). The reason is the 4-week latency between the rapid and final data, implying that this forecasting horizon is most widely demanded and after this time interval the final observations are made available, making predictions less useful (Kiani Shahvandi et al. 2023a).

It should be mentioned that there are some studies that quantify the uncertainty in the AAM and OAM data. Lehmann and Névir (2012) quantify the uncertainties in the AAM data by comparing them with radiosonde observations and analyzing the agreement between these two approaches. Quinn et al. (2019) follow a similar path for quantifying the OAM uncertainties by comparing their OAM time series with those of other studies. Koot et al. (2006), however, follow a different approach more similar to ours by analyzing the noise level in the AAM data by mathematical means rather than by comparing it with other series. Similarly, Li et al. (2023) analyze the noise level in the angular momentum data for the purpose of EOP prediction but do not quantify the uncertainties in the data. Dill et al. (2023) quantify the uncertainties in AAM but rather for the forecasts, implying what they provide is, as mentioned before, the prediction uncertainty rather than the uncertainty in the data. However, currently no uncertainty information is provided for AAM and OAM data by operational data centers. We therefore present our method for uncertainty quantification in these time series. In summary, the goals of the study are as follows:

• presenting a novel and general methodology for uncertainty quantification in time series,

• demonstrating the usefulness of the uncertainties derived for AAM and OAM by incorporating them in the prediction of LOD.

The rest of this paper is organized as follows. In Section Methods, the methodology is described in detail. In Section Data description, the data used for the numerical studies are described. Section Results and discussions is used to present the numerical results and analyses. Finally, Section Conclusions is used for drawing conclusions.

2 Methods

2.1 Rationale behind BaHaMAs

As mentioned in the Introduction, BaHaMAs is based on the assumption of the predictability of time series (see e.g., Xu et al. 2019). In fact, BaHaMAs captures epistemic uncertainty due to the model deficiencies (Sullivan 2015). Therefore, the degree to which we can predict the time series is a representation of aleatoric uncertainty. BaHaMAs essentially approximates the aleatoric uncertainties through a loss function akin to traditional least squares. Least squares estimators are shown to be unbiased (Lukman et al. 2020); therefore, the uncertainties estimated by BaHaMAs are mathematically convergent to the ’true’ uncertainties, provided that sufficient number of samples are used in the algorithm. Nevertheless, we note that these uncertainties are approximate values and do not necessarily correspond to the ’true’ yet unknown uncertainties. For instance, instrument measurement errors might not be captured if their effect in the time series is not evident. In the following sections, however, we derive a condition on the BaHaMAs that guarantees the convergence to the ’true’ uncertainties. Hence, the uncertainties derived by BaHaMAs are useful to gain insight to the time series and its inherent uncertainty.

Considering that BaHaMAs is based on autoencoders and Bayesian machine learning with HMC sampling, in the following we present the methodology in two parts. We first discuss the autoencoder part.

2.2 Autoencoders

We denote the input to the autoencoder f as X, where X can be multidimensional (such as batch dimension, feature dimension, among others). The autoencoder f itself can be written as the composition of m neural networks $f_1,...,f_m$ as in Eq. (1). It is important to mention that these neural networks are parametric functions, i.e., having parameters $\theta _1,...,\theta _m$ that must be optimized to best represent and reconstruct the input:

$$\begin{aligned} \begin{aligned} f = f_m\big (f_{m-1}...(f_1)\big ) \end{aligned} \end{aligned}$$

(1)

As mentioned in the Introduction, to create a bottleneck that allows to quantify the aleatoric uncertainty, we apply batch, layer, or instance normalization to the output of each function (except for $f_m$, which is the final output that needs to be mapped to the observations). Therefore, the following operator in Eq. (2) is applied to $f_1,...,f_{m-1}$:

$$\begin{aligned} \begin{aligned} {\mathbb {N}}(f_i) = \frac{f_i-\mu _{f_i}}{\sqrt{\sigma ^2_{f_i} + \epsilon }},\qquad i=1,...,m-1 \end{aligned} \end{aligned}$$

(2)

where $\mu$ and $\sigma$ denote the mean and standard deviation of the output $f_i$, respectively, and $\epsilon$ is a numerical stabilizer that prevents division by zero in the optimization phase (in case $\sigma ^2_{f_i}$ is too small, the numerical results would be unstable). Similar to the case of DEs, a value of $\epsilon =10^{-8}$ has proven to be quite effective in deriving accurate and representative uncertainties (Kiani Shahvandi et al. 2023a, 2024a, b) and is therefore adopted here as well. Even though the form of the normalization operator ${\mathbb {N}}$ is similar for batch, layer, and instance normalization, the computation is different. In batch normalization $\mu ,\sigma$ are computed for each feature in X separately, whereas in layer normalization they are computed for each input across all features. In case of multidimensional time series input (such as sequence of images), $\mu ,\sigma$ are computed for each batch separately and over spatial dimensions (such as rows and columns in an image). However, if the input is a one-dimensional time series the instance normalization is identical to batch normalization. The suitability of each of these three different normalization schemes depends on the problem under consideration. In Section Results and discussions we analyze the impact of these different approaches on the prediction accuracy of LOD. Finally, we note that even though our approach is based on the application of normalization operators as bottleneck, autoencoders in general do not need normalization. Even the very presence of bottlenecks is questioned (Yong and Brintrup 2022), as the main purpose of autoencoders is dimensionality reduction. However, since the application of normalization in autoencoders improves their sensivity to out of distribution samples (Zhou 2022), we have applied them here in our approach.

2.3 Bayesian HMC

With the design of the architecture of the autoencoder as above, we proceed to the optimization phase, where the difference between the output of the autoencoders and the input should be minimized. In this phase, Bayesian HMC is applied. In Bayesian HMC, the posterior distribution in the Bayes theorem (see, for instance, Jaynes 2012), denoted by $P(\theta |X)$ where $\theta$ represents the parameters of autoencoder f, is approximated as in Eq. (3):

$$\begin{aligned} \begin{aligned} P(\theta |X) \approx P(X|\theta )P(\theta ) \end{aligned} \end{aligned}$$

(3)

where $P(\theta )$ is the prior distribution of parameters and $P(X|\theta )$ is the likelihood of observations given the parameters $\theta =\big \{\theta _i\big \}_{i=1}^{m}$. $P(\theta )$ is usually formulated as having a normal distribution with zero mean and variance as a parameter that needs to be optimized in the algorithm (Yang et al. 2021). On the other hand, $P(X|\theta )$ has a Gaussian distribution with mean f. The optimization is done by minimizing $-\log \big (P(X|\theta )P(\theta )\big )=-\log \big (P(X|\theta )\big )-\log (P(\theta ))$, where $\log$ denotes the natural logarithm function. Following Yang et al. (2021), we provide the explicit form of the optimization problem in Eq. (4). Note that constants such as $\frac{N}{2}\log (2\pi )$ that would appear in the loss function are omitted, since they neither participate in the optimization nor influence the results:

$$\begin{aligned} \min _{{\theta ,\sigma _{\theta } }} - \log \left( {P\left( {\theta |X} \right)} \right) = & \min _{{\theta ,\sigma _{\theta } }} \left( { - \log \left( {P\left( {X|\theta } \right)} \right) - \log \left( {P\left( \theta \right)} \right)} \right) \\ = & \min _{{\theta ,\sigma _{\theta } }} \left( {\frac{1}{2}\sum\limits_{{i = 1}}^{N} {\left( {f\left( {X_{i} ,\theta } \right) - X_{i} } \right)^{2} } + \frac{1}{2}\sum\limits_{{j = 1}}^{{N_{\theta } }} {\log } \left( {\sigma _{{\theta _{j} }}^{2} } \right) + \frac{{|\theta _{j} |^{2} }}{{\sigma _{{\theta _{j} }}^{2} }}} \right) \\ \end{aligned}$$

(4)

In this equation, N is the total number of samples in the time series and $N_\theta$ the total number of parameters. $\sigma _{\theta }=\big \{\sigma _{\theta _i}\big \}_{i=1}^{m}$ are the standard deviation of parameters considered as hyperparameters, which can be optimized together with the parameters $\theta$ by gradient descent and its variants (Bengio 2000; Bergstra and Bengio 2012). In this case, the uncertainty of parameters is part of the parameters denoted as $\theta$, which are then jointly optimized (in other words $\theta \longleftarrow \big [\theta ,\sigma _\theta \big ]$). To optimize Eq. (4), Bayesian HMC is applied, where Q samples $\theta ^{(k)},~k=1,...,Q$ are generated and used to compute the mean and standard deviation of the autoencoder output. First, $\theta ^{(0)}$ is generated by random sampling from normal distribution ${\mathcal {N}}(0,1)$. Then, for each sample $\theta ^{(k)}$ we follow the optimization scheme as follows. First, an auxiliary variable r at step $k-1$ (i.e., $r^{(k-1)}$) is sampled from multidimensional normal distribution ${\mathcal {N}}(0,I_{N_\theta })$, where I denotes the identity matrix. This is due to the fact that Hamiltonian dynamics (in differential form) is represented by two intertwined variables $\theta$ and r. $\theta ^{(k-1)}$ and $r^{(k-1)}$ are subsequently used as initial values $\theta _0$ and $r_0$ for the optimization step with learning rate $\delta t$ and L epochs, as in Eq. (5):

$$\begin{gathered} U\left( \theta \right) = \frac{1}{2}\sum\limits_{{i = 1}}^{N} {\left( {f\left( {X_{i} ,\theta } \right) - X_{i} } \right)^{2} } + \frac{1}{2}\sum\limits_{{j = 1}}^{{N_{\theta } }} {\log } \left( {\sigma _{{\theta _{j} }}^{2} } \right) + \frac{{|\theta _{j} |^{2} }}{{\sigma _{{\theta _{j} }}^{2} }} \hfill \\ {\text{for}}\quad s = 0,...,L - 1:\qquad \hfill \\ r_{s} \leftarrow r_{s} - \frac{1}{2}\delta t\nabla U(\theta _{s} )\qquad \hfill \\ \theta _{{s + 1}} \leftarrow \theta _{s} + \delta t~r_{s} \qquad \hfill \\ r_{{s + 1}} \leftarrow r_{s} - \frac{1}{2}\delta t\nabla U\left( {\theta _{{s + 1}} } \right) \hfill \\ \end{gathered}$$

(5)

The final values $\theta _L$ and $r_L$ from Eq. (5) are passed through the Metropolis-Hastings step (see, for instance, Robert 2015), whereby a random number p is generated from the uniform distribution ${\mathcal {U}}(0, 1)$ and used to ensure that the derived values sufficiently satisfy the Hamiltonian condition as follows:

$$\begin{gathered} H(\theta ,r) = U\left( \theta \right) + \frac{1}{2}|r|^{2} \hfill \\ \alpha = \min \,\left( {1,\exp {\mkern 1mu} \left( {H(\theta _{L} ,r_{L} )} \right) - H\left( {\theta ^{{\left( {k - 1} \right)}} ,r^{{\left( {k - 1} \right)}} } \right)} \right) \hfill \\ {\text{if}}~p \ge \alpha :\qquad \hfill \\ \theta ^{{\left( k \right)}} = \theta _{L} \hfill \\ {\text{if}}~p < \alpha : \hfill \\ \theta ^{{\left( k \right)}} = \theta ^{{\left( {k - 1} \right)}} \hfill \\ \end{gathered}$$

(6)

After performing the mentioned steps for all the samples $k=1,...,Q$, a Markov chain is derived: $\big \{f\big (X,\theta ^{(Q-k+1)}\big )\big \}_{k=1}^{Q}$. The mean of this chain, $\mu$, is the reconstructed value. What we are interested in is the standard deviation of this chain, $\sigma$, which represents the aleatoric uncertainty in the data, computed as in Eq. (7):

$$\begin{gathered} \mu = \frac{1}{Q}\sum\limits_{{k = 1}}^{Q} f \left( {X,\theta ^{{\left( {Q - k + 1} \right)}} } \right) \hfill \\ \sigma = \sqrt {\frac{1}{Q}\sum\limits_{{k = 1}}^{Q} {\left( {f\left( {X,\theta ^{{\left( {Q - k + 1} \right)}} } \right) - \mu } \right)^{2} } } \hfill \\ \end{gathered}$$

(7)

2.4 Convergence property of BaHaMAs

Here, we discuss whether the uncertainties derived by BaHaMAs correspond to the ’true’ uncertainties. In other words, our goal is to derive certain conditions under which the uncertainties derived by BaHaMAs are convergent to the actual uncertainties in the data. For this purpose, we need to analyze the consistency (or unbiasedness) of BaHaMAs. The general approach for consistency theorems is based on theorems derived by Jennrich (1969) and Wu (1981). Following Wu (1981), we present the following proposition.

Proposition 1

The uncertainties derived by BaHaMAs correspond to the ’true’ uncertainties provided that for two different set of parameters $\theta$ and $\theta '$ the quantity $\big |f(X,\theta )-f(X,\theta ')\big |$ scales proportional to $N^{-p}$, where $\frac{1}{2}<p\le 1$.

The proof of the proposition is given in the Appendix.

Proposition 1 provides important constraints on the convergence property of BaHaMAs: to derive accurate uncertainties, the number of samples Q from the parameters space should be appropriate. Furthermore, the role and choice of bottleneck is important in that the algorithm should not fully reconstruct the input, as the condition above will be violated. In Section Results and discussions we discuss the appropriate choices of Q and bottleneck more thoroughly.

2.5 BaHaMAs applied to sequences

The relations in Eq. (7) are valid if the output appears only once in the computations. In other words, if the inputs and outputs to the algorithm are taken from a moving window in the time series, each output would appear several times in the calculations. One of the reasons to use moving windows on the time series is to take into account the temporal dependency in the data. For instance, consider that for the first sample the first 30 values (i.e., 1,...,30) of the time series are taken as input (and because the architecture is an autoencoder they are the outputs as well). Then, the second sample contains the next 30 values with one shift (i.e., 2,...,31). But these samples have 29 overlapping values, which must be taken into account when computing $\mu ,\sigma$ from Eq. (7).

Let us assume samples are taken in a moving window of length M from $X_i,~i=1,...,N$. Then, the total number of samples is $N-M+1$, each with length M. The maximum number of times a sample appears in the outputs is then equivalent to $z=\min \big (M, N-M+1\big )$. Note, however, our goal is to assign uncertainty to each individual value in the time series and therefore, $X_i,~i=1,...,N$ should be considered. The first z values in $X_i,~i=1,...,N$ for each of the N outputs appear $h_i=i,~i=1,...,z$ times respectively. The next $N-2z$ values all appear $h_i=z,~i=z+1,...,N-z$ times. Finally, the last z values appear $h_i=N-i+1,~i=N-z+1,...,N$ times. Based on these, the relations in equations in (7) are modified as in Eq. (8):

$$\begin{gathered} \mu = \frac{1}{Q}\sum\limits_{{k = 1}}^{Q} {\sum\limits_{{j = 1}}^{{h_{k} }} {\frac{1}{{h_{k} }}} } f^{{\left( j \right)}} \left( {X,\theta ^{{\left( {Q - k + 1} \right)}} } \right) \hfill \\ \sigma = \sqrt {\frac{1}{Q}\sum\limits_{{k = 1}}^{Q} {\sum\limits_{{j = 1}}^{{h_{k} }} {\frac{1}{{h_{k} }}} } \left( {f^{{\left( j \right)}} \left( {X,\theta ^{{\left( {Q - k + 1} \right)}} } \right) - \mu } \right)^{2} } \hfill \\ \end{gathered}$$

(8)

where $f^{(j)}$ represents the j-th time the value appears in the outputs. It is important to note that the definition of $\sigma$ can be changed to represent only the mean of the individual standard deviations for each $f^{(j)}$ in Eq. (8). Even though simpler, we consider Eq. (8) to be more accurate, because it takes into account the contribution of each output to the mean and standard deviation by the number of instances they appear in the output. Finally, the same convergence property as in Proposition 1 applies here as well.

2.6 Architecture of BaHaMAs

It should be mentioned that since BaHaMAs is based on autoencoders, m (i.e., number of neural networks) is usually an even number in order for the algorithm to represent a symmetrical architecture together with the input. The hidden dimension of the encoder part of autoencoders, i.e., neural networks $f_1,...,f_\ell ,~\ell =\frac{m-2}{2}$, form a descending sequence. The lowest hidden dimension is that of the intersection between the encoder and decoder parts of the autoencoder, i.e., $f_{\ell +1}$ (in our numerical analyses it has dimension 1). On the other hand, the hidden dimension of neural networks $f_{\ell +2},...,f_{m}$ form an ascending sequence and that of $f_m$ matches the dimension of the input X. Note that in the general case each of $f_i,~i=1,...,m$ are neural networks with arbitrary number of layers. However, since we are using m neural networks and to facilitate training and interpretation, we use neural networks with only one layer. Therefore, the autoencoder is a stack of m neural networks each with one layer. Finally, it should be mentioned that the form of $f_i,~i=1,...,m$ can be chosen from a vast number of possible architectures. We focus on Multi-Layer Perceptron (MLP; refer to Bishop 2006). Additionally, we use Long Short-Term Memory (LSTM; Hochreiter and Schmidhuber 1997) to analyze the influence of model architecture on the robustness of the results. In summary, in Fig. 1, we have shown the schematic architecture of BaHaMAs in its general case.

Fig. 1

Proposed architecture of BaHaMAs in its general case where the input X is multidimensional and each of the neural networks $f_i,~i=1,...,m$ has multiple layers with parameters $\theta _{i,l,n}$ (i: neural network number; l: layer number; n: hidden neuron number). Sampling of $\theta$ through HMC is performed from normal and Gaussian distributions. ${\mathbb {N}}$ represent the normalization in Eq. (2)

2.7 Application of BaHaMAs in earth rotation analysis and its importance

To analyze the influence of the derived uncertainties as in Eq. (8), we follow the approach of Kiani Shahvandi and Soja (2022a), in which the loss function is modified according to the observational uncertainty (reciprocal of the variance) to pay more attention to the values that are more accurate. The usefulness of these uncertainties can be analyzed by computing the prediction performance improvement of an independent – yet related – time series in case of using these uncertainties against the case when they are not. As mentioned, the time series that we use for this purpose is that of LOD. For the prediction of LOD, we use the model proposed by Gou et al. (2023), which is based on LSTM neural networks. This model is composed of an Encoder-Decoder LSTM architecture which contains three consecutive LSTM layers with 10, 14, and 7 hidden neurons, respectively, followed by two time-distributed dense layers with 14 and 1 hidden neurons, respectively. Our analyses are mainly based on the application of BaHaMAs to the AAM and OAM time series. However, we also analyze the case where we also use AAM and OAM forecasts to improve the prediction accuracy.

We focus on this application, because prediction of LOD is an important task in spacecraft navigation and orientation of deep space telescopes (Gou et al. 2023). Thus, by improving the LOD predictions, we contribute to the improvement in important space geodetic applications. Furthermore, since AAM and OAM are quantities that can be used for the study of weather and climate, knowledge of the uncertainty of AAM and OAM plays an important role in understanding ways to improve the accuracy of AAM and OAM, thus benefiting the environmental studies.

2.8 Evaluation metrics

We note that our metric for prediction accuracy is Mean Absolute Error (MAE), as established in various EOP prediction studies (Kalarus et al. 2010; Kiani Shahvandi et al. 2022b). If the predicted and the final value to evaluate against are ${\hat{y}}$ and y, respectively, MAE is computed as in Eq. (9) based on D prediction values (i.e., total number of predictions is D):

$$\begin{aligned} \begin{aligned} \text {MAE} = \frac{1}{D}\sum _{j=1}^{D}\big |y_j-{\hat{y}}_j\big | \end{aligned} \end{aligned}$$

(9)

The improvement (in percentage) is computed by comparing the MAE of the cases with and without the AAM and OAM uncertainties considered, denoted by $\text {MAE}^{w}$ and $\text {MAE}^{wo}$ respectively, as in Eq. (10):

$$\begin{aligned} \begin{aligned} \text {improvement} = 100\%\frac{\text {MAE}^{wo}-\text {MAE}^{w}}{\text {MAE}^{wo}} \end{aligned} \end{aligned}$$

(10)

Figure 2 summarizes the algorithm for the quantification of uncertainty in AAM and OAM, and their use in the prediction of LOD.

Fig. 2

Summary of the BaHaMAs algorithm for the quantification of AAM and OAM uncertainty and their use in the prediction of LOD

3 Data description

3.1 Data used in the analyses

For the numerical results presented in the paper, we use three different data sets. First, we use the AAM and OAM series provided by GFZ (Dobslaw et al. 2010). The temporal resolution of the data is 3 h and the data span the range 1976 to the present. Since angular momentum is either caused by mass redistribution or motion relative to the reference frame (Gross 2015), the data are divided into mass and motion terms. GFZ uses numerical weather model data of ECMWF (Uppala et al. 2005) and the climate-ocean forcing models to compute the net effect of global pressure and zonal and meridional wind variations through integral formulas defining AAM and OAM. Even though for LOD variations the motion terms are by far the most important (Gross 2015), we also analyze the mass terms. We only analyze the axial component of these series, because they are directly proportional to LOD. The equatorial components could be analyzed for the case of polar motion prediction in future studies.

Regarding LOD, we use the recent IERS EOP series named IERS 20 C04 (Petit and Luzum 2010). The reason to use this series is that Kiani Shahvandi et al. (2023a) have shown this series is more accurate than its predecessor, i.e., IERS 14 C04 (Bizouard et al. 2019) for the application of EOP prediction, mainly due to the rigorous alignment to the International Terrestrial Reference Frame 2020 (Altamimi et al. 2023).

For the highest accuracy in the prediction of EOPs, it is important to use AAM and OAM forecasts. The length of the forecasts is also important (Kur et al. 2022). For these reasons, we use the 14-day-ahead AAM and OAM forecasts provided by Geodetic Prediction Center (GPC; Soja et al. 2022) at ETH Zurich, which are shown to present state-of-the-art accuracy compared to the currently available 6-day-ahead forecasts (Kiani Shahvandi et al. 2022c). However, these series are provided without uncertainty information. We therefore apply our algorithm to these forecast data as well to first quantify the uncertainties and subsequently use them in the prediction of LOD.

Figure 3 shows the mass terms of axial AAM and OAM from 1976 to the end of 2022, i.e., our study period. Similarly, in Fig. 4 we display the motion terms of AAM and OAM in the study period. These are the input and output of the BaHaMAs algorithm. Each of the series is analyzed individually, because autoencoders take one time series at a time.

3.2 Studies and their evaluation intervals

The LOD observations since 1976 to the end of 2022 are shown in Fig. 5. Since the goal is to analyze the improvement in prediction performance once the uncertainties in the AAM and OAM data are derived and used in the prediction of LOD, we present two studies to rigorously analyze the results. In study 1, the LOD, AAM, and OAM data (with their uncertainties) in the range 1976 to the end of 2000 are used for training, and LOD predictions are made in the range 2001 to the end of 2005. In study 2, on the other hand, the training and prediction intervals are 1976 to the end of 2017, and 2018 to the end of 2022, respectively. These separate intervals are long enough to give us robust results, thus adding confidence to the conclusions drawn.

Fig. 3

Mass terms of a AAM and b OAM in the range 1976 to the end of 2022. The unit is ms

Fig. 4

Motions terms of a AAM and b OAM in the range 1976 to the end of 2022. The unit is ms

Fig. 5

LOD time series (in ms) from 1976 to the end of 2022. Two separate studies are presented. First: training in the range 1976–2001 and predicting in 2001–2006. Second: training in the range 1976–2018 and predicting in 2018–2023

4 Results and discussion

4.1 Configurations of BaHaMAs

In our default setting, we use MLP (with tangent hyperbolic activation function) as the base neural network. According to Gou et al. (2023), an input sequence length of 30 is effective for the prediction of LOD using AAM data. Considering this and that the temporal resolution of AAM and OAM data is 3 h (8 times per day), we chose our windowing size to be 240. Then, the first and second neural networks reduce the dimension from 240 to 8, and from 8 to 1. Subsequently, the decoder part increases the dimension from 1 to 8, and finally from 8 to 240 again. Furthermore, we utilize $Q=50000$ samples, learning rate $\delta t=5\times 10^{-4}$, and number of epochs $L=200$. However, we investigate the effect of sample size and model architecture on the prediction accuracy of LOD. The value chosen for the learning rate is confirmed and used in various studies (Gou et al. 2023; Kiani Shahvandi et al. 2023a, 2024a). Furthermore, the number of epochs is quite robust as convergence is usually achieved in much fewer epochs than 200. The reason is the use of early stopping (Prechelt 2012) in the computations wherein if the loss function is not reduced after a certain number of epochs (in our case 30, from Gou et al. 2023), we assume that the minimum of the loss function has already been reached. Finally, the layer normalization is applied, but the impact of this choice and the comparison with other normalizations are presented in the following analyses.

4.2 Uncertainties derived by BaHaMAs

We show the results of applying BaHaMAs to the AAM and OAM in Fig. 6. The distribution of the uncertainties are also shown (at the level of one standard deviation). From these figures, most of the uncertainties are below 0.06 ms. For AAM and OAM mass terms (as well as OAM motion term), the uncertainties are mostly around 0.01 ms. This is already close to the uncertainties of LOD time series as provided by IERS (around 0.01 ms), implying that AAM and OAM have the potential to improve the prediction performance of LOD prediction. For AAM motion term, the uncertainties in earlier times are larger and around 0.02 ms, but over time the uncertainties are reduced such that in the interval 2010–2023 they are close to 0.01 ms. By dividing the time series values with their associated uncertainty, overall the approximate signal-to-noise ratio is 90% or higher, implying that AAM and OAM time series are sufficiently accurate. The larger uncertainties around 1986 are due to the large and anomalous change of the AAM motion term in the time series itself. This could be potentially attributed to the 1986 El Niño event which had a distinctive characteristic (Chen and Li 2018), causing the LOD to change rapidly.

4.3 Validating the uncertainties: comparison with differences between two independent series

We present the differences between AAM and OAM series of GFZ and another independent series, namely SYstèmes de Référence Temps-Espace (SYRTE), in Figs. 7–8. Since these two data centers provide the time series independently, the mentioned differences might approximate the aleatoric uncertainty provided that the exact same mathematical approach is used to compute AAM and OAM. As shown from the figures, the differences for OAM mass and motion terms, as well as AAM mass term are in good agreement with the uncertainty values we derived using BaHaMAs. However, for the AAM motion term, the differences are large and almost five times bigger than those derived by BaHaMAs. This might imply that the uncertainties derived by BaHaMAs for the AAM motion term are overoptimistic. Nevertheless, it is crucial to note that there are visibly pronounced signals in the AAM motion term differences that are probably related to the omission of some contributors to the AAM variations, such as the El Niño Southern Oscillation (ENSO) – represented as Multivariate ENSO Index (MEI; Wolter and Timlin 1993) – that exhibits similar amplitude and variations on interannual time scales(Zheng et al. 2003; Yu et al. 2021). By analyzing the differences between AAM and OAM series of GFZ and several other institutes, Dobslaw and Dill (2019) also conclude that there are large discrepancies that are most probably due to the computation of the AAM and OAM themselves (such as inaccuracies in the removal of tidal variations) and not so much related to the intrinsic uncertainty in the input data. Therefore, our derived uncertainty values might in fact more accurately represent the aleatoric uncertainties. Further evidence supporting our conclusion comes from the fact that the AAM motion term can be predicted to the future as accurately as 0.01 ms (Kiani Shahvandi et al. 2022c, 2023b, 2024c), suggesting that the aleatoric uncertainty in the data is similar to the value that we derive using BaHaMAs.

Fig. 6

Reconstructed aleatoric uncertainty by BaHaMAs for a AAM mass term, b OAM mass term, (c) AAM motion term, and d OAM motion term. The distribution of uncertainties (histogram) are also shown. The unit is ms

Fig. 7

Differences between mass terms of a AAM and b OAM of GFZ and SYRTE. The unit is ms. The shaded areas represent the region containing values in one standard deviation (computed across all these difference values). The horizontal black dashed lines show the region in one standard deviation derived from BaHaMAs

Fig. 8

Differences between motion terms of a AAM and b OAM of GFZ and SYRTE. The unit is ms. The shaded areas represent the region containing values in one standard deviation (computed across all these difference values). The horizontal black dashed lines show the region in one standard deviation derived from BaHaMAs

4.4 Prediction of LOD

Next, we use the uncertainties derived alongside the AAM and OAM time series in a machine learning framework in the loss function of which the uncertainties are considered (here only for the observations). This is inspired by the weighted least squares methodology and based on the approach of Kiani Shahvandi and Soja (2022a). The predictions made are then compared with the final IERS 20 C04 time series and the MAE is computed. We present the MAE for the case where uncertainties are used and compare it with the case when no uncertainty information is used in the LOD prediction algorithm, Fig. 9. As observed, in both studies mentioned in Fig. 5 the prediction accuracy is improved, confirming the usefulness of the uncertainties derived. The average improvement across all prediction horizons is 17% and 13% for study 1 and 2 respectively. We note that, longer predictions are less accurate and therefore, the potential of improving predictions by using uncertainties is better realized (as much as 22%. The numerical values are summarized in the Appendix B).

Fig. 9

Prediction accuracy of LOD in terms of MAE in ms for both studies 1 and 2. The results of two cases are compared: with and without the use of uncertainty values derived for AAM and OAM using BaHaMAs

4.5 Analyzing the impact of sample size

To test the impact of sample size (Q) on the robustness of the uncertainties derived, we compute the average MAE of LOD across all prediction horizons for Q in the range 1 to 50,000 (with increment 1). The results are presented in Fig. 10. As seen, the average MAE with $Q<5000$ is almost twice as large as that of $Q=50000$, implying that more than 5000 samples are needed for high accuracies. As suggested by this diagram, the relationship between sample size and the average MAE is exponential, resembling the traditional Monte Carlo sampling that scales with the reciprocal of the square root of sample size (Thomopoulos 2013). Even though as the sample size increases the average MAE decreases, no significant gain in accuracy is obtained after around $Q=30000$. Nevertheless, we choose to use $Q=50000$ samples to ensure that the highest accuracy is obtained in all other analyses as well, as presented in the following.

Fig. 10

Impact of sample size Q on the average prediction accuracy of LOD, for studies 1 and 2

4.6 Analyzing the impact of model architecture

We analyze the influence of the model architecture on the robustness of the derived uncertainties by analyzing the corresponding LOD prediction accuracy. All the analyses so far have been based on the architecture of MLP. In Fig. 11, however, we compare the MAE of MLP with that of LSTM. While LSTM is less accurate than MLP in study 1 by as much as 0.5%, it presents a 2% improvement on average compared to MLP in study 2. These values confirm that the role of model architecture on the uncertainties derived is marginal. This also implies that the epistemic uncertainties due to the model deficiencies are captured well enough by sampling a large number of candidates for the prior such that changing the architecture from MLP to LSTM does not fundamentally alter the results. Therefore, we proceed to follow the MLP architecture in the remaining analyses.

Fig. 11

Impact of the model architecture on the prediction accuracy of LOD using BaHaMAs algorithm for the determination of AAM and OAM uncertainties

4.7 Analyzing the impact of normalization choice

To analyze the influence of the bottleneck (i.e., normalization type) in the architecture of BaHaMAs, we compare the batch and layer normalization approaches (note that instance normalization in our problem is equivalent to batch normalization). The results are presented in Fig. 12. Similar to the model architecture, it appears that the role of normalization type on the uncertainties derived and the prediction accuracy of LOD is marginal. More precisely, however, layer normalization is on average 0.5% and 3% more accurate than the batch normalization, confirming the suitability of our choice of normalization type.

Fig. 12

Impact of the normalization type in BaHaMAs on the prediction accuracy of LOD

4.8 Comparison of BaHaMAs with VAEs and DEs as baselines

To compare BaHaMAs with alternative uncertainty quantification algorithms, we present the results of VAEs and DEs. Similar to BaHaMAs, VAEs use $Q=50000$ samples. For DE, 10 models are trained, as suggested in several studies (Kiani Shahvandi et al. 2023a, 2024a) to present robust analyses for the prediction of EOPs (in fact we also tested the impact of the number of DE models, but after 10 models we did not see significant improvement in the results, implying that the results presented here are robust). The results are presented in Fig. 13, indicating a higher accuracy of BaHaMAs. In study 1, the average improvement of using BaHaMAs instead of VAEs and DEs is 5% and 7%, respectively. On the other hand, in study 2 the corresponding values are 9% and 10%.

Fig. 13

Impact of the uncertainty algorithm on the prediction accuracy of LOD. The results of BaHaMAs, VAEs, and DEs are compared for study 1 and 2 in (a) and (b), respectively

4.9 Analyzing the impact of using AAM and OAM forecasts and comparison with independent predictions as baselines

Since it has been shown that AAM and OAM forecasts have significant impact on the prediction accuracy of LOD (Gou et al. 2023), we proceed to use 14-day-ahead forecasts provided by ETH Zurich, quantify their uncertainty, and use them in the prediction of LOD. As observed in Fig. 14(a), the inclusion of AAM and OAM forecasts with their uncertainties derived from BaHaMAs significantly improves the LOD prediction accuracy, on average by 30% and 31% for studies 1 and 2, respectively. In addition, to compare the prediction accuracy of our algorithm with an independent data provider, we use the LOD predictions of SYRTE (Bizouard et al. 2019). Since we have archived these operational predictions only since May 20, 2021, we only present the results for a part of the Test 2 time frame. The results are presented in Fig. 14(b). Comparisons with the predictions of SYRTE from May 2021 until the end of 2022 reveal that BaHaMAs predictions are on average 66% more accurate, demonstrating the high accuracy of our algorithm.

We also compare our results with the best result obtained in the second Earth Orientation Parameters Comparison Campaign (EOPPCC; Śliwińska et al. 2023, Śliwińska-Bronowicz et al. 2024), under the campaign conditions. The results are shown in Fig. 15. As it can bee seen, we can improve the prediction performance of LOD by 12% on average (we note that these results are before various filtering applied by EOPPCC). This further confirms that using the uncertainties derived by BaHaMAs in the prediction of LOD is advantageous.

Fig. 14

a Impact of including AAM and OAM forecasts on the prediction accuracy of LOD, when the uncertainties are determined using BaHaMAs. b Comparison of the prediction accuracy of our algorithm with the independent predictions provided by SYRTE. Note here that the results are based on the predictions in the range May 20, 2021 to the end of 2022

Fig. 15

Comparison of the prediction accuracy (in MAE) of LOD for the best results obtained during the second EOPPCC and those from our approach. The unit is ms

5 Conclusions

We present BaHaMAs, a method for quantifying the aleatoric uncertainty in time series using autoencoders based on Bayesian machine learning through Hamiltonian Monte Carlo sampling. In case a time series is provided without the information regarding uncertainty, BaHaMAs can be used to approximate the aleatoric uncertainty in the time series from itself by analyzing how predicable or stochastic the time series is. We apply the method to the AAM and OAM time series of GFZ and use the derived uncertainties alongside the AAM and OAM themselves for the short-term prediction of LOD. The prediction performance is improved by around 17% on average. Furthermore, BaHaMAs is more accurate than VAEs or DEs. Among the parameters that can impact the accuracy of BaHaMAs the most important is the sampling size (around 50000 is recommended). In terms of data, the most significant contribution to the improvement of prediction accuracy of LOD is from 14-day-ahead forecasts of AAM and OAM.

Prediction of LOD is important for a host of applications such as spacecraft navigation and orientation of deep space telescopes (Dobslaw and Dill 2018). As a result, the second EOPPCC (Śliwińska-Bronowicz et al. 2024) was organized to compare the prediction accuracy of different methods under operational settings. In the 30-day horizon our predictions were the most accurate in terms of MAE, providing us with incentive to enhance our methodology to further improve the LOD predictions. With BaHaMAs we have made a step toward this goal and hope to motivate other researchers to consider incorporating the uncertainty quantification in their methods as well. We note, however, that one of the main shortcomings of BaHaMAs is that there is no universal, independent approach to compare the results of BaHaMAs against. Therefore, for various applications different ways of comparison should be designed based on prior knowledge. One way, as we have shown in this paper, is to compare the data of different institutions provided that the modelling approach is the same for all data centers. Finally, the uncertainties derived by BaHaMAs are approximate and do not necessarily correspond exactly to the ’true’ yet unknown uncertainties. However, we derived conditions under which the uncertainties derived by BaHaMAs correspond to the actual uncertainties. Hence, these uncertainties are useful for the prediction purposes, as we have demonstrated in this paper.

Appendices

Appendix A

1.1 Proof of proposition 1

According to Jennrich (1969) and Wu (1981), an estimator ${\hat{\theta }}$ is said to be consistent (weakly or strongly) if the quantity $\frac{1}{N}D_N(\theta ,\theta ')$ uniformly converges to a continuous function $D(\theta ,\theta ')$, and $D(\theta ,\theta _0)=0\iff \theta =\theta _0$. D has the form as in Eq. (11):

$$\begin{aligned} \begin{aligned} D_N(\theta ,\theta ') = \sum _{i=1}^{N}\Big (f(X_i,\theta )-f(X_i,\theta ')\Big )^2 \end{aligned} \end{aligned}$$

(11)

Now, we note that f is an autoencoder and thus, we can write

$$\begin{gathered} f\left( {X_{i} ,\theta } \right) = X_{i} + \xi _{i} \hfill \\ f\left( {X_{i} ,\theta ^{\prime}} \right) = X_{i} + \eta _{i} \hfill \\ \end{gathered}$$

(12)

in which $\xi _i$ and $\eta _i$ are functions of $\theta$ and $\theta '$, as well as X. Putting Eq. (12) to (11) we get Eq. (13):

$$\begin{aligned} \begin{aligned} D_N(\theta ,\theta ')= \sum _{i=1}^{N}\Big (\xi _i-\eta _i\Big )^2 \end{aligned} \end{aligned}$$

(13)

As Wu (1981) states, the condition $\lim _{N\rightarrow \infty }D_N(\theta ,\theta ')\longrightarrow \infty$ should be satisfied. According to theorems on divergence of infinite series (see e.g., Bourchtein and Bourchtein (2022)) if we assume $\big |\xi _i-\eta _i\big |\sim N^{-p}$, then $D_N(\theta ,\theta ')\sim N^{-2p+1}$ and therefore $p\le 1$. On the other hand, $\frac{1}{N}D_N(\theta ,\theta ')\sim N^{-2p}$ should be convergent and therefore, $p>\frac{1}{2}$. Therefore, the $D_N(\theta ,\theta ')$ should in general scale as $N^{-p},~\frac{1}{2}<p\le 1$. Turning to second condition, if $\theta =\theta _0$ then $\xi _i=\eta _i$, implying $D_N(\theta ,\theta ')=0$ and therefore $D(\theta ,\theta ')=\lim _{N\rightarrow \infty }\frac{1}{N} D_N(\theta ,\theta ')\longrightarrow 0$. On the other hand, if $D(\theta ,\theta ')=0$, then since $D_N(\theta ,\theta ')>0$ we conclude that $\xi _i=\eta _i,~\forall i$. Considering that $\xi _i$ and $\eta _i$ are continuous functions in terms of $\theta$ and $\theta '$, the only possible way to satisfy $\xi _i=\eta _i,~\forall i$ is $\theta =\theta _0$.

Appendix B

Numerical values of MAE and improvement in Fig. 9

The numerical values of the MAE and improvement in Fig. 9 are given in the following Table 1.

Table 1 Numerical values of the MAE (in ms) and improvement (in percentage) related to Fig. 9