Evaluation of selected short-term predictions of UT1-UTC and LOD collected in the second earth orientation parameters prediction comparison campaign

Abstract

Advanced geodetic and astronomical tasks, such as precise positioning and navigation require forecasted Earth Orientation Parameters (EOP). The Second Earth Orientation Parameters Prediction Comparison Campaign (2nd EOP PCC) aims to compare various EOP forecast methods implemented by different institutes from all over the world. Here we focus on universal time (UT1-UTC) and Length-of-Day (LOD) predictions received in the period between September 1st, 2021 and May 29th, 2022. The forecasts are preliminarily evaluated against the EOP 14 C04 solution delivered by the International Earth Rotation and Reference System Service (IERS) by using the mean absolute error (MAE) as the prediction quality measure. Exemplarily, we compare forecasts from IERS delivered by U.S. Naval Observatory (USNO) and a selected campaign participant to assess the impact of both input data and computation methodology on predictions. We show that improper treatment of long-periodic ocean tides has severely degraded LOD forecasting until this issue has been brought to the attention of the participant during a meeting of the 2nd EOP PCC. We consider this as a good example for the benefit of the campaign to the overall scientific community by providing specific feedback to individual processing centres on deficits in their products, which lead to quick and effective adaptations. The lessons learned from this analysis could be applied to other EOP forecasting methods based on Effective Angular Momentum (EAM) predictions.

Graphical Abstract

Main text

Introduction

The five Earth Orientation Parameters (EOP), comprising nutation or celestial pole offsets (CPO), polar motion (PM), and differences between universal time and coordinated universal time (UT1-UTC; or its time-derivative Length-of-Day (LOD) change), describe the time-variable rotation of our planet and link the terrestrial and the celestial reference frames. EOP are regularly measured by modern space-geodetic techniques, such as very long baseline interferometry (VLBI, Nilsson et al. 2010, 2011, 2014), global navigation satellite systems (GNSS, Byram and Hackman 2012; Mireault et al. 1999; Zajdel et al. 2020), satellite laser ranging (SLR, Coulot et al. 2010), lunar laser ranging (LLR, Pavlov 2020), and Doppler orbitography and radiopositioning integrated by satellites (DORIS, Angermann et al. 2010; Moreaux et al. 2016). The current accuracy of EOP determination is about 0.05 mas (milliarcseconds), which corresponds to 1.5 mm of horizontal displacement on the Earth’s surface (Petit and Luzum 2010). Official EOP products are routinely delivered by the International Earth Rotation and Reference Systems Service (IERS) in the form of time series determined from a combination of different measurement techniques (Bizouard and Gambis 2009). The latest available version of EOP series from IERS is EOP 14 C04 (Bizouard et al. 2019). The Earth Orientation Center, established by the IERS and hosted at Paris Observatory, is responsible for EOP monitoring and the delivery of monthly and long-term EOP data, release of time dissemination (UT1-UTC), and leap second announcements (Bizouard et al. 2019; Gambis 2004; Gambis and Luzum 2011; IERS Annual Report 2020).

Work on combining independent Earth orientation measurements using space-geodetic techniques is also carried out by scientists at the Jet Propulsion Laboratory (JPL), who exploit the Kalman filter to develop combined SPACE2018, COMB2018, and POLE2018 solutions, which are used to support the tracking and navigation of spacecrafts (Gross 2000; Ratcliff and Gross 2019). More recently, the European Space Agency (ESA) has also been working towards an independent capacity to routinely estimate and predict EOP (Bruni et al. 2021) from consistently processed geodetic input products (Schoenemann et al. 2020).

Accurate EOP are required for a number of operational tasks, including communication with spacecraft operating in the solar system, the positioning of astronomical instruments on the ground, and the positioning and navigation capabilities on the Earth’s surface by GNSS, such as GPS, Galileo, GLONASS, and BeiDou. For all those tasks, EOP information must be available in real time. Due to the delayed availability of various geodetic observations and external correction models, the final (and most precise) EOP solutions are only delivered with a latency of several weeks. It is therefore important to forecast EOP for a certain period of time in the future based on preliminary processed rapid solutions of geodetic data to ensure that valid EOP data are available in any moment for a certain navigation or communication task.

A first comprehensive assessment of different EOP prediction capabilities has been performed in the frame of the EOP Prediction Comparison Campaign (EOP PCC), prepared by Vienna University of Technology and Centrum Badań Kosmicznych Polskiej Akademii Nauk (CBK PAN) in Warsaw (Kalarus et al. 2010). Since that time, considerable progress has been made in terms of processing geodetic observations (Bizouard et al. 2019; Karbon et al. 2017; Nilsson et al. 2014). The importance of geophysical angular momentum data for EOP predictions became evident, and knowledge about the role of geophysical fluid layers in Earth rotation disturbances has increased (Dill et al. 2013). Delays in providing geodetic and geophysical data have been also reduced drastically. Several institutions from different countries are routinely processing and predicting EOP using different forecasting methods and diverse observational datasets as input (Akyilmaz et al. 2011, Belda et al. 2018, Chin et al. 2004, Dill et al. 2019, Kosek et al. 2008, Modiri et al. 2018, Nastula et al. 2020, Schuh et al. 2002, Shen et al. 2017, Stamatakos et al. 2011, Wang et al. 2018, Xu et al. 2012). Scientists are constantly seeking new forecasting methods that offer the best possible accuracy. The strong interest in EOP forecasting ultimately led to the establishment of the 2nd EOP PCC by a working group of the IERS. The campaign is again run by CBK PAN with support from the German Research Centre for Geosciences (GFZ). It is an international initiative with 28 registered institutions from 9 different countries, involving over 60 people who regularly deliver predictions based on 56 different methods (Table 1). Forecasts of all EOP are submitted every Wednesday and are evaluated once the geodetic final EOP observations from the forecasted period eventually become available. More detailed information can be found on the campaign website eoppcc.cbk.waw.pl (accessed 12.10.2022).

This paper provides insight into the early results obtained so far by one specific participant in the campaign for UT1-UTC and LOD forecasts. We rely on 39 submissions during the period of September 1st, 2021 to May 29th, 2022. All forecasts are evaluated against the IERS 14 C04 solution (Bizouard et al. 2019). With the selected example, we aim to outline the benefits of this activity for both individual institutions engaged in EOP prediction as well as the international user community that relies on high-quality information about the orientation of the solid Earth with respect to inertial space.

The mean absolute error (MAE) was used to conduct a quantitative comparison of predictions:

$$MA{E}_{i}=\frac{1}{{n}_{p}}{\sum }_{j=1}^{{n}_{p}}\left|{x}_{i}^{obs}-{x}_{i,j}^{pred}\right|,i=\mathrm{1,2},..., I,$$

where ${n}_{p}$ is the number of valid prediction files submitted by a campaign participant under a single identifier, ${x}_{i}^{obs}$ is the EOP reference data for the $i$ th day of reference series, ${x}_{i,j}^{pred}$ is the value for the $i$ th day of the $j$ th prediction, and $I$

Table 1 Statistics of participants and methods in the 2nd EOP PCC

is the maximum forecast horizon (i.e. the time period predicted into the future) that is used to compute the MAE, which equals to 10 days in this article. MAE₁ thus characterises the accuracy of an EOP prediction for just one day into the future, whereas MAE₇ represents the forecast capabilities for a full week. As commonly known from numerical weather prediction, forecasts for seven days ahead of time can be expected to be much worse than for the next day, implying that MAE₁ is usually smaller than MAE₇ with a steady grow for any forecast horizon in between.

Along with root mean square error (RMSE), MAE is the most frequently used parameter to assess the quality of EOP forecasts (e.g. Jin et al. 2021; Kiani Shahvandi and Soja 2022; Modiri et al. 2018, 2020). MAE was also the primary measure of prediction accuracy during the first EOP PCC (Kalarus et al. 2010). The calculation of both MAE and RMSE is based on taking the differences between observed and predicted values, but for the RMSE the differences will be squared. Consequently, large differences have a much greater influence on the total RMSE than the smaller deviations. This means that final RMSE values are much more sensitive to single large differences than MAE values. Therefore, MAE is recommended over the RMSE to assess the quality of deterministic forecasts (Willmott and Matsuura 2005).

Predictions of UT1-UTC and LOD

We focus in this article on two particular contributions to the campaign: (i) the official IERS predictions processed every day by the U.S. Naval Observatory (USNO; IERS Annual Report 2020) and (ii) predictions provided by GFZ which rely heavily on Effective Angular Momentum (EAM) forecasts (Dill et al. 2019).

IERS/USNO predictions of UT1-UTC are based on observations from VLBI, GNSS, and SLR as well as atmospheric angular momentum (AAM) data (Stamatakos et al. 2011). The AAM data used to predict UT1-UTC were obtained from a combination of the operational National Centers for Environmental Prediction (NCEP) and Navy Global Environment Model (NAVGEM) (IERS Annual Report 2020). AAM-based forecasts are used by USNO to determine the UT1-UTC predictions for a forecast horizon of up to 7.5 days into the future. For longer prediction horizon, the LOD excitations are combined smoothly with the longer-term UT1-UTC predictions. The longer-term UT1-UTC predictions are based on using differences between smoothed version of UT1 (with periodic seasonal and long period variations due to tides removed) and International Atomic Time (TAI), and then restoring the known effects in order to obtain the prediction of UT1-UTC (IERS Annual Report 2020). The IERS/USNO predictions received the ID = 200 assigned by the EOP PCC Campaign Office.

Predictions developed by GFZ are based the final EOP time series IERS 14 C04 and operational geophysical fluid excitation functions (including their 6-day forecasts): AAM, OAM (oceanic angular momentum), HAM (hydrological angular momentum), and SLAM (sea-level angular momentum). The EAM prediction, which is developed by GFZ, is then used to integrate it from the latest available initial EOP values (rapid EOP from Bulletin A) into 90-day EOP predictions with least-squares extrapolation and autoregressive modelling (Dill et al. 2019). The GFZ predictions are labelled with ID = 105 within the campaign.

In Table 2, we shortly summarise the prediction accuracy of UT1-UTC and LOD for GFZ and IERS/USNO in comparison with values for submissions from all participants. These values were computed for all predictions for which the final EOP reference data are already available. Those results indicate the better performance of UT1-UTC predictions processed by GFZ compared to the forecasts delivered by IERS/USNO or other institutes.

Table 2 Summary of UT1-UTC and LOD predictions accuracy from the differences between submission and reference series (IERS 14 C04). Values marked as Total are computed for submissions from all participants treated as one participant to show general performance of the forecasts

Fig. 1

Mean absolute error (MAE) for UT1-UTC predictions from GFZ (ID = 105) and the IERS/USNO (ID = 200) together with results for all other participants for forecast horizons between 0 and 10 days

For all available submissions of UT1-UTC, MAE values for forecast horizons between Day 0 and Day 10 are depicted in Fig. 1. In this plot, Day 0 corresponds to observed values at 00:00 UTC of the day of submission of any forecast, where rapidly processed geodetic data should be already available to the participants. A low MAE at day zero indicates that the participant made substantial efforts to utilise geodetic data available just shortly before the submission deadline. Day 10 refers to the error of a prediction of 10 days into the future after the submission was made. It is worth recalling that most predictions extend up to 90 days into the future.

Focusing initially on the results for Day 0, we note that the IERS/USNO forecasts almost perfectly matches the reference time series. For the GFZ submission, we note that the values for Day 0 deviate somewhat from the IERS 14 C04 solution, indicating that the geodetic information entering into those forecasts is not optimal. In fact, GFZ typically processes the EOP prediction already around 12:00 UTC as soon as the latest EAM forecast calculations have been completed, and therefore uses the latest IERS rapid solution available at that time. To further improve the GFZ forecasts, extra efforts to include low-latency geodetic data are strongly encouraged.

Moving on to forecasts for later days, we note a gradual increase in MAE for both IERS/USNO predictions and GFZ predictions, but this growth rate is smaller in case of GFZ. After forecast Day 5, MAE eventually becomes, in fact, smaller for predictions from GFZ and the gap between the two solutions is largest on Days 9 and 10. The results also show that the GFZ and IERS/USNO predictions perform better than most of forecasts processed by other campaign participants.

It is also evident from Fig. 1 that for the first six forecast days, the predictions developed by GFZ are characterised by one of the lowest MAE growth rates with an increase on the day of the forecast. This might be caused by the fact that in EOP forecasting GFZ uses not only final values of EAM derived from atmospheric analysis data constrained by observations but also on dedicated predicted time series for those EAM. GFZ currently utilises predictions for atmosphere, ocean, and land surface dynamics for up to six days into the future. Therefore, it may be beneficial for other campaign participants to use the EAM forecasts to reduce errors of their EOP predictions. Availability of such EAM forecasts for more days ahead could potentially improve EOP prediction accuracy for prediction horizons above 6 days. Nowadays, EAM predictions are processed also by the team at ETH Zurich, but again only for 6 days ahead (Kiani Shahvandi et al. 2022). Increasing the lead time of EAM predictions appears to be the most promising option to further improve the EOP prediction accuracies at time scales longer than a week. The results also confirm the very good quality of the IERS/USNO predictions for UT1-UTC, which are only outperformed by two competitors in the campaign for forecast horizon below four days and two other participants for forecasts between six and ten days.

The campaign also calls for submission of separate predictions for LOD despite the fact that it can be differentiated from UT1-UTC. This is because of the generally biased results for UT1-UTC derived from the data from GNSS systems, since linear drifts in the satellite constellation cannot be reliably separated from changes in the Earth’s rotational angle. LOD is unaffected; however, so it can be readily estimated from GNSS without the need for a consistent combination with VLBI.

When looking at the MAE of the GFZ submission for LOD (Fig. 2a), we note a prominent sinusoidal oscillation with a period of 7 days that is not apparent in any of the other submissions. This problem has been traced back to an incorrect treatment of effects from long-periodic ocean tides (see Bizouard et al. 2022), which are by convention part of the IERS rapid solution and consequently also need to be predicted. While correctly handled in the GFZ submission for UT1-UTC discussed above, the mistake in LOD predictions only became apparent during a meeting of the campaign participants on November 25th, 2021 and was swiftly corrected thereafter. The consequences of this error (and its eventual detection and correction) are also nicely visualised in Fig. 3 that presents differences between individual LOD predictions against the IERS 14 C04 solution. It can be seen from this plot that the prominent weekly oscillations completely vanish in early December 2021. The MAE plot redrawn for a limited period of time (between December 1st, 2021 and May 29th, 2022) demonstrates that an adequate prediction of LOD with inclusion of long-periodic ocean tides substantially reduced the MAE for the GFZ prediction (Fig. 2c). In contrast, for the period between September 1st and November 24th the oscillation in MAE is even bigger than in the case of the whole considered period (Fig. 2b). We consider this as a good example of the benefit of the campaign to provide specific feedback to an individual processing centre on deficits in its products, which lead (in this case) to very effective adaptations.

Fig. 2

Mean absolute error for LOD predictions from GFZ (ID = 105) together with results for all other participants for forecast horizons between 0 and 10 days: a calculated for a period between September 1st 2021 and May 29th, 2022, b calculated for a limited period between September 1st and December 1^st, 2021, and c calculated for a limited period between December 1st, 2021 and May 29th, 2022

Fig. 3

Differences of individual LOD predictions from GFZ (ID = 105) against the 14 C04 solution. Oscillatory patterns with periods of 14 days are related to long-periodic ocean tides that were not properly accounted for before December 1st, 2021 (MJD 59,549). Figure shows differences for predictions submitted between December 1st, 2021 and April 27th, 2022 to present a full 30 days ahead. X ticks represent the first submission day in a month

Summary and conclusions

The 2nd EOP PCC, under the auspices of IERS, commenced in September 2021 and run nominally until December 2022. The results from the previous EOP PCC reported in Kalarus et al. (2010) show that for the 29 months of the operational part of the campaign, MAE for the 10th day of prediction was between 0.60 and 1.40 ms for the UT1-UTC forecast and between 0.13 and 0.40 ms for the LOD forecast even after the exclusion of 2% of the submissions flagged as possible outliers. Nine months after the start of the second campaign, the corresponding values were determined as 0.36–3.13 ms for UT1-UTC and 0.07–0.28 ms for LOD, which nicely confirms the steep progress made in EOP forecasting during recent years.

Preliminary evaluation of both UT1-UTC and LOD with the use of IERS 14 C04 series as a reference confirms the very good quality of the IERS/USNO predictions, which can be wholeheartedly recommended for operational applications. It was also shown that the UT1-UTC and LOD predictions benefit from the use of EAM data and in particular its forecasts. Moreover, long-periodic ocean tides need to be treated properly, since those signals are conventionally part of the final EOP series. Nevertheless, our analysis also indicates room for improvement in individual aspects, like, e.g. the incorporation of rapidly processed geodetic data at the GFZ. Although it is too early to compare the results from the two campaigns quantitatively as we have obtained results for 9 months only so far, the outcomes clearly point to the substantial progress in EOP forecasting achieved during recent years. Those results are expected to become even more robust in the coming months over the course of the 2nd EOP PCC, when more observational data and additional predictions will become available.

During the course of the campaign, we will evaluate the quality of individual predictions also on seasonal or even annual time scales. The possible impact of the reference data for the estimated prediction accuracy will be also investigated in more detail and findings will be shared with all participants in dedicated workshops planned for the year 2023. In that way, the 2nd EOP PCC will hopefully contribute to even further improvements in EOP processing and prediction, which is central to the scope of the IERS.