Geomagnetic observatory monthly series, 1930 to 2010: empirical analysis and unmodeled signal estimation
 Jiaming Ou^{1, 3},
 Nicolas Gillet^{2}Email author and
 Aimin Du^{1, 3}
https://doi.org/10.1186/s406230140173z
© Ou et al.; licensee Springer. 2015
Received: 18 July 2014
Accepted: 15 December 2014
Published: 30 January 2015
Abstract
Groundbased magnetic observatory series are the main source of information for constructing timedependent spherical harmonic geomagnetic field models from subannual to pluridecadal time scales. Assessing the reliability of such models requires accurate estimation of the data errors. We propose an analysis of observatory monthly means over the period 1930 to 2010, where we sequentially isolate (i) a stochastic regression for the main field at every site, performed in the framework of Gaussian processes, (ii) a local fit to annual and semiannual signals, (iii) a month by month estimate of global, large lengthscale external and induced fields. We then estimate the unmodeled signal level (UMSL, which refers to the instrumental noise plus extra signals not captured by the above data treatment) from the standard deviation of the residuals to the sequential analysis. This may be used to estimate data error covariances in future field modeling studies. Mainly a function of the geomagnetic latitude, the UMSL is larger towards auroral regions and carries the temporal signature of solar activity. While the UMSL shows rather similar magnitudes in all three components in recent epochs (typically a few nT), a significant decrease is found in the downward component of the field around 1960, which correlates with the introduction of proton magnetometers. We detail the geographic distribution of the periodic signals and confirm the variation of their amplitude at pluridecadal time scales. From the spherical harmonic description of horizontal and vertical fields, we isolate the main patterns of the inducing field in Z. These are dominated by a zonal structure of degree 1 (and to a lesser extent, of degree 3) in dipole coordinates. We nevertheless isolate secondary, nonzonal sources that are most active during the 1960s and around 1990, periods of particularly large solar activity, denoting an unusual morphology of the inducing field.
Keywords
Background
Dissociating external (ionospheric, magnetospheric) and internal (induced, crustal, core dynamo) magnetic sources is a major limitation to the spherical harmonic global modeling of geomagnetic records (e.g. Olsen et al. 2010a). This has repercussions on studies concerning both core physics and external fields. Indeed, the recovery of core motions at periods shorter than a few years is hindered by the domination of external sources towards high frequencies, and the reconstruction of changes in the magnetospheric ring current over decadal periods (e.g. Gillet et al. 2013; McLeod 1996; Sabaka et al. 1997) is made ambiguous because of nonglobal coverage of observatory sites.
Observatory series are the only records that permit the analysis of the geomagnetic field from subannual to pluridecadal periods. Several global modeling avenues may be employed using such data. In a comprehensive approach, Sabaka et al. (2002, 2004) simultaneously inverted all sources from hourly values spanning 1962 onward. Alternatively, the complex signature from external fields is considered as contributing to the data error budget when recovering main field changes from the annual means series starting in 1840 (Jackson et al. 2000). Intermediate strategies considering a simple external field parameterization (Gillet et al. 2013), or a sequential elimination of the sources, may also be used.
However, assessing the robustness of the output models is made difficult by the crude estimates of the observation errors. Accurate magnitudes of the data error covariances are required in order to provide realistic a posteriori confidence levels for the field models. For instance, by allocating constant error bars on observatory annual means series, Jackson et al. (2000) and Gillet et al. (2013) have relatively underweighted the information contained in the most accurate data when building, respectively, the gufm1 and COVOBS field models. With such an approach, a posteriori uncertainties tend to be overestimated towards recent epochs (when the data are of better quality), and time changes in the variance of some unmodeled signals are ignored (when they should be accounted for through the data error covariance matrix).
One could, in principle, consider the best documented period (the satellite era) to characterize unmodeled sources at ancient epochs. However, this approach is limited by the existence of external field variations for periods longer than a decade. Furthermore, there is no consensus on the variance one should consider to characterize satellite data errors (Finlay et al. 2012; Lesur et al. 2010; Olsen et al. 2010b). In a situation where there exists a frequency overlap within the different sources (given the recent estimates of the mantle conductivity, there is possibly a signature in magnetic series of core processes at periods shorter than a year, see Velímski and Finlaý (2011)), it is tempting to analyze the observatory monthly mean (OMM) series to try to better isolate the several contributions at periods ranging from a few months to a few decades.
When allocating error variances to observatory series, residuals from smooth fits to individual series, or from an initial timedependent global field model prediction, have sometimes been considered. As an example, the generalized crossvalidation method was applied to individual observatory annual mean series (see Bloxham and Jackson 1992) for the construction of the gufm1 and COVOBS models, and Wardinski and Holme (2006) and Wardinski and Lesur (2012) calculated error crosscovariances between residuals from threecomponent OMM series in constructing the C ^{3} FM models from 1957 onward. This pragmatic avenue nevertheless implicitly relies on the modeling assumptions, through a regularization process involving free damping parameters and specific choices of temporal basis functions. Those govern the temporal smoothness through their differentiability properties and density and thus implicitly state the cutoff between the noise and the signal.
Using principal component analysis, Wardinski and Holme (2011) found large correlations between the residuals from a main field model and the Dst magnetic index. This led them to remove from the data series a statistical proxy for unmodeled signals of external origin. Such an approach is promising in the quest for detailed secular variation patterns at observatories. However, indices such as Dst do not necessarily represent well the external activity worldwide (especially at high latitudes). Furthermore, it may be biased at periods longer than a year because of the time changes of the Dst baseline (see for instance Lühr and Maus 2010; Olsen et al. 2005b). The alternative RCindex (Olsen et al. 2014) has recently been introduced to avoid this specific issue.
In an ideal situation, in the context of global spherical harmonic modeling, one should follow a comprehensive approach where all signal sources are simultaneously inverted for, in order to see the unmodeled signals vanish as much as possible. This is unfortunately not possible when looking back to the early observatory era before 1960, as common measurement protocols had not yet been adopted via the INTERMAGNET global network of magnetic observatories (e.g., Love and Chulliat 2013), and the measurement quality was poorer (e.g., Matzka et al. 2010). In this configuration, modeling errors have to be considered along with measurement uncertainties.

First, a stochastic fit for the main field signal is obtained separately for each observatory using a Gaussian process regression that includes our prior knowledge of the core field temporal crosscovariances.

Next, signals with annual and semiannual periods are fit to the residuals between the OMM series and the main field regression, performed separately for each observatory.

Finally, we perform a monthbymonth large lengthscale global inversion of the remaining residuals. This approximates the main external source together with its associated induced fields.
Using a monthly sampling rate, we have to consider the large signals of periods of 1 year and half a year. If these have been isolated in observatory series for more than a century (e.g., Chapman and Bartels 1940), their mechanisms are still under debate. For this reason and because of their complex geographic distribution, we must fit them separately from the global, large lengthscale external fields.
After presenting the OMM series used throughout this study (Section ‘Data’), we detail in Section ‘Methods’ the three steps of our sequential analysis. The several contributions to the OMM series are presented in Section ‘Results and discussion’, together with the variations of the UMSL in space and time. Finally, we summarize our analysis in Section ‘Conclusions’.
Data

Some series for which hourly means are available are not included in the Chulliat and Telali (2007) database (either because of lack of continuity, too low quality, or the presence of incompatibilities with the annual means provided by the WDC). Instead, we consider that those data may be helpful to infer some information on the core signal at ancient epochs and find it interesting to estimate their associated UMSL.

We recalculate the OMM in a slightly different way. In anticipation of a future purpose of main field modeling, we use only the midnight 3h data in order to avoid the diurnal variation from the ionospheric Sq current system, which is large (weak) in the local daytime (midnight). We are aware that induced fields, with amplitudes of a few nT, are associated with Sq variations (Olsen et al. 2005a), but these essentially consist of large lengthscale patterns that should be accounted for through the global large lengthscale fit. To retrieve the UMSL of the OMM, we thus do not consider the impact from Sq variations.
We paid much attention to the correction for discontinuities in the monthly series. Since a large number of these ‘jumps’ are neither documented nor integrated in the WDC archives, we follow the strategy of Chulliat and Telali (2007) to diagnose and fix the baseline change issues in the series of daily values of midnight to 3h means. It is unfortunately not possible to recognize all these offsets, especially the small ones, which will be hidden by the signature of natural rapid variations from external or induced origin. We thus consider that this type of undiscovered discontinuities contribute to the unmodeled signals.
List of the observatories used throughout this study
Observatory  Code  λ  ϕ  Period  Observatory  Code  λ  ϕ  Period 

Abinger  ABN  51.2  359.6  19251965  La Quiaca  LQA  22.1  294.4  19631981 
Abisko  ABK  68.4  18.8  19502010  Lanzhou  LZH  36.1  103.9  19802006 
Addis Ababa  AAE  9.0  38.8  19591995  Las Mesas  TEN  28.5  343.7  19611992 
Agincourt  AGN  43.8  280.7  19311970  Learmonth  LRM  22.2  114.1  19902010 
Alert  ALE  82.5  297.6  19622005  Leirvogur  LRV  64.2  338.3  19622010 
Alibag  ABG  18.6  72.9  19222010  Lerwick  LER  60.1  358.8  19252010 
Alice Springs  ASP  23.8  133.9  19912010  Livingston Island  LIV  62.7  299.6  19962010 
Alma Ata  AAA  43.2  76.9  19622010  Loparskaya  MMK  68.2  33.1  19601990 
Annamalainagar  ANN  11.4  79.7  19671994  Lovo  LOV  59.3  17.8  19302004 
Apia  API  13.8  188.2  19332010  Lunping  LNP  25.0  121.2  19802000 
Arctowski  ARC  62.2  301.5  19781995  Lvov  LVV  49.9  23.7  19572010 
Arti  ARS  56.4  58.6  19722010  MBour  MBO  14.4  343.0  19522010 
Ascension Island  ASC  7.9  345.6  19932010  Macquarie Island  MCQ  54.5  158.9  19572010 
Ashkhabad  ASH  37.9  58.1  19601990  Manhay  MAB  50.3  5.7  19952010 
Baker Lake  BLC  64.3  264.0  19512010  Manzhouli  MZL  49.6  117.4  19952007 
Bangui  BNG  4.3  18.6  19552010  Martin de Vivies  AMS  37.8  77.6  19812010 
Bar Gyora  BGY  31.7  35.1  19902010  Mawson  MAW  67.6  62.9  19632010 
Barrow  BRW  71.3  203.4  19642010  Meanook  MEA  54.6  246.7  19312010 
Bear Island  BJN  74.5  19.2  19872006  Memambetsu  MMB  43.9  144.2  19582010 
Beijing  BJI  40.0  116.2  19572005  Mirny  MIR  66.6  93.0  19561997 
Beijing Ming Tombs  BMT  40.3  116.2  19962007  Mizusawa  MIZ  39.1  141.2  19802010 
Belsk  BEL  51.8  20.8  19662010  Molodezhnaya  MOL  67.7  45.8  19671977 
Bereznyaki  KGD  49.8  73.1  19651976  Mould Bay  MBC  76.3  240.6  19621996 
Borok  BOX  58.1  38.2  19802010  Muntinlupa  MUT  14.4  121.0  19631972 
Boulder  BOU  40.1  254.8  19722010  Nagpur  NGP  21.1  79.0  19952009 
Brorfelde  BFE  55.6  11.7  19812009  Nagycenk  NCK  47.6  16.7  19922010 
Budkov  BDV  49.1  14.0  19942010  Narsarsuaq  NAQ  61.2  314.6  19682009 
Cambridge Bay  CBB  69.1  255.0  19722010  Newport  NEW  48.3  242.9  19662010 
Canberra  CNB  35.3  149.4  19812010  Niemegk  NGK  52.1  12.7  19012010 
Cape Chelyuskin  CCS  77.7  104.3  19561985  Novolazarevskaya  NVL  70.8  11.8  19611978 
Cape Wellen  CWE  66.2  190.2  19561987  Novosibirsk  NVS  54.8  83.2  19672010 
Casey  CSY  66.3  110.5  19892005  Nurmijarvi  NUR  60.5  24.7  19532009 
ChambonlaForet  CLF  48.0  2.3  19362010  Ottawa  OTT  45.4  284.4  19682010 
Changchun  CNH  43.8  125.3  19952007  Pamatai  PPT  17.6  210.4  19672010 
Changli  CHL  39.7  119.0  19952007  Panagjurishte  PAG  42.5  24.2  19632010 
Charters Towers  CTA  20.1  146.3  19902010  Paratunka  PET  53.0  158.2  19691995 
Cheltenham  CLH  8.7  283.2  19011956  Phuthuy  PHU  21.0  105.9  19962010 
Chengdu  CDP  31.0  103.7  19952007  Pilar  PIL  31.7  296.1  19411980 
Chongqing  COQ  29.4  106.6  19992006  Pleshenitzi  MNK  54.5  27.9  19611995 
Chichijima  CBI  27.1  142.2  19912010  Pondicherry  PND  11.9  79.9  19952008 
College  CMO  64.9  212.1  19482010  Port Alfred  CZT  46.4  51.9  19742009 
Dalian  DLG  39.0  121.5  20002007  Port Moresby  PMG  9.4  147.2  19571993 
Dallas  DAL  33.0  263.2  19641974  Port Stanley  PST  51.7  302.1  19942009 
De Bilt  DBN  52.1  5.2  19061938  PortauxFrancais  PAF  49.4  70.3  19572009 
Dedu  DED  48.6  126.1  19952005  PostedelaBaleine  PBQ  55.3  282.3  19842007 
Del Rio  DLR  29.5  259.1  19822009  Qaanaaq  THL  77.5  290.8  19472010 
Dikson  DIK  73.5  80.6  19571986  Qeqertarsuaq  GDH  69.3  306.5  19462009 
Dombas  DOB  62.1  9.1  19572009  Qianling  QIX  34.6  108.2  19952006 
Dourbes  DOU  50.1  4.6  19572010  Resolute Bay  RES  74.7  265.1  19562010 
Dumont d’Urville  DRV  66.7  140.0  19572010  Rude Skov  RSV  55.8  12.4  19271981 
Dusheti  TFS  42.1  44.7  19562001  Sable Island  SBL  43.9  300.0  19992010 
Dymer  KIV  50.7  30.3  19581991  St. Johns  STJ  47.6  307.3  19692010 
Ebro  EBR  40.8  0.5  19952010  San Fernando  SFS  36.7  354.1  19962010 
Eilat  ELT  29.7  34.9  19982010  San Juan  SJG  18.1  293.8  19212010 
Ekaterinburg  EKT  56.8  60.6  19011925  San PabloToledo  SPT  39.5  355.6  19962010 
Esashi  ESA  39.2  141.4  19972010  Sanae III  SNA  70.3  357.6  19611990 
Eskdalemuir  ESK  55.3  356.8  19112010  Scott Base  SBA  77.8  166.8  19572010 
Eyrewell  EYR  43.4  172.4  19802010  Sheshan  SSH  31.1  121.2  19322010 
Faraday Islands  AIA  65.2  295.7  19572010  Simosato  SSO  33.6  135.9  19571975 
Fort Churchill  FCC  58.8  265.9  19572010  Sitka  SIT  57.1  224.7  19042010 
Fredericksburg  FRD  38.2  282.6  19562010  Sodankyla  SOD  67.7  26.6  19132009 
Fresno  FRN  37.1  240.3  19822010  Stekolnyy  MGD  60.1  151.0  19661990 
Fuquene  FUQ  5.5  286.3  19552010  Stennis Space Centre  BSL  30.3  270.4  19862010 
Furstenfeldbruck  FUR  48.2  11.3  19402010  Stepanovka  ODE  46.8  30.9  19571991 
Glenlea  GLN  49.6  262.9  19841997  Surlari  SUA  44.7  26.3  19572009 
Gnangara  GNA  31.8  115.9  19572010  Tamanrasset  TAM  22.8  5.5  19532010 
Golmud  GLM  36.4  94.9  19952007  Tianshui  TSY  34.6  105.9  19952007 
Gornotayezhnaya  VLA  43.7  132.2  19571990  Tihany  THY  46.9  17.9  19572009 
Great Whale River  GWC  55.3  282.2  19651981  Tirunelveli  TIR  8.7  77.8  19992008 
Guam  GUA  13.6  144.9  19572010  Tonghai  THJ  24.0  102.7  19952007 
Guangzhou  GZH  23.1  113.3  19602009  Trelew  TRW  43.3  294.6  19572010 
Guimar  GUI  28.3  343.6  19932010  Trivandrum 2  TRD  8.5  76.9  19571999 
Guiyang  GYX  26.6  106.8  19952005  Tromso  TRO  69.7  18.9  19572009 
Halley Bay  HBA  75.5  333.4  19571967  Tsumeb  TSU  19.2  17.6  19642009 
Hangzhou  HZC  30.2  120.1  19952007  Tucson  TUC  32.2  249.3  19102010 
Hartebeesthoek  HBK  25.9  7.7  19722009  Ujjain  UJJ  23.2  75.8  19802003 
Hartland  HAD  51.0  355.5  19572010  Urumqi  WMQ  43.8  87.7  19952007 
Hatizyo  HTY  33.1  139.8  19862009  Val Joyeux  VLJ  48.8  2.0  19011936 
Heiss Island  HIS  80.6  58.0  19601970  Valentia  VAL  51.9  349.7  19572010 
Hel  HLP  54.6  18.8  19662010  Vassouras  VSS  22.4  316.3  19152010 
Hermanus  HER  34.4  19.2  19412010  Victoria  VIC  48.5  236.6  19642010 
Honolulu  HON  21.3  202.0  19052010  Vieques  VQS  18.1  294.5  19051924 
Hornsund  HRN  77.0  15.5  19782010  Visakhapatnam  VSK  17.7  83.3  19952009 
Huancayo  HUA  12.0  284.7  19212010  Voeikovo  LNN  59.9  30.7  19471988 
Hurbanovo  HRB  47.9  18.2  19062010  Vostok  VOS  78.4  106.9  19581979 
Iqaluit  IQA  63.8  291.5  19952010  Vysokaya Dubrava  SVD  56.7  61.1  19301974 
Irkutsk  IRT  52.2  104.4  19572010  Watheroo  WAT  30.3  115.9  19191959 
IstanbulKandilli  ISK  41.1  29.1  19502000  Wien Kobenzl  WIK  48.3  16.3  19572005 
Jaipur  JAI  26.9  75.8  1980209  Wingst  WNG  53.7  9.1  19422010 
Jinghai  JIH  38.9  116.9  19952007  Witteveen  WIT  52.8  6.7  19381987 
Juyongguan  JYG  39.8  98.2  19982007  Wuhan  WHN  30.5  114.6  19952007 
Kakadu  KDU  12.7  132.5  19952010  Yakutsk  YAK  62.0  129.7  19572009 
Kakioka  KAK  36.2  140.2  19132010  Yangi Bazar  TKT  41.3  69.6  19571981 
Kanoya  KNY  31.4  130.9  19582010  Yellowknife  YKC  62.5  245.5  19772010 
Kanozan  KNZ  35.3  140.0  19612010  Yinchuan  YCB  38.5  106.3  19952007 
Kiruna  KIR  67.8  20.4  20012010  Yongning  YON  22.8  108.5  19962006 
Kourou  KOU  5.2  307.3  19962010  Yuzhno Sakhalinsk  YSS  46.9  142.7  19571988 
Krasnaya Pakhra  MOS  55.5  37.3  19572005  Zaymishche  KZN  55.8  48.8  19641989 
L’Aquila  AQU  42.4  13.3  19602010 
Methods
Local stochastic regression of the main field
Here, the element in the ith row and jth column of the matrix C _{ pd }=E(p d ^{ T }), of size N _{ p }×N _{ d }, is C _{ pd } _{ ij }=(t _{ p } _{ i }−t _{ d } _{ j }); and similarly for C _{ dd }=E(d d ^{ T }), of size N _{ d }×N _{ d }, we have C _{ dd } _{ ij }=(t _{ d } _{ i }−t _{ d } _{ j }).
where ε(t) is a white noise process^{a}. We use the correlation time τ _{ c }=1,730 years and a typical standard deviation of σ=40,000 nT. This corresponds to the a priori function used for the dipole only series by Gillet et al. (2013)  see their Equations 14 to 16. Of course, observatory records contain more than the dipole signal. However, we wish here to keep the definition of (τ) as simple as possible, still allowing the predicted series to potentially display the same continuity properties at monthly periods as they do at interannual periods. We performed a test using a more complex a priori function (such as a sum of AR2 processes, to account for higher order spherical harmonic coefficients), and found that it did not significantly change the local estimate of residuals to the AR2 fit (this is because we consider all observatories separately, and not all together through a global model). We estimate, at all sites, the BLUE (1) separately for all three components of B. We note B _{ A R2} the stochastic regression of the observed series B. We discuss in Section ‘On the sensitivity of the residuals to the Gaussian process regression’ the sensitivity of the variances of the residuals (B−B _{ A R2}) to the a priori error variances entering C _{ ee }.
Local analysis of annual and semiannual signals
Annual and semiannual signals clearly appear in the OMM series and indices (e.g., Lyatsky and Tan 2003). This is illustrated by the peaks in the power spectral density shown in Figure 2. They essentially result from ionospheric and magnetospheric fields (plus their induced counterparts) and exist worldwide in observatory records (see for instance Wardinski and Mandea 2006). Their short latitudinal extent at high latitude nevertheless prevents them from projecting entirely onto the large lengthscale model estimated in Section ‘Spherical harmonic modeling of remaining large lengthscale fields’, which is why we treat them separately at this stage of our empirical analysis.
In the construction of comprehensive field models, periodic signals originating from ionospheric fields are accounted for through a fit of global functions in quasidipole coordinates, calibrated a priori in time on the solar radiation flux index F10.7 (see Sabaka et al. 2002). However, this index does not account for the whole modulation of periodic signals. Indeed, the underlying mechanisms responsible for annual variations are subject to discussion and may vary depending on the geographical site (Malin and Winch 1996). Malin and Isikara (1976) suggested that they are generated by seasonal changes in the morphology of the auroral electrojet and the ring current system and imply variations of conductivity in the ionosphere (see for instance Liu et al. 2009).
Variations with a period of half a year are related to a larger occurrence and intensity of geomagnetic storms and substorms at equinoxes than at solstices (e.g., Cliver et al. 2000). For a long time, semiannual variations have been linked to the largest geomagnetic activity, occurring when the southward component of interplanetary magnetic activity is at a maximum in geocentric solar magnetospheric coordinates (Russell and McPherron 1973). A more complex picture has emerged since then. The latter mechanism is now believed to explain only part of the observed signal, and a more important role is given to the equinoxial hypothesis (see for instance Cliver et al. 2000; Lyatsky et al. 2001), by which the angle between the Earth dipole axis and the solar wind direction governs the efficiency in the response of the magnetosphere to solar wind flow.
Given the complex origin of periodic signals, we favor an empirical approach where the amplitude and phase of the periodic signal are kept completely free, along the lines of the studies by Le Mouël et al. (2004a,b). These authors calculated the Fourier coefficients of annual and semiannual periodic signals from bandpass filtered observatory hourly means. Their analysis indicates that both the amplitude and phase of these signals are changing with time. A similar result was found from wavelet analysis of the semiannual signal in the aa magnetic index (Elias et al. 2011).
we obtain the slowly varying functions ϕ _{1}(t), ϕ _{2}(t), \(\hat {X}_{1}(t)\), and \(\hat {X}_{2}(t)\), with ω _{1}=ω _{2}/2=2π rad/year (we use similar notations for the Y and Z components). This defines the annual and semiannual signals B _{1}(t) and B _{2}(t), whose characteristics are presented in Section ‘Worldwide description of annual and semiannual signals’.
Spherical harmonic modeling of remaining large lengthscale fields
Some of the remaining residuals are due to resolvable external and induced magnetic fields at large lengthscales. We approximate them by means of global modeling, constructing a discrete set of monthly models. These are considered independent from one epoch to another. Indeed, the power spectrum observed for external contributions, excluding annual and semiannual peaks, is only slightly red (with a slope about −1, see Figure 2).
Below, \(h_{n}^{mc}\) and \(h_{n}^{ms}\) stand for the real and imaginary part of the complex coefficient \({h_{n}^{m}}\) (with similar notation for \(v_{n}^{mc}\) and \(v_{n}^{ms}\)). We will thus model X and Y data on the one hand, and Z data on the other, resorting to a leastsquares fit. We emphasize that in this context, what will be isolated is a combination of globalscale fields that may arise from sources both external (e.g., magnetospheric) and internal (e.g., induced or remaining core signals) to the observation sites.
with \({\sf H}_{h} = \left [{\sf A}_{h}^{T}{\sf C}_{h}^{1} {\sf A}_{h} + {\sf R}_{h}^{1}\right ]^{1}\) the covariance matrix defining the a posteriori error statistics of the model h (with similar definitions for v). From \(\hat {\textbf h}\) and \(\hat {\textbf v}\), we build the large lengthscale field vector B _{ LLS }.
The quantities entering the a priori data error and model error covariance matrices are a priori unknown. For the sake of simplicity, we consider uncorrelated errors with stationary statistics, i.e., \({\sf C}_{h}={e^{2}_{h}}{\sf I}_{h}\) at every epoch (I _{ h } stands for the identity matrix of rank N _{ x }+N _{ y }), with similar notation for C _{ v }. Of course, errors are probably increasing towards the past, when records were less accurate. However, we tested several values for \({e^{2}_{h}}\) and \({e^{2}_{v}}\) and found that within a reasonable range, the final estimate of B _{ LLS }, and then of the UMSL, was not significantly affected. The results presented in Sections ‘Main patterns of remaining external and induced fields’ to ‘The UMSL throughout 1930 to 2010’ are obtained using e _{ h }=e _{ v }=6 nT. In practice, we fix the truncation level to N=3, which results in 15 unknowns per epoch. This is much smaller than the number of observatories after 1960 but not necessarily in the first half of the twentieth century.
If we were retrieving B _{ LLS } from Equation 8 in geographic coordinates, the model crosscovariances would vary with time depending on the dipole orientation. Thus, in order to solve Equation 8 using stationary variance properties, we rotate at each epoch the forward equations in dipole coordinates (for which we use the COVOBS internal dipole field). Thus, we build matrices R _{ h } and R _{ v } from the variances of the coefficient series \({h_{n}^{m}}(t)\) and \({v_{n}^{m}}(t)\) in dipole coordinates, first obtained over the period 1960 to 2010 for which the model is weakly sensitive to the a priori matrices (we would underestimate the variances of the model parameters by considering the era before 1960, where fewer, less accurate data are available). Coefficients are then rotated back into geographic coordinates.
and K _{ v }(t) (with similar notation).
Results and discussion
On the sensitivity of the residuals to the Gaussian process regression
In this section, we test the sensitivity of the variance of the residuals δ B=B−B _{ A R2}, obtained using the BLUE (Section ‘Local stochastic regression of the main field’), to the a priori noise amplitude that enters the covariance matrix C _{ ee } in Equation 1. We first study the case of synthetic data, before we analyze the geophysical series.
Ratio between the true and retrieved noise magnitude when varying the prior noise variance
Prior / true noise variance ratio  

Case  1/100  1/10  1  10  100 
A  0.91  0.95  0.98  1.01  1.16 
B  0.97  0.98  0.99  1.00  1.04 
C _{ a }  0.94  0.97  0.99  1.00  1.00 
C _{ b }  0.94  0.97  0.99  1.01  1.11 
Dimensional r.m.s. of the residuals to an AR2 fit at several observatory sites
\(\sqrt {\eta }\)  (nT)  3.3  10.0  33.3  \(\sqrt {\eta }\)  (nT)  3.3  10.0  33.3 

X  9.3  10.1  11.9  X  8.1  8.6  9.6  
KAK  Y  2.5  2.9  4.3  HER  Y  1.8  2.6  4.5 
Z  7.3  8.2  10.4  Z  3.2  3.4  6.6  
X  8.1  8.4  9.1  X  13.7  14.3  15.0  
ESK  Y  4.1  4.5  5.3  SIT  Y  6.1  6.4  7.4 
Z  9.4  11.0  13.1  Z  13.5  14.9  16.4  
X  11.0  12.5  15.2  X  10.6  11.2  12.7  
ABG  Y  4.4  4.9  6.4  GUA  Y  1.9  2.2  3.5 
Z  8.7  9.7  13.0  Z  3.4  4.3  8.4 
Worldwide description of annual and semiannual signals
We also see decadal changes in the phase and amplitude of the annual signal. These variations vary depending on the location, the phase showing, for instance, shifts as large as a couple of months. Several origins have been proposed for annual changes, including local effects of induction in the ocean, temperature dependency of the soil magnetization around sensors (Mishima et al. 2013), or spurious temperature effects in the Z component before the introduction of fluxgate sensors (Le Mouël et al. 2004b). Some effects related to measurement issues cannot be discarded from early epochs. Indeed, Figure 6 indicates that annual signals display less dispersion among observatories towards more recent periods.
The annual signal is generally larger in X, and its amplitude strongly increases at latitudes above 60°. A secondary maximum clearly appears in X at midlatitudes. Also at midlatitudes, nonzonal contributions are observed in the X and Y components, which may be associated with regional induction effects, as suggested by Wardinski and Mandea (2006). We note some discrepancies between the latitudinal dependence of annual and semiannual signals (see for instance in X). This may indicate that the source fields responsible for annual changes are different from those proposed by Fujii and Schultz (2002).
Main patterns of remaining external and induced fields
Examples of the contributions from periodic and large lengthscale fields to the three components of observatory series are shown, from low to high latitudes, in Figures 3,4 and 5. In the horizontal components, modulations with the solar cycle are observed for the large lengthscale field at mid to low latitudes (more obvious in the X direction), and for periodic signals, in horizontal components at high latitudes. The modulation of periodic signals at longer time scales, highlighted in the previous section, also clearly appears.
The UMSL throughout 1930 to 2010
We finally focus on the evolution, in space and time, of the UMSL as defined by Equation 10. Two main sources contribute to the UMSL: instrumental noise and unmodeled regional signals. Since the number of available observatories becomes limited for retrieving largescale external coefficients from the very early twentieth century, we present our UMSL estimate from 1930 onward. Examples of the final residuals and of the obtained UMSL series are presented in Figures 3,4 and 5.
In several observatories (see for instance the series for Alibag or ChambonlaForêt in Figures 3 and 4), periodic signals in the Z component tend to decrease towards recent epochs, which coincides with a decrease of the UMSL in Z. A link to instrumental effects is suspected, such as the introduction of proton magnetometers for absolute measurements after 1960 (Turner et al. 2007) and of fluxgate sensors for vector variometry, as suggested by Le Mouël et al. (2004b). Indeed, measurement improvements have been recorded by Chulliat and Telali (2007) at this particular time at Alibag and ChambonlaForêt. This is also illustrated by Figure four in Gillet et al. (2010) and Figure fourteen in Gillet et al. (2009), where an important decrease is observed for the scatter in the difference between two Z series at nearby observatories (the difference removing the effect of external fields that are coherent at the two sites).
At latitudes higher than 60°, a significant reduction in the UMSL after 1960 is found for Z data (on average, from 10 to 6 nT). In the series of the horizontal components, the median UMSL is more stationary (around 7 and 4 nT for X and Y, respectively). The decrease observed on average before 1950 is essentially due to the fact that we only have access to a restricted group of sites from this period, which does not include the observatories that display the higher UMSL values later on. Note that some sites display a UMSL as large as 20 nT in X at periods of maximum solar activity, a value much larger than found for Y and Z at recent epochs.
Conclusions
 1.
A local, stochastic regression for the main field, constructed in the framework of Gaussian processes. We use an a priori function based on a −4 slope temporal power spectral density, observed at periods longer than 5 years and extrapolated for shorter periods. This method appears robust, in the sense that it is only weakly sensitive to the a priori data errors that enter the definition of the BLUE in Equation 1.
 2.
A local analysis of annual and semiannual signals. We retrieve their modulation over long time scales, previously put forward by Le Mouël et al. (2004a,b). We suspect that part of these slow variations in amplitude is instrumental in origin, especially in the Z component. The geographic distribution of the amplitude of periodic signals is essentially zonal in geomagnetic coordinates, presenting a sharp gradient in the auroral zone. We see only a little longitudinal dependence, possibly due to regional induction effects.
 3.
A global spherical harmonic analysis of the remaining large lengthscale fields originating from external and induced sources. In order to avoid the ambiguity between internal and external sources, we fit horizontal and vertical components separately, before reconstructing the internal and external coefficients. We then describe the main patterns of the inducing field in Z, showing a nonnegligible role of components that are not carried by \({P_{1}^{0}}\). We observe an unusual morphology of this field in the 1960s and around 1990, periods of particularly intense solar activity.
We end up with the final residual series, from which we define the UMSL. At recent epochs, it is mainly a function of the geomagnetic colatitude, and its magnitude is rather similar for all three components (at least at mid to low latitudes). This reflects the standards required to enter the INTERMAGNET network. If some UMSL variations denote changes in the quality of the instrumentation, as clearly seen in the Z component around 1960, part of the UMSL fluctuations is related to the solar activity. This highlights the difficulty of modeling globally the complex magnetic fields at high latitudes from monthly values. A strategy similar to that presented here could be followed for hourly values if one were to also model Sq variations (see Stening and Winch 2013). This would not only have implications for the analysis of annual and semiannual signals; indeed, it would also require accounting for periodic fields at higher frequencies (see Love and Rigler 2014).
The UMSL could enter the data error covariance matrix in future field modeling studies if one wishes to invert jointly the internal, induced, and external signals at periods shorter than a few years. Indeed, modeling internal and external coefficients through the \(\tilde {h}_{n}^{m}\) and \(\tilde {v}_{n}^{m}\) associated, respectively, with horizontal and vertical data, it is possible to constrain the mantle conductivity through C responses (Olsen 1999). Furthermore, it seems possible that some core signals may reach the Earth’s surface at short periods (Velímský and Finlay 2011), making the joint inversion of core and induced signals relevant. Since the signature of core flows are relatively weak at short periods, accurate estimates of error covariances are required to assess the reliability of the reconstructed motions. In this situation, it may be of importance to consider the UMSL time changes at every site, since accurate error statistics are required to best extract the information contained in OMM series. For this purpose, one may consider the strategy put forward by Haines (1993) when building data error covariance matrices for differentiated series.
Endnote
^{a} Note that the SDE (3) differs from that defined by Equation (12) in (Gillet et al. 2013). Both SDEs define processes with the same autocovariance function (2), but their Equation (12), contrary to our Equation (3), corresponds to a nonstationary process. This does not affect their results that have been computed from the covariance function and not the SDE.
Declarations
Acknowledgements
We thank the national institutions that support ground magnetic observatories and INTERMAGNET for promoting high standards of practice and making the data available. Jiaming Ou’s two visits to France were financed by the Institute of Geology and Geophysics, Chinese Academy of Sciences. Discussions with Dominique Jault, Nils Olsen, Chris Finlay, and Jurgen Matzka have helped us throughout the evolution of the present study. We thank the two anonymous referees, whose comments helped improve the quality of the manuscript. This work was supported in part by the French Centre National d’Etudes Spatiales (CNES) for the preparation of the Swarm mission of ESA. This work has been partially supported by the French ‘Agence Nationale de la Recherche’ under grant ANR11BS56011. This work is also supported by the National Basic Research Program of China (2014CB845903 and 2012CB825604) and the National Natural Science Foundation of China (41174122, 41031066, 41104093).
Authors’ Affiliations
References
 Bloxham, J, Jackson A (1992) Timedependent mapping of the magnetic field at the coremantle boundary. J Geophys Res 97: 19,537–63.View ArticleGoogle Scholar
 Chapman, S, Bartels J (1940) Geomagnetism, Vol. 2. Clarendon Press, Oxford.Google Scholar
 Chulliat, A, Telali K (2007) World monthly means database project. Publs Inst Geophys Pol Acad ScC99(398): 537–552.Google Scholar
 Cliver, E, Kamide Y, Ling A (2000) Mountains versus valleys: semiannual variation of geomagnetic activity. J Geophys Res 105(A2): 2413–2424.View ArticleGoogle Scholar
 Currie, RG (1968) Geomagnetic spectrum of internal origin and lower mantle conductivity. J Geophys Res 73: 2779–2786.View ArticleGoogle Scholar
 De Santis, A, Barraclough D, Tozzi R (2003) Spatial and temporal spectra of the geomagnetic field and their scaling properties. Phys Earth Planet Int 135(2): 125–134.View ArticleGoogle Scholar
 Elias, AG, Silbergleit VM, de Gonzalez ALC (2011) Longterm variation of the semiannual amplitude in the aa index. J Atmos Sol Terr Phys 73: 1492–1499.View ArticleGoogle Scholar
 Finlay, CC, Jackson A, Gillet N, Olsen N (2012) Core surface magnetic field evolution 2000–2010. Geophys J Int 189: 761–781. doi:10.1111/j.1365246X.2012.05395.x.View ArticleGoogle Scholar
 Fujii, I, Schultz A (2002) The 3d electromagnetic response of the earth to ring current and auroral oval excitation. Geophys J Int 151(3): 689–709.View ArticleGoogle Scholar
 Gillet, N, Pais MA, Jault D (2009) Ensemble inversion of timedependent core flow models. Geochem Geophys Geosyst 10(6): Q06,004. doi:10.1029/2008GC002,290.View ArticleGoogle Scholar
 Gillet, N, Lesur V, Olsen N (2010) Geomagnetic core field secular variation models. Space Sci Rev 155: 129–145. doi:10.1007/s11,21400995866.View ArticleGoogle Scholar
 Gillet, N, Jault D, Finlay CC, Olsen N (2013) Stochastic modelling of the earth’s magnetic field: inversion for covariances over the observatory era. Geochem Geophys Geosyst 14(4): 766–786. doi:10.1029/2012GC004355.Google Scholar
 Haines, GV (1993) Modelling geomagnetic secular variation by mainfield differences. Geophys J Int 114(3): 490–500. doi:10.1111/j.1365246X.1993.tb06982.x.View ArticleGoogle Scholar
 Hulot, G, Olsen N, Sabaka TJ (2007) The present field In: In: Kono M, Schubert G (eds) Treatise in Geophysics, Geomagnetism, 33–75.. Elsevier, New York.Google Scholar
 Jackson, A, Jonkers ART, Walker MR (2000) Four centuries of geomagnetic secular variation from historical records. Phil Trans R Soc Lond A 358: 957–990.View ArticleGoogle Scholar
 Le Mouël, J, Blanter E, Chulliat A, Shnirman M (2004a) On the semiannual and annual variations of geomagnetic activity and components. Ann Geophys 22: 3583–3588.View ArticleGoogle Scholar
 Le Mouël, J, Blanter E, Shnirman M (2004b) The sixmonth line in geomagnetic long series. Ann Geophys 22: 985–992.View ArticleGoogle Scholar
 Lesur, V, Wardinski I, Asari S, Minchev B, Mandea M (2010) Modelling the Earth’s core magnetic field under flow constraints. Earth Planets Space 62: 503–516.View ArticleGoogle Scholar
 Liu, L, Zhao B, Wan W, Ning B, Zhang ML, He M (2009) Seasonal variations of the ionospheric electron densities retrieved from constellation observing system for meteorology, ionosphere, and climate mission radio occultation measurements. J Geophys Res: Space Phys 114(A02302). doi:10.1029/2008JA013819.Google Scholar
 Love, JJ, Chulliat A (2013) An international network of magnetic observatories. Eos, Trans Am Geophys Union 94(42): 373–374.View ArticleGoogle Scholar
 Love, JJ, Rigler EJ (2014) The magnetic tides of Honolulu. Geophys J Int 197(3): 1335–1353.View ArticleGoogle Scholar
 Lühr, H, Maus S (2010) Solar cycle dependence of quiettime magnetospheric currents and a model of their nearEarth magnetic field. Earth Planets Space 62: 843–848.View ArticleGoogle Scholar
 Lyatsky, W, Tan A (2003) Latitudinal effect in semiannual variation of geomagnetic activity. J Geophys Res: Space Phys 108(A5): 1978–2012.View ArticleGoogle Scholar
 Lyatsky, W, Newell P, Hamza A (2001) Solar illumination as cause of the equinoctial preference for geomagnetic activity. Geophys Res Lett 28(12): 2353–2356.View ArticleGoogle Scholar
 Malin, S, Isikara AM (1976) Annual variation of the geomagnetic field. Geophys J R Astron Soc 47(3): 445–457.View ArticleGoogle Scholar
 Malin, S, Winch D (1996) Annual variation of the geomagnetic field. Geophys J Int 124(1): 170–174.View ArticleGoogle Scholar
 Matzka, J, Chulliat A, Mandea M, Finlay C, Qamili E (2010) Geomagnetic observations for main field studies: from ground to space. Space Sci Rev 155: 29–64. doi:10.1007/s1121401096934.View ArticleGoogle Scholar
 McLeod, MG (1996) Spatial and temporal power spectra of the geomagnetic field. J Geophys Res 101: 2745–2763.View ArticleGoogle Scholar
 Menvielle, M, Berthelier A (1991) The kderived planetary indices: description and availability. Rev Geophys 29(3): 415–432.View ArticleGoogle Scholar
 Mishima, T, Owada T, Moriyama T, Ishida N, Takahashi K, Nagamachi S, Yoshitake Y, Minamoto Y, Muromatsu F, Toyodome S (2013) Relevance of magnetic properties of soil in the magnetic observatories to geomagnetic observation. Earth Planets Space 65(4): 337–342.View ArticleGoogle Scholar
 Olsen, N (1998) The electrical conductivity of the mantle beneath Europe derived from cresponses from 3 to 720 hr. Geophys J Int 133(2): 298–308.View ArticleGoogle Scholar
 Olsen, N (1999) Longperiod (30 days–1 year) electromagnetic sounding and the electrical conductivity of the lower mantle beneath Europe. Geophys J Int 138(1): 179–187.View ArticleGoogle Scholar
 Olsen, N (2002) A model of the geomagnetic field and its secular variation for epoch 2000 estimated from ørsted data. Geophys J Int 149(2): 454–462.View ArticleGoogle Scholar
 Olsen, N, Lowes F, Sabaka TJ (2005a) Ionospheric and induced field leakage in geomagnetic field models, and derivation of candidate models for DGRF 1995 and DGRF 2000. Earth Planets Space 57: 1191–1196.View ArticleGoogle Scholar
 Olsen, N, Sabaka TJ, Lowes F (2005b) New parameterization of external and induced fields in geomagnetic field modeling, and a candidate model for igrf 2005. Earth Planets Space 57(12): 1141–1149.View ArticleGoogle Scholar
 Olsen, N, Glassmeier KH, Jia X (2010a) Separation of the magnetic field into external and internal parts. Space Sci Rev 152(14): 135–157.View ArticleGoogle Scholar
 Olsen, N, Mandea M, Sabaka T (2010b) TøffnerClausen L. The CHAOS3 geomagnetic field model and candidates for the 11th generation IGRF. Earth Planets Space 62: 719–727.Google Scholar
 Olsen, N, Luhr H, Finlay CC, Sabaka TJ, Michaelis I, Rauberg J (2014) TøffnerClausen, L. The CHAOS4 geomagnetic field model. Geophys J Int 197(2): 815–827.Google Scholar
 Rasmussen, CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge.Google Scholar
 Russell, C, McPherron R (1973) Semiannual variation of geomagnetic activity. J Geophys Res 78(1): 92–108.View ArticleGoogle Scholar
 Sabaka, T, Langel R, Baldwin R, Conrad J (1997) The geomagnetic field 19001995, including the largescale field from magnetospheric sources, and the NASA candidate models for the 1995 revision of the IGRF. J Geomag Geoelectr 49: 157–206.View ArticleGoogle Scholar
 Sabaka, TJ, Olsen N, Langel RA (2002) A comprehensive model of the quiettime, nearearth magnetic field: phase 3. Geophys J Int 151(1): 32–68. doi:10.1046/j.1365246X.2002.01774.x.View ArticleGoogle Scholar
 Sabaka, TJ, Olsen N, Purucker ME (2004) Extending comprehensive models of the Earth’s magnetic field with Oersted and CHAMP data. Geophys J Int 159: 521–547.View ArticleGoogle Scholar
 Stening, R, Winch D (2013) The ionospheric sq current system obtained by spherical harmonic analysis. J Geophys Res: Space Phys 118(3): 1288–1297.View ArticleGoogle Scholar
 Turner, GM, Rasson JL, Reeves CV (2007) Observation and measurement techniques In: Kono M, Schubert G (eds) Treatise in Geophysics, 93–146, Geomagnetism.Google Scholar
 Velímský, J, Finlay CC (2011) Effect of a metallic core on transient geomagnetic induction. Geochem Geophys Geosyst 12(5): Q05,011. doi:10.1029/2011GC003557.View ArticleGoogle Scholar
 Wardinski, I, Holme R (2006) A timedependent model of the Earth’s magnetic field and its secular variation for the period 1980–2000. J Geophys Res 111. doi:10.1029/2006JB004,401.Google Scholar
 Wardinski, I, Holme R (2011) Signal from noise in geomagnetic field modelling: denoising data for secular variation studies. Geophys J Int 185: 653–662.View ArticleGoogle Scholar
 Wardinski, I, Lesur V (2012) An extended version of the C ^{3}FM geomagnetic field model – application of a continuous frozenflux constraint. Geophys J Int 189: 1409–1429. doi:10.1111/j.1365246X.2012.05384.x.View ArticleGoogle Scholar
 Wardinski, I, Mandea M (2006) Annual and semiannual variations of the geomagnetic field components analysed by the multitaper method. Earth Planes Space 58(6): 785.View ArticleGoogle Scholar
 Yaglom, AM (1962) An introduction to the theory of stationary random functions. PrenticeHall, Upper Saddle River.Google Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.