Skip to main content

Volume 65 Supplement 11

Swarm Science Data Processing and Products

Rapid modelling of the large-scale magnetospheric field from Swarm satellite data

Abstract

As part of the European Space Agency (ESA) Swarm mission, ESA has commissioned an independent scientific consortium known as the Swarm satellite Constellation Application and Research Facility (SCARF) to develop and operate the Level 2 Processor (L2PS). Its purpose is to derive high quality scientific products from the mission’s data. One such product is the Fast-Track Magnetospheric Model (FTMM), which is a model of the large scale vector magnetospheric field and its induced counterpart. This model is generated once per satellite orbit, in near real-time by a robust, autonomous algorithm. Its intended use is similar to that of the Disturbance storm time Index (Dst): characterising the rapidly varying magnetospheric field, as an input to other global field models, and for the space weather community. In this paper we describe in detail the FTMM algorithm and assess its ability to recover the magnetospheric component from the consortium’s test satellite data set as well as real data from the CHAMP satellite.

1. Introduction

Swarm is a European Space Agency (ESA) mission consisting of three low Earth-orbiting satellites designed to measure the global magnetic and electric fields with greater accuracy than has previously been achieved (Friis-Christensen et al., 2006). Two satellites begin the mission at an altitude of 490 km in near identical orbits, whilst the third orbits at a higher altitude, initially 550 km. Their altitudes will descend over the mission, most quickly for the lower pair. The orbit planes of the two lower satellites drift relative to the upper satellite over the course of the mission, achieving local time separation that will improve the spatiotemporal sensitivity of the data. The proximity of the lower two satellites ’ orbits will improve spatial sensitivity.

All three satellites are equipped with a vector field magnetometer (VFM) on a fixed optical bench with three star cameras to determine its orientation, and a low noise absolute scalar magnetometer isolated at the end of a long boom to provide a reliable reference for the vector magnetometer.

The raw satellite measurements are processed by ESA’s Payload Data Ground Segment (PDGS) to produce the so-called ‘Level 1b ’ data, which represent the best estimate of the magnetic field at each point in space and time in both the VFM’s reference frame and the geographic North, East, Centre (NEC) frame. The Level 1b products are released by ESA to the scientific community.

In addition to the Level 1b data, ESA has commissioned a consortium of scientific experts to produce the so-called ‘Level 2 ’ products, which include magnetic field models to act as standards for the general community. The Swarm Satellite Constellation Application and Research Facility (SCARF) is a consortium of six European institutes with acknowledged capability in these areas (Olsen et al., 2013).

In this paper we describe the ESA Swarm Level 2 Processing System’s (L2PS’s) Fast-Track Magnetospheric Model (FTMM) and processing chain. The term ‘FastTrack ’ is used to denote products of the L2PS that emphasise rapid production. In the case of the FTMM, the objective is to rapidly produce a model of the magnetospheric field and its induced counterpart at the largest spatial scales. This is in contrast to the Comprehensive Inversion’s Mag-netospheric Model (CIMM) (Sabaka et al., 2013), which determines the magnetospheric field to a higher resolution than the FTMM but typically does so as part of that processing chain’s six to twelve month release cycle.

In some respects the FTMM is similar to the Disturbance storm time Index (Dst) produced by Kyoto University 1 Footnote 1 and the Vector Magnetic Disturbance Index (VMD; Thomson and Lesur, 2007). However, unlike these indices it combines vector information with rapid production. In addition, the FTMM has a baseline determined by the subtraction of non-magnetospheric sources using models of the core, lithospheric, and ionospheric fields. As a result, the FTMM will include components of the magnetospheric field that cannot be easily distinguished from the local unwanted core, crustal and ionospheric fields in the observatory-based Dst and VMD indices. For these indices the unwanted fields are removed by subtracting polynomials, i.e. non-potential models of the unwanted fields.

The FTMM is intended to be of use to other processing chains within the L2PS that cannot wait for the CIMM to be produced. However, it should also to be of benefit to the wider community—for example, as an index of geomagnetic disturbance for space weather studies and data selection. The delivery of relatively high cadence models of the magnetospheric and induced fields should be useful in induction studies to better understand mantle conductivity and as a tool for understanding the temporal behaviour of the large-scale magnetosphere.

The specification of the FTMM requires it to be provided daily for the duration of the Swarm mission. To meet this requirement, its processing chain must be robust and, to a large extent, automated with a level of validation sufficient to ensure the quality of the product. However, some tasks still require scientific expertise in, for example, retrospectively assessing the model’s quality, deciding on which core, lithospheric, and ionospheric models to use, as well as resolving any unforeseen technical problems. For these reasons there is an important role for a ‘Scientist In the Loop ’ (SIL).

In Section 2 we describe in detail the FTMM processing algorithm. In Section 3 we show test results using synthetic Swarm satellite data for different configurations of the algorithm and also verify some results using real CHAMP satellite data. Finally, in Section 4 we summarise the results of these tests and their implications for the FTMM.

2. Algorithm and Operation

The magnetospheric field is dominated by the magnetopause, and ring- and tail-currents that are most efficiently described in the Sun-synchronous Solar Magnetic (SM) and Geocentric Solar Magnetospheric (GSM) frames (Lühr and Maus, 2010). To a good approximation, the fields observed at satellite altitudes from these external sources can be represented by a uniform magnetic field — a degree 1 external source in the spherical harmonic representation. Time variations of these fields as well as the rotation of the Earth relative to any Sun-synchronous frame will induce currents in the conducting crust and mantle producing corresponding internal sources of the field. The true conductivity structure of the Earth will result in induced sources at all spatial scales (spherical harmonic degrees) even for a uniform (external) magnetospheric field. Such three-dimensional conductivity models are used for the CIMM but for the FTMM, whose purpose is to rapidly and robustly model the dominant, largest scale terms, the induced field’s relationship with the inducing external sources can be approximated by a conductivity model that varies only with radius, in a similar manner to that used to separate Dst into its external (Est) and induced internal (Ist) components (Maus and Weidelt, 2004; Olsen et al., 2005).

The FTMM algorithm is based on a concept of a vector magnetospheric field model developed in Thomson and Lesur (2007) and Lesur et al. (2008). When expressed in any Cartesian reference frame, it can be shown that the magnetic field from degree 1 external spherical harmonic terms is uniform in space (Lesur et al., 2008). Averages of those field components over one orbit are easily related to the Gauss coefficients. Integrating over a complete orbit also filters out some fields from internal sources but in practice, due to data selection, all the internal and external sources must be separated after the average is taken. The resulting algorithm is robust and produces a set of internal and external coefficients every 90 minutes. A flow chart of the FTMM processing chain is shown in Fig. 1.

Fig. 1.
figure 1

The Swarm L2PS FTMM processing chain. The items feeding into the algorithm from the top are input data and models. Names containing: ‘AUX ’ are produced externally to the L2PS; ‘IGR’, ‘COR’, and ‘MCO ’ are core field models; ‘LIT ’ and ‘MLI ’ are lithospheric field models, and ‘MIO SHA 2C ’ is an ionospheric field model. The inputs with ‘F10’, ‘DST’, ‘KP’, and ‘IMF ’ are the F10.7 solar radio flux, Dst, Kp, and interplanetary magnetic field indices, respectively. ‘AUX OBS 2 ’ is observatory data and ‘MAGx LR 1B ’ is the Level 1b 1 Hz Swarm satellite data. ‘CM4 ’ is the Comprehensive Model version 4. The processing steps are described in Sections 2.1 to 2.11. The output of the chain is the FTMM, ‘MMA SHA 2F’.

New Level 1b satellite magnetic data, MAGx LR 1B in the ESA naming convention (MAGnetic data, Low Rate, level 1B), will be made available daily by the PDGS with a delay of no more than three days. The software controlling the processing chain will regularly check for new data and when available pull this over the internet from the PDGS before initiating the algorithm (Fig. 1).

Since the FTMM is a large scale (degree 1) model, the full 1 Hz time resolution is not required and would unnecessarily increase its computation time. The Level 1b data is therefore sub-sampled every 40 s, which keeps the processing time within acceptable limits without compromising the model quality (see Section 3.5).

The software will use the most appropriate input models available at the time, e.g. the lithospheric field model from the Comprehensive Inversion chain (Sabaka et al., 2013) or the corresponding Dedicated Inversion model (Thébault et al., 2013). Also used are core field model(s) (Rother et al., 2013; Sabaka et al., 2013), an ionospheric field model (Chulliat et al., 2013; Sabaka et al., 2013), and auxiliary data such as the Dst index used primarily for validation of the FTMM’s quality. During most of the mission, the input models will be taken from the other L2PS chains but at the start of the mission, the FTMM will be in operation before such models are ready. Therefore, in this early phase, auxiliary models based on non-Swarm data will be used instead. The choice of models can be directed by the SIL. Since the models, and possibly the data, will be revised during the mission, it is essential that the FTMM can be reprocessed without interrupting its daily delivery. Once data and models have been successfully ingested, the algorithm executes each processing step shown in Fig. 1, and described in Sections 2.1 through 2.11.

When available, data from the previous execution of the FTMM chain can be re-used to significantly speed up the processing. This is important because the method used to separate the internal and external sources (Section 2.8) requires 30 days of processed satellite data. But as each daily execution of the FTMM chain shares most of these data with the previous execution, there are significant efficiencies to be gained by applying Steps 1 to 5 only to the new data. At the end of the processing chain, the processed satellite data from Step 5 are trimmed to contain only the those required for the next execution and then saved. This re-use of data would also allow the separation of sources step to accumulate more than 30 days of data without significantly increasing the processing time.

2.1 Step 1: reject high magnetic-latitude data

After the data and models have been ingested, the algorithm rejects data poleward of °50° geomagnetic latitude. This is to avoid auroral current systems and field aligned currents that are not part of the large scale magnetospheric field. These are currently not well modelled by other chains and so cannot be reliably removed from the satellite data.

2.2 Step 2: remove core and lithospheric fields

In order to isolate the magnetospheric (and induced) magnetic field signals from the Swarm satellite data, modelled values of the core, lithospheric (and later the ionospheric, see Section 2.4) field are subtracted from each datum. During normal mission operation, models from the L2PS will be used with an expected maximum spherical harmonic degree up to 133. At the start of the mission, alternative (auxiliary) models will be used. The L2PS models will be periodically updated as the Swarm data set grows. For the FTMM, this updating is particularly important for the core field subtraction. Errors in the core model’s secular variation (SV) estimate will result in some core field signals remaining in the data through the remainder of the algorithm. As the Earth rotates beneath the satellites ’ orbits, the residual core field could produce a daily signal and harmonics within the FTMM. The SCARF consortium has anticipated the importance of accurate core field SV estimates and has specified a Fast-Track Core Field model (Rother et al., 2013), to be produced every three months.

This step in the processing chain is responsible for a significant part of the overall computation time. The litho-spheric field is static but has a large number of spherical harmonic terms to evaluate. The core field is also computationally expensive despite having many fewer terms. This is because it is time-dependent, which could in principle mean interpolating or extrapolating the coefficients to the time coordinate of every satellite data point. However, the core field varies only very slowly relative to the frequency of the satellite data and so the FTMM extracts the core field coefficients only once per day. The resulting error has been checked using the 11th generation of the International Geomagnetic Reference Field (IGRF-11; Finlay et al., 2010) and is found to be sub-nanotesla and therefore negligible when calculating the FTTM.

2.3 Step 3: separate out night-side data

The ionospheric field is produced predominantly by solar ionisation and high altitude winds on the daylit side of the Earth. These signals are often removed in global field modelling by selecting only unlit (dark/night-time) data. Separating out the unlit data allows the option of generating some FTMMs with a reduced risk of contamination from ionospheric sources. This could provide a check on models making use of both dark and sunlit data. However, unlit data cannot be the sole input to the FTMM chain because a continuous supply of such data is not guaranteed. For example, dawn-dusk orbits at the start of the mission, when all three satellites are in similar orbit planes, would result in no unlit data and therefore an unacceptable gap in the time series of the FTMM. Only if the selection was widened to cover at least 12 hours local time could a continuous FTMM be guaranteed. However, tests using synthetic Swarm data (Section 3.2) suggest that a model based on this minimum 12 hour window would be inferior to one produced without local-time selection.

2.4 Step 4: remove ionospheric field

The FTMM is expected to use all available data, including sunlit data. These data will therefore require correction to remove fields generated in the ionosphere and their corresponding induced sources. The L2PS will produce two ionospheric models: a dedicated model (Chulliat et al., 2013) and a model that is co-estimated by the Comprehensive Inversion (Sabaka et al., 2013). Either of these models can be used during the operational phase of the mission, except early in the mission, when the Comprehensive Model version 4 (CM4; Sabaka et al., 2004) will be used until the L2PS versions become available.

All these models use the F10.7 solar radio flux as an input parameter to the effect of solar irradiance on the upper atmosphere. For example, CM4 takes a three month average of F10.7. Up-to-date F10.7 indices will be made available to the FTMM chain during the mission by the PDGS but clearly it can only make use of past values. Therefore a 1.5 month average of the most recent indices will be used. The L2PS ionospheric models may use a different mean or none at all and the FTMM will be correspondingly configured.

2.5 Step 5: transform to Cartesian coordinates

As far as possible, the processed satellite data should now contain only magnetospheric and corresponding induced sources. These data are now transformed into an Earth-fixed, right-handed Cartesian coordinate frame with the z-axis through the geographic pole and the x-axis through the equator on the prime meridian.

2.6 Step 6: partition data by orbit

A single snapshot of the FTMM is produced for each satellite orbit by averaging the processed magnetic data over that orbit. To facilitate this, the satellite data time series is separated into individual orbits prior to averaging. The data from each satellite are combined (Section 2.7) into a single data set prior to averaging and so are not processed individually. Since Swarm A and B are at a lower altitude than Swarm C (by an amount that increases over the mission but ~ 100 km), the exact period of one orbit is ambiguous. However, the difference in orbit periods between the upper and lower satellites is only of the order of a couple of minutes and so the notion of ‘an orbit ’ for the three satellites is still useful. By default, ascending (south to north) equator crossings by Swarm A are used to define this interval but a different satellite could be selected by the SIL. This orbit interval is then applied to partition the data from all three satellites.

2.7 Step 7: average over orbit

The processed input satellite data from all three satellites are averaged over each orbit (as defined in Section 2.6). The difference in the magnetic field measured by Swarm A and B is small and probes a spatial scale that is not significant for the FTMM. Giving both Swarm A and B the same weight, individually, as Swarm C in the average would bias the model towards the local-time sampling of the lower pair. Therefore, to make the greatest use of any local time differences in the orbit of Swarm C compared with A and B, the two lower satellites are each given half the weighting of Swarm C in the average.

The mean over each orbit is performed in the Earth-fixed Cartesian coordinate frame (Section 2.5). It could be argued that performing this average in a Sun-synchronous frame would be more appropriate for magnetospheric sources. At least some of the apparent time variation of these fields in the Earth-fixed frame will be due to its rotation relative to static sources in the SM and GSM frames. However, tests using synthetic satellite data did not show any improvement in performing this step in the SM frame (Section 3.3) and implementing it would require an a priori assumption about the orientation of the Sun-synchronous frames. It was therefore decided to compute the mean in the Earth-fixed frame.

At the end of this step, the processed satellite data consists of a time-series of averaged magnetic field values in Earth-fixed Cartesian coordinates, one per component.

2.8 Step 8: separate internal and external sources

The fields from external degree 1 sources are uniform in space and therefore the orbit averages of these contributions can be easily related to the Gauss coefficients (Lesur et al., 2008). However, the contribution to the average from the internal degree 1 sources will depend on each satellite’s orbit altitude and configuration, and the spatial coverage of the data. This relationship between internal coefficient and contribution to orbit average cannot be reliably computed in advance, especially since unexpected gaps in the data cannot be ruled out. Since the FTMM requires automatic and therefore robust algorithms that will complete in most circumstances, a two-stage separation method has been implemented.

The first stage calculates the sum of the corresponding internal and external degree 1 Gauss coefficients by solving Eq. (1) to obtain a rough internal-external separation of the coefficients and then summing the results. This assumes a single frequency independent factor, α = 0.27, relating the size of the primary to associated induced coefficients corresponding to an insulator overlying an infinitely conducting layer (Langel and Estes, 1985) at depth of 1200 km. This α is independent of the factors (βs in Eq. (1)) relating the induced field coefficients to their contributions to the orbit averages (Ms in Eq. (1)). Each internal coefficient has one β associated with each component of the orbit average. For example, the coefficient is related to its contribution to the orbit average of the X-component (M x ) by the factor βg10,X in Eq. (1).

Corresponding β s for the external coefficients are given in Eq. (1) as –1s since the fields from external degree 1 coefficients are uniform in space and hence their contributions to the orbit means are independent of orbit position or gaps in the data. It is because the contributions from the internal degree 1 fields are dependent on their β s that we must first solve for the Gauss coefficients using Eq. (1) before going onto the second stage.

((1))

Since the β values are dependent upon the orbit configuration and the data distribution, they are calculated for each orbit prior to solving Eq. (1). This is done by generating magnetic field values from unit strength internal coefficients at the same locations as the satellite data over the orbit. The β values can then be calculated by performing orbit averages (Section 2.7) of these sets of synthetic field values. This method is robust in its operation but gaps in satellite data may still affect the quality of the model. Therefore the number of data used from each satellite for each FTMM model are output along with the coefficient values (Table 1). The corresponding internal and external coefficients from the solution to Eq. (1) are then summed. These re-combined time series are then interpolated onto a fixed 90 minute sampling rate to ensure a regular model output regardless of any variations in orbit length over the mission.

Table 1. The primary outputs of the Fast-Track Magnetospheric Model. The outputs will be released in Common Data Format (CDF) that will include basic meta-data describing the output.

The second stage performs a more accurate separation in a similar manner to Olsen et al. (2005). This assumes the one-dimensional conductivity model of Utada et al. (2003), and this can be updated when the L2PS models become available (Püthe and Kuvshinov, 2013). To derive the internal degree 1 sources, the time series of orbit averages is first detrended then convolved with a 30 day Q-response filter in the time domain. These time series of internal coefficients are then subtracted from the orbit averages to obtain the external coefficients.

2.9 Step 9: rotate to Solar Magnetic frame

The coefficients produced in Section 2.8 are relative to the Earth-fixed geographic reference frame. As a convenience to users of the FTMM, these coefficients are rotated into the SM reference frame. The main field model used to define the SM coordinates will be specified in the CDF files ’ meta data. The coefficients in the geographic frame make no assumption about orientation of the dipole main field, other than to reject data by geomagnetic latitude.

2.10 Step 10: automatic validation

Since the FTMM must be produced automatically and is typically disseminated before it can be reviewed by a scientist, there is a need for a level of internal automatic validation sufficient to catch any gross errors in the product. This is done using a combination of range checking on model outputs and comparisons with independent data, such as the separated Dst index (Est and Ist) and observatory data, when available. If internal validation fails, the model is not released and the SIL is alerted so that it can be approved or reprocessed as required with the minimum delay. Clearly, the FTMM cannot be expected to be identical to other measures of magetospheric activity and tolerances will be chosen to catch only gross errors. Upon successful validation, the FTMM is made available in Common Data Format (CDF) to the PDGS for dissemination to the L2PS and the wider community. The primary outputs of the FTMM are described in Table 1.

2.11 Review by scientist in loop

It is anticipated that the FTMM will operate autonomously with minimal intervention. However, to ensure the quality of the product the model’s output will be regularly reviewed, retrospectively, by the SIL. This will involve a comparison with independent models such as the magnetospheric model from the L2PS’s Comprehensive Inversion, when available. However, unlike the internal validation (Section 2.10), it is not prescriptive. Rather, it is intended that the SIL use their judgement to perform whatever kind of validation they deem appropriate to verify the quality of the model. Based on the SIL’s assessment, the FTMM can be reprocessed.

The SIL can also assess the available data and models input to the processing chain and substitute these when, for example, updated models or reprocessed Level 1b data become available. It is intended that elective changes to the processing chain such as these will always be performed without interrupting the scheduled output of the FTMM.

3. Test Results

In this section we review the performance of the FTMM derived from the Category 1 Test Data Set (TDS-1), version ‘0101’. The SCARF consortium uses a common test data set for the purpose of development and testing of the Level 2 processing chain algorithms. The TDS-1 was constructed from spherical harmonic reference models of the core, lithospheric, ionospheric and magnetospheric fields to construct a synthetic data set along Swarm satellite orbits. To check the recovery of the magnetospheric fields, the computed FTMMs from the Level 1b data were compared with the magnetospheric reference model (primary and induced) used to generate the TDS-1 data set. Although such a comparison will not be available during the operational phase, it is the best method for validating the ability of the FTMM chain to recover the ‘true ’ magnetospheric signal from all the (simulated) sources. The reference model’s degree 1 terms have a time resolution of 60 minutes, compared with the FTMM’s 90 minutes, and must be interpolated to the latter’s time coordinates before a direct comparison can be made. The synthetic Level 1b data was also used by other processing chains to compute the L2PS magnetic field models input to the FTMM algorithm.

The metrics used to assess the test models ’ qualities are all defined by comparing corresponding coefficients from the FTMM and the TDS-1 Magnetospheric reference model over the entire simulated mission: the Root-Mean-Square (RMS) difference; the squared correlation coefficient; the gradient of the best fit straight line between the two models; the y-intercept of the best fit line (reference model value when FTTM is zero); and the Magnitude Squared Coherence (MSC) defined as

where Pftmm,tds is the cross power spectral density between coefficients from FTMM and TDS-1 reference model, PFTMM and PTDS are their respective power spectral densities, and f is temporal frequency.

The results of several model runs are described in detail in Sections 3.1 through 3.5 and their distinguishing features are summarised in Table 2. However, the models all have some features in common. They all use the TDS-1 data set (version ‘0101’) and data from all three satellites, all reject data at geomagnetic latitudes poleward of °50°, and all use the Comprehensive Inversion’s ionospheric and induced field model, modulated by a mean of the F10.7 index over the 45 days prior to the model date.

Table 2. Summary of the different models produced to test the FTMM algorithm and their distinguishing parameters. The models are: ‘Standard’, which is the base model for comparison; ‘12hr’, which selects data within a 12 hour local window centred on midnight; ‘SM ’ performs the average over each orbit in Solar Magnetic Coordiantes; ‘C.I. ’ uses only Comprehensive Inversion models to subtract non-magnetospheric fields; and ‘20s ’ uses double the data sampling rate of the other models. ‘GEO ’ refers to the Earth-fixed Geographic frame; and ‘Reference ’ indicates that the TDS-1 reference models were used to subtract the core and lithospheric fields (but the Comprehensive Inversion ionospheric-field model is used in all cases).

3.1 Standard Model

The term ‘Standard Model ’ will refer to the version of the FTMM used as the basis for comparing the test runs. This model uses the distinguishing parameters defined under the heading ‘Standard ’ in Table 2.

Figure 2 shows the time series of the FTMM’s and TDS-1 reference model’s Gauss coefficients that produce fields aligned with the Earth’s rotation axis . Being aligned close to the Z-axis of the SM and GSM coordinates systems, it is the dominant magnetospheric component in the geographic Earth-fixed, Earth-centred coordinate frame. There is good visual agreement between the two models with periods of high activity being captured by the FTMM. No obvious trends or large offsets appear in the differences, although they show some modulation over the entire 4.5 year span of the models. These modulations seem to follow the background level of the magnetospheric field. However, during brief periods of very high disturbance, such as those around 2000.5, 2001.25, and 2001.75, the differences are in the opposite sense (negative) with the FTMM underestimating the peak disturbance relative to the (true) reference model. The RMS difference between the two models is 3.52 nT on a range of disturbances of the order of 300 nT.

Fig. 2.
figure 2

Time series in nT of (external) coefficient from the FTMM (top) and the TDS-1 reference model (middle) and their difference (FTMM minus TDS-1, bottom). There is good agreement between the two models although there is some evidence of increased differences during periods of strong magnetospheric field.

Figure 3 shows the coefficients from the reference model plotted against those from the FTMM. There is good visual correlation between the two models and the square correlation coefficient is 0.99. The gradient of the best-fit line is very close to unity (1.01) although there is some evidence of a steepening for the most disturbed days, which is consistent with the differences seen in Fig. 2. It is not clear why the FTMM is losing sensitivity on the most disturbed days. It may be a result of its lower time-resolution compared to the reference model. The y-intercept of the best-fit line is −3.10 nT, which indicates that there is a slight positive bias in the zero level of the FTMM but this is small relative to the range of disturbances.

Fig. 3.
figure 3

Plot of (external) coefficients from the FTMM against the TDS-1 reference model. Also shown is the best-fit straight line to the data. The correlation is very good but there is some evidence of a steepening of the gradient for the most disturbed periods (towards top right of plot).

Figure 4 shows the Magnitude Squared Coherence (MSC) between the two models over all periods down to three hours and also in more detail for periods greater than two days. The MSC is very good (close to one) for low frequencies with a minimum of 0.99 for periods greater than two days. At shorter periods, the MSC declines. Dips at frequencies of 1 and 2 per day are likely due to an imperfect ionospheric field subtraction, exacerbated by the chain’s averaging of the F10.7 index of the preceding 45 days when no averaging is used by the TDS-1 reference model.

Fig. 4.
figure 4

The Magnitude Squared Coherence between the (external) coefficients from the FTMM and the TDS-1 reference model for all frequencies (top) and for periods greater than two days (bottom). The coherency is very good for low frequencies but falls off sharply for periods less than about 5 hours. Dips at 24 and 12 hours are likely due to ionospheric field contamination.

Figures 5 and 6 are the correlation and MSC plots for the (internal) coefficient, whose dipole magnetic field axis passes through the prime meridian and equator of the geographic coordinate system. In this case the RMS difference is only 0.67 nT but on a smaller range of values than q10. Again, the correlation is good with a squared coefficient of 0.90. The best-fit gradient in Fig. 5 is 1.2 and the y-intercept is very small (−0.003 nT). The MSC in Fig. 6 is not as good as with but is greater than 0.66 for periods longer than two days.

Fig. 5.
figure 5

Plot of (internal) coefficient from the FTMM against the TDS-1 reference model. Also shown is the best-fit straight line to the data. The correlation is good but there is some evidence of a steepening of the gradient for the most disturbed periods (towards top right of the plot).

Fig. 6.
figure 6

The Magnitude Squared Coherence between the (internal) coefficients from the FTMM and the TDS-1 reference model for all frequencies (top) and for periods greater than two days (bottom). The coherency is good for periods greater than two days but falls off steadily for shorter periods. A sharp dip at 24 hours is likely due to ionospheric field contamination.

A summary of the metrics for the other degree 1 coefficients is shown in Table 3. In general the spherical harmonic order zero coefficients match the reference model better than the order 1 coefficients. In all cases, the Standard Model’s coefficients expressed in the geographic reference frame show good agreement in all the metrics. This is particularly satisfying given the need for the two stage separation process described in Section 2.8. Also shown in Table 3 are the metrics for one order zero and one order one coefficient given in the Solar Magnetic (SM) frame (Section 2.9), defined using IGRF-11 at the model dates, which are indicative of the quality of the other degree 1 coefficients. The quality of remains high and may actually improve with a reduction in RMS misfit. For the coefficient the minimum coherence declines significantly although the other metrics remain good. This will be due to the orientation of the dominant component of the magnetospheric field along the SM Z-axis (through the geomagnetic pole). In that frame, the contrast is greatest between the less well resolved order one coefficients and the dominant order zero coefficient. In the geographic frame, the dominant signal is resolved partly into the order one coefficients and their metrics are correspondingly better. The entire run took a little under four days to complete on a 2 GHz Dual-Core Processor. From this we can estimate a turnaround time of approximately five minutes to process a day’s worth of new satellite data during the operational phase of the mission.

Table 3. Summary of the quality metrics for the Standard Model (Section 3.1) for all model coefficients given in the geographic frame. Also shown in parenthesis are the metrics calculated for the and coefficients in the Solar Magnetic (SM) frame. The metrics in the geographic frame show good recovery of the magnetospheric signal, especially for the order zero coefficients. In the SM frame, the metrics remain good for and mostly for except for the minimum coherence.

Many of the parameters of the processing chain shown in Fig. 1 are configurable by the SIL. In Sections 3.2 to 3.5 we assess the effect of changing some of these parameters relative to the Standard Model. The results are summarised in Table 4 for the and coefficients. These coefficients are indicative of the relative performance of the other model coefficients.

Table 4. Results of the test runs of the FTMM defined in Table 2 and Section 3. For clarity, only the results for the and coefficients are shown but these are indicative of the relative behaviour of the test models for all coefficients. Note that both ‘20s’ run also use input models from the Comprehensive Inversion chain and should therefore be compared with the ‘C.I.’ run. Only the ‘12hr ’ model shows any significant decrease in quality.

3.2 12 hour local time selection

The Standard FTMM requires the subtraction of the ionospheric field signal using an a priori model. As discussed in Section 2.3, a true night-time (unlit) data selection cannot guarantee sufficient continuity of data for regular model production throughout the mission lifetime. However, a minimum 12 hour local time (LT) selection will guarantee a continuous supply of data and may avoid much of the contamination from ionospheric signals for orbits in a day-night orientation. Since the ionospheric signal will still be present in dawn-dusk orbits, an ionospheric field model is still subtracted.

However, from Table 4 it can be seen that this model shows poorer agreement with the reference model than does the Standard Model. Although the term is almost as good as the Standard Model, has notably poorer MSC.

The superiority of the Standard Model when using simulated data could be the result of an unrealistically optimistic ionospheric field subtraction using the Comprehensive Inversion’s model, which may not be as effective when used with real data and imperfect assumptions about the Earth’s conductivity structure. To better assess the 12 hour local-time model, it and the Standard Model were re-run using only the magnetospheric, ionospheric, and corresponding induced sources from TDS-1 (and so requied no core or lithospheric field correction). The results are shown in Fig. 7 (top and second from top panels, respectively). No corrections were made for the ionospheric field and so the models will include the maximum contamination possible from TDS-1. The problem with the 12 hour model is evident from the larger differences from the reference model, which are highly periodic. The shorter period (~4–5 months) is consistent with the local-time evolution of the satellites shown in Olsen et al. (2013). There is also evidence of a longer period envelope that sees a reduction in the short period oscillation towards ~ 2001 followed by an increase towards the end of the time series. This is consistent with the growth and reduction of the orbit-plane separation between Swarms A/B and Swarm C, with the greatest local-time separation at 2001 (6 hours).

Fig. 7.
figure 7

Differences between coefficient from TDS-1 reference model and FTMMs produced from (top) magnetospheric and ionospheric sources using all local times; (second from top) magnetospheric and ionospheric sources using 12 hour local-time window centred on midnight; (third from top) magnetospheric sources only using 12 hour window; (bottom) magnetospheric sources produced from degree 1 terms only of TDS-1 and using 12 hour window.

The origin of these errors will be contamination from unmodelled magnetospheric sources associated with higher spherical harmonic degrees (> 1) in TDS-1 since the periodicity remains even when only magnetospheric and associated induced sources of data are used (Fig. 7, third panel from top). The simulated data contains magnetospheric sources up to spherical harmonic degree 3 and order 1 and induced degrees and orders up to 45 (Olsen et al., 2006). Due to a narrow local-time sampling by the Swarm satellites, especially at the start of the mission when all three satellites are in the same orbit plane, a source characterised by a spherical harmonic term of order 1 fixed in local-time could appear as a spurious signal in the FTMM with a period equal to the local-time repeat period of the satellites. When a test data set is generated from the TDS-1 magne-tospheric reference model using only the degree 1 internal and external coefficients, the periodicity is much reduced (Fig. 7, bottom). In fact, this model closely resembles that derived using magnetospheric and ionospheric sources from all local times (Fig. 7, top), which suggests the un-modelled magnetospheric signals are the dominant source of error when the local-time selection is used.

While the relative importance of unmodelled magneto-spheric and ionospheric sources could change when real data are used, it seems from this test that averaging over both day and night-side parts of the orbit has significant advantages in filtering out these unmodelled sources. And contamination from umodelled magnetospheric sources, most likely originating from asymmetries in the ring current, have been observed in CHAMP data by Balasis et al. (2004) in their degree 1 magnetospheric models and by Kunagu et al. (2013) in mean magnetic field components calculated each orbit using the same 12 hour local-time window used above.

3.3 Orbit average in Solar Magnetic frame

The external magnetospheric sources are strongly affected by the solar wind and are usually described most efficiently in the Solar Magnetic (SM) and Geocentric Solar Magnetospheric (GSM) reference frames. The separation into primary external and induced internal sources (Section 2.8) should be performed in an Earth-fixed frame but the average over each orbit (Section 2.7), which is also performed in the Earth-fixed frame in the Standard Model, could be computed in a Sun-synchronous frame.

To test if this change of reference frame has any impact on the ability of the FTMM to recover the magnetospheric signal, the Standard Model was modified to average in the SM frame and the results are shown in Table 4. It can be seen that computing the orbit average in SM coordinates has no significant effect on the recovery of the magneto-spheric field. As a further test we fit models in both frames using one year of CHAMP data (2004). Although no reference model exists for the real data we can compare the models from the SM and GEO frames with each other using the metrics in Table 1. How these metrics compare with those computed between the Standard Model and TDS-1 in Table 4 can be used to estimate how significant the choice of reference frame is compared to the overall errors. It is found that the two CHAMP based models are much closer to each other than the Standard Model is to the TDS-1 reference model. For the coefficients from the CHAMP models, the RMS difference is (0.16, 0.33) nT, the squared correlation (0.99993, 0.98), and the magnitude squared coherence for periods > 2 days (≥ 0.99996, ≥ 0.95). However, the RMS difference for the coefficient is smaller but not insignificant compared with the TDS-1 runs in Table 4 (0.67). The best frame in which to perform the orbit averaging will be considered further as part of ongoing improvements to the processing chain.

3.4 Using only Comprehensive Inversion input models

In the Standard Model, the core and lithospheric field subtraction (Section 2.2) used the TDS-1 reference models. This is equivalent to having perfect models of these sources since they are also used to generate the TDS-1 Level 1b data set. During the operation phase of the mission, models of these sources from the other L2PS chains will be used when available. Since these models will themselves be approximations to the core and lithospheric fields, any errors with respect to the true field may also affect the FTMM. To better simulate the FTMM during the operational phase, we have re-run the Standard Model using the Comprehensive Inversion’s models for all field subtractions.

As expected, there is a slight increase in the RMS differences and best-fit y-intercept with respect to the reference model (Table 4). However, this modification has very little effect on the squared correlation coefficient, MSC, or best-fit gradient. It should be noted that during the operational phase, the Fast-Track Core-Field Model will be used in addition to the Comprehensive or Dedicated Core-Field models.

3.5 Double data sampling rate

The Standard Model and the test models in Sections 3.2 through 3.4 have all sub-sampled the 1 Hz input Level 1b satellite data by selecting only every 40th datum. The resulting number of data points per 90 minute FTMM, after rejection of high geomagnetic-latitude data (Section 2.1) is usually greater than two-hundred, which is thought sufficient for a degree 1 magnetospheric model. However, to check that model quality is not being compromised by under-sampling, the FTMM using the Comprehensive input models (Section 3.4) was re-run using every 20th datum, doubling the amount of input data per orbit.

It can be seen from Table 4 that the extra data has almost no effect on the FTMM compared with the ‘C.I. ’ model, validating the use of the lower sampling rate.

4. Summary and Conclusions

The processing chain developed for the ESA Swarm L2PS’s Fast-Track Magnetospheric Model is rapid and robust enough to operate autonomously, with periodic supervision by an expert. The algorithm is effectively real-time, in the sense that the latency between receipt of data and output of the model is only a small fraction of the orbit period. The extent to which the FTMM can be provided to the user in real-time is limited only by the speed by which the PDGS can deliver satellite data to the L2PS and retrieve the model outputs.

However, this algorithm does have its limitations that should be clearly stated. The simplicity of the model and the desire to release the results in near-real time mean that spurious signals could contaminate output. Fictitious periods of one day and harmonics are possible due to inaccurate a priori models used to isolate the magnetospheric signal in the Swarm data. Of particular importance is the core-field model, where secular variation errors could manifest themselves as daily and shorter harmonic periods in the FTMM. At longer periods, local-time dependent magneto-spheric signals, not modelled by the FTMM, can still manifest themselves due to the periodicities in the satellites ’ own local-time sampling, particularly at the start of the mission before the upper and lower satellite orbits diverge. However, it has been shown, in principle, that at least some of the unmodelled signal can be filtered out by using data from all local times. Finally, it is also possible that some unmodelled signals (such as the 27-day period found by Kunagu et al., 2013) may have strong radial gradients, perhaps from field-aligned currents not modelled by TDS-1, that could result in different fields being measured by the upper versus lower two Swarm satellites. An attempt to quantify and perhaps mitigate some of these effects could be attempted using existing CHAMP and Øersted data or by using more detailed models of the magnetospheric field (e.g. Tsyganenko and Sitnov, 2005).

Nevertheless, the test runs have shown that the FTMM can recover a good representation of the magnetospheric and corresponding induced fields, particularly for the dominant order zero terms although less well for the order one terms. Variants of the modelling chain are currently being investigated with the intention of improving its recovery of the degree 1 signal.

Notes

  1. 1 http://wdc.kugi.kyoto-u.ac.jp/dstdir/index.html

References

  • Balasis, G., G. Egbert, and S. Maus, Local time effects in satellite estimates of electromagnetic induction transfer functions, Geophys. Res. Lett., 31, 16, L16610, 2004.

    Article  Google Scholar 

  • Chulliat, A., P. Vigneron, E. Thébault, O. Sirol, and G. Hulot, Swarm SCARF Dedicated Ionospheric Field Inversion chain, Earth Planets Space, 65, this issue, 1271–1283, 2013.

    Article  Google Scholar 

  • Finlay, C. C., S. Maus, C. D. Beggan, T. N. Bondar, A. Chambodut, T. A. Chernova, A. Chulliat, V. P. Golovkov, B. Hamilton, M. Hamoudi, R. Holme, G. Hulot, W. Kuang, B. Langlais, V. Lesur, F. J. Lowes, H. Lühr, S. Macmillan, M. Mandea, S. McLean, C. Manoj, M. Menvielle, I. Michaelis, N. Olsen, J. Rauberg, M. Rother, T. J. Sabaka, A. Tangborn, L. Toffner-Clausen, E. Thébault, A. W. P. Thomson, I. Wardinski, Z. Wei, and T. I. Zvereva, International Geomagnetic Reference Field: the eleventh generation, Geophys. J. Int., 183(3), 1216–1230, 2010.

    Article  Google Scholar 

  • Friis-Christensen, E., H. Lühr, and G. Hulot, Swarm: A constellation to study the Earth’s magnetic field, Earth Planets Space, 58, 351–358, 2006.

    Article  Google Scholar 

  • Kunagu, P., G. Balasis, V. Lesur, E. Chandrasekhar, and C. Papadimitriou, Wavelet characterization of external magnetic sources as observed by CHAMP satellite: evidence for unmodelled signals in geomagnetic field models, Geophys. J. Int., 192, 946–950, 2013.

    Article  Google Scholar 

  • Langel, R. and R. Estes, Large-scale, near-Earth magnetic fields from external sources and the corresponding induced internal field, J. Geophys. Res., 90, 2487–2494, 1985.

    Article  Google Scholar 

  • Lesur, V., I. Wardinski, M. Rother, and M. Mandea, GRIMM: the GFZ Reference Internal Magnetic Model based on vector satellite and observatory data, Geophys. J. Int., 173, 382–394, 2008.

    Article  Google Scholar 

  • Lühr, H. and S. Maus, Solar cycle dependence of quiet-time magneto-spheric currents and a model of their near-Earth magnetic fields, Earth Planets Space, 62, 843–848, 2010.

    Article  Google Scholar 

  • Maus, S. and P. Weidelt, Separating the magnetospheric disturbance magnetic field into external and transient internal contributions using a 1D conductivity model of the Earth, Geophys. Res. Lett., 31(12), L12614, 2004.

    Article  Google Scholar 

  • Olsen, N., T. Sabaka, and F. Lowes, New parameterization of external and induced fields in geomagnetic field modeling, and a candidate model for IGRF 2005, Earth Planets Space, 57, 1141–1149, 2005.

    Article  Google Scholar 

  • Olsen, N., R. Haagmans, J. Sabaka, A. Kuvshinov, S. Maus, M. E. Pu-rucker, M. Rother, V. Lesur, and M. Mandea, The Swarm End-to-End mission simulator study: A demonstration of separating the various contributions to Earth’s magnetic field using synthetic data, Earth Planets Space, 58, 359–370, 2006.

    Article  Google Scholar 

  • Olsen, N., E. Friis-Christensen, R. Floberghagen, P. Alken, C. D Beggan, A. Chulliat, E. Doornbos, J. T. da Encarnação, B. Hamilton, G. Hulot, J. van den IJssel, A. Kuvshinov, V. Lesur, H. Lühr, S. Macmillan, S. Maus, M. Noja, P. E. H. Olsen, J. Park, G. Plank, C. Püthe, J. Rauberg, P. Ritter, M. Rother, T. J. Sabaka, R. Schachtschneider, O. Sirol, C. Stolle, E. Thébault, A. W. P. Thomson, L. TØffner-Clausen, J. Velímský, P. Vigneron, and P. N. Visser, The Swarm Satellite Constellation Application and Research Facility (SCARF) and Swarm data products, Earth Planets Space, 65, this issue, 1189–1200, 2013.

    Article  Google Scholar 

  • Püthe, C. and A. Kuvshinov, Determination of the 1-D distribution of electrical conductivity in Earth’s mantle from Swarm satellite data, Earth Planets Space, 65, this issue, 1233–1237, 2013.

    Article  Google Scholar 

  • Rother, M., V. Lesur, and R. Schachtschneider, An algorithm for deriving core magnetic field models from the Swarm data set, Earth Planets Space, 65, this issue, 1223–1231, 2013.

    Article  Google Scholar 

  • Sabaka, T. J., N. Olsen, and M. Purucker, Extending comprehensive models of the Earth’s magnetic field with Ørsted and CHAMP data, Geo-phys. J. Int., 159, 521–547, 2004.

    Article  Google Scholar 

  • Sabaka, T. J., L. TØffner-Clausen, and N. Olsen, Use of the Comprehensive Inversion method for Swarm satellite data analysis, Earth Planets Space, 65, this issue, 1201–1222, 2013.

    Article  Google Scholar 

  • Thébault, E., P. Vigneron, S. Maus, A. Chulliat, O. Sirol, and G. Hulot, Swarm SCARF Dedicated Lithospheric Field Inversion chain, Earth Planets Space,65, this issue, 1257–1270, 2013.

    Article  Google Scholar 

  • Thomson, A. and V. Lesur, An improved geomagnetic data selection algorithm for global geomagnetic field modelling, Geophys. J. Int., 169, 951–963, 2007.

    Article  Google Scholar 

  • Tsyganenko, N. A. and M. I. Sitnov, Modeling the dynamics of the inner magnetosphere during strong geomagetic storms, J. Geophys. Res., 110, A03208, 2005.

    Google Scholar 

  • Utada, H., T. Koyama, H. Shimizu, and A. Chave, A semi-global reference model for electrical conductivity in the mid-mantle beneath the north Pacific region, Geophys. Res. Lett., 30(4), 1194, 2003.

    Article  Google Scholar 

Download references

Acknowledgments

The author would like to thank Vincent Lesur and Nils Olsen for their helpful input on source separation. The former and one anonymous referee provided useful feedback on this manuscript. The author would also like to thank Ciarán Beggan, Susan Macmillan, and Alan Thomson for their input throughout the model’s development. This paper is published with the permission of the Executive Director of the British Geological Survey (NERC).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Brian Hamilton.

Rights and permissions

Open Access  This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Hamilton, B. Rapid modelling of the large-scale magnetospheric field from Swarm satellite data. Earth Planet Sp 65, 1295–1308 (2013). https://doi.org/10.5047/eps.2013.09.003

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.5047/eps.2013.09.003

Key words