We studied three test case scenarios in this paper. The main results are presented in a scenario with: (A) identical true and imposed magnetic fields, which means that the imposed proxies of the OIMF equal the true oceanic contribution in the observations. This case represents an ideal scenario, where the scale factor should be precisely one at each time step and each location (see Eq. 3). Imposing the exact OIMF from circulation may not be the most realistic scenario. However, it exhibits the general principle and the upper limit of our proposed method. Afterward, we present a second test case scenario with: (B) deviating true and imposed magnetic fields, which means that the imposed presumed proxies of the oceanic field slightly differ from true oceanic contribution in the observations. This second scenario is designed to examine the robustness and the optimality within the scale factor approach of our method and analyses the effect of these small deviations on the Kalman filter predictions. Within this study, we investigate the influence of spatial under-sampling and temporal smoothing of the presumed proxies on the scale factor determination. Finally, a third test case scenario is presented to show a possible application of the scale factor approach. This scenario uses a: (C) deviating conductivity of the imposed magnetic field, which means that the imposed proxies are calculated assuming a slightly different ocean conductivity.
For the analysis, the assimilation algorithm provides two types of quantities associated with the scale factor: the posterior mean and the posterior standard deviation of the ensemble of scale factors at each node of the Gauss–Legendre grid. We use the posterior mean \({\mathbf{k}}(x,y)\) as measure of OIMF detectability and consider its standard deviation \(\sigma _{\mathbf{k}}(x,y)\) as the uncertainty of the scale factor. Note that the chosen distribution \({\mathcal {N}}(0,1)\) of the scale factors at the beginning of the assimilation implies that the Kalman filter initially assumes no contribution from the OIMF. Under the influence of observations, the Kalman filter updates both the scale factor and its associated uncertainty, and through the assimilation of 7.1 years of Swarm-like artificial data, these quantities converge to their actual values. Eventually, the final determined scale factor \({\mathbf{k}}_{f}(x,y)\) (year 2021.0) is used to rescale the imposed OIMF:
$${\mathbf{B}}_{r}^{\text{rescaled}}(t,x,y) = {\mathbf{k}}_f(x,y) \cdot {\mathbf{B}}_{r}^{\text{imposed}}(t,x,y).$$
(7)
Because at the beginning of the assimilation, no oceanic contribution was assumed, the rescaled field \({\mathbf{B}}_{r}^{\text{rescaled}}(t,x,y)\) reflects the detected OIMF and emphasizes the part of the signal gained from the true OIMF by the assimilation (under the given imposed field). Furthermore, the rescaled field is used to analyze the assimilation results. We utilize a normalized root mean square deviation (NRMSD) as a quality measure of how well the rescaled field reflects the true field, which is defined as follows:
$$\mathbf{NRMSD}(x,y) = \sqrt{ \frac{\sum _{t_i=0}^N \left( {\mathbf{B}}_{r}^{\text{rescaled}}(t_i,x,y) - {\mathbf{B}}_{r}^{\text{true}}(t_i,x,y) \right) ^2}{\sum _{t_i=0}^N {\mathbf{B}}_{r}^{\text{true}}(t_i,x,y)^2} },$$
(8)
where \(t_i\) denotes the time steps and the normalization accounts for the relation of deviations to the strength of the true signal.
Test case scenario A: identical true and imposed magnetic field
In this scenario, the imposed oceanic field is identical to the true oceanic field included in the Swarm-like artificial data. This test case aims to demonstrate the main results, identify essential dependencies, and evaluate the determination of the scale factor. Since it is an idealistic scenario, it also serves as a reference. First, the results of the assimilation A.1 only using the imposed field as prior information for the oceanic source are presented. Subsequently, another improved assimilation result A.2 is shown, where prior spatial correlations are imposed to enforce spatial constraints on the scale factor determination. Finally, assimilation A.3 is used to investigate the influence of the ionosphere on the method’s sensitivity.
The first assimilation results A.1 obtained with a diagonal prior covariance matrix are presented in Fig. 4. The upper left plots show the final scale factor’s deviation from the true value \({\mathbf{k}}_{f}=1\) in this test case scenario. Note that in this ideal case, the deviation \(|{\mathbf{k}}_{f} -1|\) is equal to the NRMSD (see Eq. 8). The results show that the Kalman filter strongly reduces the deviation of the scale factor in nearly all oceanic areas. Expectedly, in large parts of the continental regions, where no magnetic signals from the ocean arise, the deviations of the scale factor remained close to the initial value of one. Remarkably, large parts of the Indian Ocean and the Western Pacific, as well as some northern parts in the area of the Antarctic Circumpolar Current ACC (e.g., close to Australia) and individual areas in the Atlantic Ocean (e.g., near the Gulf Stream), evince finally only very small deviations, which indicates a successful scale factor determination in large parts of the ocean. In general, the deviations increase in the Eastern Pacific, the Atlantic, and the Northern and Southern Oceans.
Moreover, the standard deviation of the final factor \(\sigma _{{\mathbf{k}}_{f}}\), which is considered as associated uncertainty (shown in the upper right of Fig. 4), also exhibits a significant reduction in the oceanic areas. The pattern of this reduced uncertainty correlates well with the high variance of the OIMF (see Fig. 2, right panel). Areas characterized by high variability of the OIMF evince low uncertainties, whereas areas with a tiny OIMF variability evince high uncertainties. Note that the mean of the OIMF does not affect the results of the scale factor determination (Fig. 2, left panel). However, it can be seen that at higher latitudes (larger than \(60^{\circ }\)), the standard deviation of the scale factor strongly increases. This latitude dependence is explainable by the fact that the influence of the ionospheric magnetic field contribution also increases strongly at higher latitudes and corresponds to the chosen nighttime data selection below \(60^{\circ }\). Overall, regions with small deviations of the scale factor from the true value coincide with areas of low standard deviation of the scale factor, and conversely, larger deviations occur in areas with higher standard deviation. We conclude that the posterior standard deviation of the scale factor is a valuable measure to assess the quality of the assimilated scale factor.
Finally, the bottom plots in Fig. 4 present the mean rescaled OIMF and its deviation from the mean true OIMF included in the observations. As stated before, the rescaled field reflects the detectable part of the OIMF and emphasizes the part of the signal gained from the true OIMF by the assimilation. The mean rescaled field is very similar to the mean of the true OIMF (compare to Fig. 2). The globally slight deviations demonstrate remarkable accordance between the rescaled field and the true oceanic field. The rescaled field is used to calculate the NRMSD at each location (note that in the ideal scenario, the NRMSD is equivalent to \(|{\mathbf{k}}_{f} -1 |\)). In this test case scenario, the averaged NRMSD over the oceans is 0.26 (the grid points over the ocean are again selected using the ETOPO1 dataset). The average \(\sigma _{{\mathbf{k}}_{f}}\) in the oceanic area is 0.38.
Further insight can be gained by looking at the development of the scale factor over time. The scale factor evolution in two different areas is shown: One area with high and another with relatively low variability of OIMF. The high-variability area is located in the Southern Indian Ocean between Africa and Australia, around \(45^{\circ }\) South, and the low-variability area is located in the Eastern Pacific close to the coast of central South America, around \(15^{\circ }\) South (see also Fig. 2). Both the point-wise local results from 10 connected grid points as well as the spatial mean over these areas are presented in Fig. 5. As can be seen from this figure, the factor evolves remarkably fast towards the true value of 1.0. The spatial mean in the high-variability area converges already close to one after less than one year of assimilated data. It is clearly visible that the convergence in the case of the low-variability area takes a longer time. However, the spatial mean in this area also approaches the correct value after approximately two years of assimilated data. In comparison, the spread of the local \({\mathbf{k}}\)-results is significantly narrower for the high-variability area, which corresponds well to the lower standard deviation of the scale factor. Promisingly, the fast convergence of the scaling factors emphasizes the method’s high sensitivity to magnetic signals from OIMF.
These results confirm the detectability of the OIMF in this ideal test case scenario using assimilation. We deduce that under the condition of imposing the correct OIMF, our proposed method allows the identification of OIMF in geomagnetic satellite observations.
As stated before, the results can be further improved by incorporating an exponentially decaying spatial correlation structure at the beginning of the assimilation. The results of an assimilation (A.2) using such a prior spatial covariance with a correlation length of 2000 km are presented in Fig. 6. As compared to Fig. 4, there is an overall improvement in the scale factor determination. The deviation as well as the standard deviation of the scale factor are further reduced compared to the previous result. Due to the spatial correlation, \(\sigma _{k_f}\) decreased significantly also at latitudes larger than \(60^{\circ }\). Furthermore, the improved scale factor determination is also emphasized by the smaller deviations of the rescaled radial magnetic field from the true radial magnetic field (bottom right plot in Fig. 6). Expectantly, the enforced spatial relation between the scale factor at the beginning of the assimilation results in a slightly smoother rescaled radial magnetic field (compare bottom left plots of Fig. 6 and Fig. 4). By looking at the factor evolution in the same areas as chosen before (see Fig. 7), one can clearly see the effect of the type of prior spatial covariance. On the one hand, the spread of the local point-wise results is much narrower according to the lower uncertainty, and on the other hand, the spatial correlation expedites the convergence toward the correct value at the beginning of the assimilation. The overall average over the ocean of the NRMSD decreased from 0.26 before down to 0.17. Correspondingly, the average standard deviation of the scale factor was reduced from 0.38 to 0.22. Thus, we conclude that imposing a priori spatial correlations increases the accuracy of factor determination.
Finally, we used the ideal test case scenario to investigate a little further the influence of the magnetic component resulting from the ionosphere. In general, the Kalmag assimilation distinguishes between internal and external sources. As one of the internal sources, the ionospheric field is arguably a strong competitor for the separation of ocean-induced magnetic fields. In comparison, the other two internal sources from the core and lithosphere differ more clearly in time behavior since they develop much slower. Most of the ionospheric influence is avoided by the selection of nighttime data and low geomagnetic activity. Moreover, the crucial areas of this assimilation are the mid-latitudes (due to the applied data selection below \(60^{\circ }\) and low ocean-induced signals at the geomagnetic equator). In this case, the solar-quiet magnetic fields dominate the ionospheric component. Since studies like Suzuki (1978) and Takeda (2002) found that low degrees of SH can capture the solar-quiet fields, we simulated the ionospheric component in our OSSE up to an SH degree of 5 in the first place. However, the treatment of the ionospheric magnetic field still affects the method’s sensitivity. Thus, we performed two additional assimilations A.3 to demonstrate possible ionospheric influences. Here, we simulated the ionospheric component and included it in the artificial observations by a much higher spatial resolution of SH 10 and SH 50. The effect on the detectability of the OIMF is illustrated in Fig. 8. Both assimilation results, for the ionosphere up to SH 10 and up to SH 50, show a similar pattern compared to the previous assimilation using the ionosphere up to SH 5 (see Fig. 6). However, the scale factor deviation increases slightly for the SH 10 version and, more significantly, for the SH 50 version, which indicates that the influence of the ionosphere can complicate the separation of the OIMF. Similarly, the associated uncertainty of the factor increases in both cases. Due to the spatially expanded resolution of the ionosphere, the averaged NRMSD over the ocean increased from 0.17 to 0.21 (ionosphere SH 10) and up to 0.29 (ionosphere SH 50). Accordingly, the average standard deviation of the final scale factor increases from 0.22 up to 0.27 (ionosphere SH 10) and 0.43 (ionosphere SH 50). We deduce from these assimilations that the ionosphere can affect the scale factor determination. Probably, this has to be taken into account when applying the method outside of this OSSE. However, one also can conclude from these assimilations that the associated uncertainty of the scale factor covers the influence of other magnetic sources like the ionosphere.
Test case scenario B: deviating true and imposed magnetic field
In this second test case scenario, the imposed oceanic field is not identical anymore to the true oceanic field included in the Swarm-like artificial data. The objective of this test case is to show the robustness and the optimality of the proposed method in the presence of slight deviations between the imposed and true OIMF. To ensure comparability, we left the imposed field the same as in the assimilations before and replaced the true oceanic contribution in the Swarm-like artificial data. In doing so, the results are not affected by differently imposed OIMF variability. We consider two simple types of differences between the imposed and true OIMF: On the one hand, spatial differences, where the true OIMF primarily differs in the spatial domain, and on the other hand, temporal differences, where mainly the temporal behavior of the true OIMF is changed. Of course, this is not strictly distinguishable since the magnetic field is not restricted to a fixed point in space both spatial and temporal differences influence each other. However, each case emphasizes the main cause of the differences.
In order to test the effect of spatial deviations, we investigate spatial oversampling of the true OIMF. Instead of using the same spatial resolution as the imposed OIMF, we included the true OIMF in the artificial satellite data with a higher spatial resolution. Two assimilations are performed in this setup: One with true OIMF with spherical harmonics up to degree 45 (SH 45) and one with up to degree 60 (SH 60). These assimilations mimic the more realistic scenario where the presumed proxies of the OIMF are imposed with lower spatial resolution as the true field. To examine the effect of temporal deviations, we consider a temporally noisy true OIMF. Again two further assimilations are studied: one with low-frequency noise (LF noise) and another one with high-frequency noise (HF noise). In both cases, the noise added at each position is randomly drawn from a gaussian distribution with a standard deviation equal to the STD of the true OIMF at the corresponding location. In the case of LF noise, it is added to the SH 30 imposed OIMF every 60 days and interpolated within the interval. However, to create HF noise, the noise is added daily to the SH 30 imposed OIMF. In doing so, the HF noise can be seen as a scenario where daily variations of the OIMF are not covered correctly by the imposed OIMF. In contrast, the LF noise mimics a case where the imposed field misses parts of monthly or longer variations in the true OIMF. The deviations between imposed and true OIMF are exemplarily illustrated in Fig. 9.
The results of all four considered assimilations with differing true OIMF are shown in Fig. 10. The presented results are taken from the improved assimilations using the prior covariance accounting for spatial correlations. Due to the deviations of the imposed and true OIMF, the true value of the scale factor is not precisely one at each location and time step anymore. Therefore, the results are presented directly as NRMSD (Eq. 8) between the rescaled and true OIMF anomalies. We used the magnetic field anomalies for the NRMSD calculation since our method is not sensitive to the mean values. Expectantly, the NRMSD shows the familiar pattern of large values over land and decreased values in oceanic areas. Despite the slightly incorrect imposed OIMF, large parts show significantly reduced NRMSD. Remember, at the beginning of the assimilation, the scale factor is set to zero, which corresponds to an NRMSD of one at each location. Overall, the NRMSD clearly evince more significant deviations for all assimilations compared to the previous ideal scenario (Fig. 6, where the \(|{\mathbf{k}}_{f} -1|\) corresponds to the NRMSD). In general, the spatial deviations exhibit a larger NRMSD than the considered temporal deviations. For the temporal deviations, the areas of decreased NRMSD extended more over the entire oceans. The assimilation results using a true OIMF with SH 60 compared to the version using SH 45 have a similar pattern, but the NRMSD gets larger. Likewise, the temporal deviations caused by high-frequency noise show a slightly higher NRSMD than those caused by low-frequency noise. However, when evaluating the results and comparing them to the ideal test case scenario, it must be taken into account that there is already a significant NRMSD between the imposed and the true OIMF. Moreover, it is vague to what extent the scale factor approach generally can correct these imposed deviations. In the context of this OSSE, this can be clarified by calculating a theoretically optimal scale factor for this scenario. At each grid point, this optimal scale factor is determined by a least-square-fit of the time series between the true and the scaled imposed field. In doing so, the optimal factor minimizes the NRMSD between the true and an optimal scaled imposed OIMF anomalies. We considered this optimal factor as the best possible result of the Kalmag assimilation.
We used this factor to illustrate the optimal achievable NRMSD. Exemplary for both the imposed NRMSD (deviation between imposed and true OIMF anomalies), as well as the optimal NRSMD (deviation between optimal rescaled and true OIMF anomalies), is shown for the assimilation using the true OIMF with SH 45 in Fig. 11. From this figure, it can be seen that both strongly coincide with each other. We draw two conclusions from this: First, the imposed deviations in this scenario can hardly be corrected by any scale factor, which also emphasizes a limitation of the scale factor approach. Second, the increased NRMSD compared to the ideal scenario is primarily explainable by the imposed NRMSD. Thus, in this test case scenario, the imposed NRMSD could arguably be considered a lower limit of the achievable NRMSD of the rescaled field. Apart from that, the resulting scale factor from the assimilation is still a measure of detectability and how well the imposed signal can be recovered from the artificial observational data.
Finally, Fig. 12 summarizes the averaged NRMSD of all assimilations considered in test case scenario B and compares them to the imposed NRMSD. This figure underlines, that more significant deviations between the true and the imposed OIMF result in a larger NRMSD between the rescaled and true field after the assimilation (e.g., lower NRMSD for temporal deviations compared to spatial ones). In all cases, the resulting NRMSD after the assimilation is very close to the imposed NRMSD. Moreover, the average imposed NRMSD is within the uncertainty range of the results. We conclude that the proposed method allows a reasonable scale factor determination regardless of slight deviations between the imposed and true OIMF. Furthermore, Fig. 12 presents the assimilation results with and without imposed a priori spatial correlations. This comparison confirms the benefits of accounting for prior spatial correlations, with an averaged NRMSD slightly smaller in all test cases. From Fig. 12, it is also clearly visible that the usage of prior correlations reduces the final uncertainty of the scale factors and the resulting NRMSD, indicating an overall better factor determination.
We summarize from test scenario B that the Kalman filter algorithm determines an appropriate scale factor despite deviations in the presumed proxies. As a result, the final rescaled OIMF reflects the true OIMF as comparably good as the imposed one. Consequently, larger deviations in the presumed proxies result in a poorer correspondence between rescaled and true OIMF. Apart from that, the results indicate robustness to all sources of deviations (different spatial resolutions as well as low and high-frequency noise). Lastly, the posterior standard deviation proves to be a suitable measure for the uncertainty again, and the use of a prior correlation information improves the results in all cases.
Test case scenario C: deviating conductivity of the imposed magnetic field
This last test case scenario demonstrates a possible application of the presented method and is used further to explore the limitations of the scale factor approach. Moreover, this scenario provides an outlook on how the scale factor can find possible use beyond its measure of detectability.
Again, we examine deviations between the true and imposed fields. We kept the artificial observations the same as in the ideal scenario. However, this time, we slightly changed the presumed proxies by using a different ocean conductivity for their calculations. Instead of deriving the ocean conductivity from the ECCO2 model, we took the annual mean conductivity from the World Ocean Atlas 2018 (WOA18) dataset (Tyler et al. 2017). We chose the 3D ocean conductivity version with a spatial resolution of \(1^{\circ }\) and adapted it to the vertical layers of the ECCO2 velocities. All other calculation steps remained unchanged. Thus, the imposed OIMF deviates from the true OIMF only due to the different ocean conductivity. The assimilation is then performed like in the previous test case scenario using prior spatial correlations. The results of this assimilation are present in Fig. 13 as NRMSD of the rescaled and the true OIMF anomalies and the associated uncertainty of the scale factor. Clearly, the NRMSD (top left plot in Fig. 13) is decreased over the oceanic areas but, in general, more extensive compared to the ideal test case scenario (see Fig. 6). However, the low uncertainty of the scale factor (top right plot in Fig. 13) indicates an appropriate scale factor determination over the oceanic areas in this test case scenario.
Eventually, we used this test case scenario to investigate if the presumed proxies assuming a different conductivity could be corrected by assimilating artificial observational data. In this case, the deviations of the imposed field should be reduced by the determined scale factor. Therefore, the differences between the imposed NRMSD (between true and imposed OIMF anomalies) and the rescaled NRMSD (between true and rescaled OIMF anomalies) are considered. This is shown as assimilation improvement in Fig. 13 (bottom left plot). The figure points out areas (marked in red) where the rescaled OIMF anomalies are in better accordance with the true OIMF anomalies than the imposed ones. At these locations (e.g., along the East Asian coast, the Gulf of Mexico, southeast of Australia, and southwest of Africa), the imposed OIMF anomalies are adequately corrected by the scale factor resulting from the assimilation. On the downside, in some areas (marked in blue), the imposed OIMF anomalies could not be improved through the scale factor. However, especially over land, it is not expected to obtain such a scale factor from the assimilation (note again that the assimilation initially assumes a scale factor of zero). To better evaluate these assimilation improvements, we compare them again with a theoretical optimal scaling of the imposed OIMF. For this purpose, an optimal factor was calculated as described in the previous section to identify the best possible effect due to a scale factor correction. As before, the optimal improvement is described as the difference between the imposed NRMSD and the optimal rescaled NRMSD, indicating how much the imposed deviations can be reduced by optimal rescaling. This optimal improvement is presented in Fig. 13 (bottom right plot). In large areas, the best possible improvement is close to zero, which implies that even an optimal scale factor cannot correct the imposed deviations. But remarkably, the areas of significant improvement through optimal rescaling coincide exceptionally well with the pattern of improvement through assimilation. Additionally, oceanic areas where no improvement could be achieved by assimilation overlap with areas in which also an optimal factor does not lead to any improvement. This evaluation once again demonstrates the optimality of the scale factor determination. We conclude from this test case scenario that the assimilation results in a reasonable scale factor and that its application can possibly be used to obtain valuable information from observational data in the future.
On the applicability to real observations
With regard to the application of the method on real Swarm satellite data, we compared artificial satellite magnetometer data used in the presented OSSE with the real observations from the Swarm satellite mission. Therefore, we subtracted the core and lithosphere components of the Kalmag model from both datasets. The comparison of the remaining magnetic field measurements is presented in Fig. 14 exemplary for the year 2014. The residuals of all three vector components (\(B_r\), \(B_\theta\), \(B_\phi\)) of each 100th measurement are shown in dependence on the satellite latitude position. On the one hand, the comparison shows that the residuals of both artificial and Swarm magnetic field measurements are in the same order of magnitude and similar shape. On the other hand, Fig. 14 discloses differences between the artificial and the real Swarm measurements. The variability of the residuals in the artificial data, especially in the radial component, is larger around the equator and lower than than the variability in the Swarm observations at higher latitudes in the theta and phi components. The lower variability in the higher latitudes of the non-radial components could be well explained by missing field-aligned currents, which are not considered when building the synthetic data set. The increased variability of the artificial data residuals at lower latitudes, particularly the radial component, could indicate an overestimation in the prior of the ionosphere component around the equator. Even though Baerenzung et al. (2022) demonstrated that the ionosphere-induced magnetic source of the Kalmag model is consistent within induction processes, the deviations that occurred between the artificial and the real Swarm satellite data may require further investigation.
Apart from the differences between the artificial and the real Swarm data, another issue arises when it comes to application on real data. In contrast to the OSSE discussed in this paper the induced magnetic signal contained in the real satellite data is not known a priori. Although the analysis of the proposed method indicated a certain robustness and a reasonable scale factor determination, the results of this paper rely purely on the ECCO2 transport estimations. Consequently, the imposed signals and the signals included in the artificial measurements are still based on the same model. However, the oceanic transport predictions and, therefore, the OIMF anomalies may vary more widely. A potential incompatibility of the imposed and the real signal in the Swarm satellite data could affect the assimilation and scale factor determination to a larger extent. Towards the application of the presented method on real Swarm data, the authors propose to expand the variety of imposed magnetic fields and analyze potentially larger incompatibilities by taking into account also different ocean models or observational data.