A simple method to evaluate the uncertainty of magnetotelluric forward modeling for practical three-dimensional conductivity structure models

Abstract

The forward calculation of magnetotelluric (MT) responses is generally assumed to be sufficiently accurate compared with typical observational errors in practical modeling and inversion studies. Although the uncertainty of the forward calculation may be examined by comparison with analytical or other numerical solutions for some simple models, such an examination does not guarantee that the uncertainty is similar for more realistic complex structures. In this study, I propose a simple method to evaluate the uncertainty of MT forward modeling for practical three-dimensional (3D) conductivity structure models in a Cartesian coordinate system. The method is based on the idea that the horizontal coordinate system can be selected arbitrarily for a general 3D structure. The synthesized MT responses are ideally identical irrespective of the selection but are different because of the difference in discretization angles, boundary values, and numerical errors. By synthesizing MT responses to the model in several different coordinate systems, the mean, standard deviation, and coefficient of variation can be calculated. These statistics provide quantitative information on how stably the forward calculations synthesize MT responses under the given conditions of the structure model, observation array, periods, numerical algorithm for the forward modeling, and mesh design. The proposed method was applied to two practical situations of seafloor MT arrays in the northwestern Pacific and southern Atlantic and a land MT array in Hokkaido, Japan. The results show that the uncertainty is comparable to real observation errors and is significantly dependent on the MT impedance element, period, site, structure model, and horizontal coordinate system. The uncertainty of the forward calculation should be considered for each element, period, and site to quantitatively evaluate how well a given model explains the data. I propose a new root-mean-square in which the residuals are normalized by both the standard errors of the MT responses observed and synthesized. This would help avoid overfitting data in the inversion analysis by ignoring the uncertainty of the forward calculation. This method is also useful for testing the appropriate selection of the coordinate system and mesh design.

Graphical Abstract

Introduction

The magnetotelluric (MT) method is utilized to image the electrical conductivity structure of the Earth’s interior. The MT impedance tensor is defined as the frequency response of the horizontal electric field variations to the horizontal magnetic field variations at the observation site:

$$\left[ {\begin{array}{*{20}c} {E_{x} \left( {f,{\mathbf{r}}} \right)} \\ {E_{y} \left( {f, {\mathbf{r}}} \right)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {Z_{xx} \left( {f, {\mathbf{r}}} \right)} \, {Z_{xy} \left( {f,{\mathbf{r}}} \right)} \\ {Z_{yx} \left( {f,{\mathbf{r}}} \right)} \,{Z_{yy} \left( {f,{\mathbf{r}}} \right)} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {B_{x} \left( {f,{\mathbf{r}}} \right)} \\ {B_{y} \left( {f,{\mathbf{r}}} \right)} \\ \end{array} } \right],$$

(1)

where $E_{x}$, $E_{y}$, $B_{x}$ and $B_{y}$ are the horizontal components of the electric and magnetic fields, respectively, and $Z_{xx}$, $Z_{xy}$, $Z_{yx}$, and $Z_{yy}$ are the four elements of the MT impedance tensor in the Cartesian coordinate system, which is simply called the MT response. These are functions of the frequency $f$ (or its inverse, period $T$) and position ${\mathbf{r}}$. Electrical conductivity structure models can be obtained by trial-and-error forward modeling and/or inversion of the MT responses. Imaging three-dimensional (3D) conductivity structure is now more common because of the development of numerical methods to solve forward and inverse problems (e.g., Siripunvaraporn et al. 2005; Kelbert et al. 2014; Usui 2015) and the acquisition of data by array observations (e.g., Patro and Egbert 2008; Baba et al. 2010; Dong et al. 2013; Robertson et al. 2016).

Evaluating how well an electrical conductivity structure model explains the observed MT responses is critical for justifying the reliability of the model and its interpretation. One of the most popular methods for quantitative evaluation involves calculating the root-mean-square (RMS) of the residuals between the MT responses observed and synthesized from the model by forward modeling, as shown in (2). The residuals are generally normalized by the standard error of the observed MT responses:

$${\text{RMS}}_{1} = \sqrt {\frac{1}{2N}\mathop \sum \limits_{i = 1}^{N} \frac{{\left| {Z_{i}^{{{\text{obs}}}} - Z_{i}^{{{\text{syn}}}} } \right|^{2} }}{{\left( {\varepsilon_{i}^{{{\text{obs}}}} } \right)^{2} }}} ,$$

(2)

where $Z_{i}^{{{\text{obs}}}}$ and $Z_{i}^{{{\text{syn}}}}$ denote the MT impedances observed and synthesized, respectively, for a particular tensor element, frequency (period), and observation site. $\varepsilon_{i}^{{{\text{obs}}}}$ is the standard error of the observed MT response. $i$ counts the number of MT impedance elements, wherein the total number of independent variables is $2N$, because MT impedance is a complex value. The evaluation using ${\text{RMS}}_{1}$ implicitly assumes that $Z_{i}^{{{\text{syn}}}}$ is more accurately synthesized than $\varepsilon_{i}^{{{\text{obs}}}}$. However, this assumption is not always valid.

The uncertainty of the forward calculation may be separated into a bias component, which is systematical offset from the true value, and a random component, which arises from random fluctuations to the true value. Developers of numerical methods and modelers frequently discuss the uncertainty of the forward solutions by comparing with the analytical solution to a simple structure, or by comparing with other numerical solutions to a simple numerical model structure (e.g., Miensopust et al. 2013). These comparisons show that the calculated MT responses are different at some level from the analytical solution (or other numerical solutions), depending on the numerical algorithm and discretization, among other factors. The differences observed in these comparisons indicate the bias component of the uncertainty of the forward calculations. However, whether the uncertainty indicated by the comparisons for such simple models holds true for more practical complex-structure models, which are of interest to MT practitioners, is unclear. The random component of the uncertainty is an indication of how stably the MT responses can be calculated under the given conditions, such as numerical algorithms and mesh discretization. This study focuses on the random component of the uncertainty of forward modeling to practical 3D structure models, which has not been extensively studied in the literature. Hereafter, the term “uncertainty” refers to the random component of uncertainty, unless stated otherwise.

In the following, I describe a simple method for evaluating the random component of the forward modeling uncertainty in a practical 3D conductivity structure model. I demonstrate the details of the method and its application to two practical cases in “Methods” and “Applications” sections, respectively. Then, I discuss the advantages of the method, the implications from the two applications, a new quantitative evaluation, and some other conveniences of using this method before concluding.

Methods

Horizontal axes may be selected in arbitrary directions for a 3D structure, wherein synthesized MT responses should be ideally invariant but in practice are dependent on the orientation. Suppose a 3D conductivity structure and observation array. First, the model dimension (i.e., how large a volume is included in the numerical calculation) and discretization (i.e., how fine or coarse a model is discretized into numerical blocks or elements) in the $x$-, $y$-, and $z$-directions, respectively, are determined to express numerically the 3D conductivity structure. The 3D structure is then discretized in the same manner in $M$ different coordinate systems, in which the azimuth of the $x$-axis is randomly selected from $- 90^\circ$ to $+ 90^\circ$. Each horizontal coordinate system and the azimuth of the $x$-axis are denoted as $\left( {x_{j} ,y_{j} } \right)$ and $\theta_{j}$, respectively, where $j = 1, 2, \ldots , M$. MT impedances in the array are synthesized by forward calculations for the $M$ models discretized in each different coordinate system.

The MT impedance tensor at a site and frequency is changed by the rotation of the horizontal coordinate system from $\left( {x,y} \right)$ to $\left( {x^{\prime},y^{\prime}} \right)$ with a rotation angle $\varphi$:

$$\left[ {\begin{array}{*{20}c} {Z_{{x^{\prime}x^{\prime}}} } \, {Z_{{x^{\prime}y^{\prime}}} } \\ {Z_{{y^{\prime}x^{\prime}}} }\, {Z_{{y^{\prime}y^{\prime}}} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} { \quad \cos \varphi } \,\,{\sin \varphi } \\ { - \sin \varphi } \,\, {\cos \varphi } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {Z_{xx} } \, {Z_{xy} } \\ {Z_{yx} } \,{Z_{yy} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {\cos \varphi } { - \sin \varphi } \\ {\sin \varphi } { \quad \cos \varphi } \\ \end{array} } \right].$$

(3)

Each tensor element is written as

$$Z_{{x^{\prime}x^{\prime}}} = \quad \frac{1}{2}\left( {Z_{xx} + Z_{yy} } \right) + \frac{1}{2}\left( {Z_{xx} - Z_{yy} } \right)\cos 2\varphi + \frac{1}{2}\left( {Z_{xy} + Z_{yx} } \right)\sin 2\varphi ,$$

(4)

$$Z_{{x^{\prime}y^{\prime}}} = \quad \frac{1}{2}\left( {Z_{xy} - Z_{yx} } \right) + \frac{1}{2}\left( {Z_{xy} + Z_{yx} } \right)\cos 2\varphi - \frac{1}{2}\left( {Z_{xx} - Z_{yy} } \right)\sin 2\varphi ,$$

(5)

$$Z_{{y^{\prime}x^{\prime}}} = - \frac{1}{2}\left( {Z_{xy} - Z_{yx} } \right) + \frac{1}{2}\left( {Z_{xy} + Z_{yx} } \right)\cos 2\varphi - \frac{1}{2}\left( {Z_{xx} - Z_{yy} } \right)\sin 2\varphi ,$$

(6)

$$Z_{{y^{\prime}y^{\prime}}} = \quad \frac{1}{2}\left( {Z_{xx} + Z_{yy} } \right) - \frac{1}{2}\left( {Z_{xx} - Z_{yy} } \right)\cos 2\varphi - \frac{1}{2}\left( {Z_{xy} + Z_{yx} } \right)\sin 2\varphi .$$

(7)

The first term on the right-hand side of Eqs. (4)–(7) is invariant to the rotation angle:

$$Z_{{{\text{tr}}}} = \frac{1}{2}\left( {Z_{xx} + Z_{yy} } \right) = \frac{1}{2}\left( {Z_{{x^{\prime}x^{\prime}}} + Z_{{y^{\prime}y^{\prime}}} } \right),$$

(8)

$$Z_{{{\text{sk}}}} = \frac{1}{2}\left( {Z_{xy} - Z_{yx} } \right) = \frac{1}{2}\left( {Z_{{x^{\prime}y^{\prime}}} - Z_{{y^{\prime}x^{\prime}}} } \right),$$

(9)

where the subscripts tr and sk represent the matrix trace (sum of the diagonal elements) and skew (difference of the off-diagonal elements), respectively. Therefore, $Z_{{{\text{tr}}}}$ or $Z_{{{\text{sk}}}}$ calculated from the MT response for the $M$ models should ideally be identical. The diagonal and off-diagonal elements of the MT impedance tensor vary with respect to the rotation angle. One can draw a trajectory of any element of the MT impedance tensor to the rotation angle by rotating them with Eqs. (4)–(7) for various rotation angles. The trajectory for the $M$ models should ideally be identical to each other. If one selects a coordinate system, for example, the $x$-axis pointing in the north direction $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$, one can collect $M$ samples of the four elements of the MT impedance tensor, $Z_{{x_{{\text{N}}} x_{{\text{N}}} }}$, $Z_{{x_{{\text{N}}} y_{{\text{E}}} }}$, $Z_{{y_{{\text{E}}} x_{{\text{N}}} }}$, and $Z_{{y_{{\text{E}}} y_{{\text{E}}} }}$, for a site and a frequency in the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system by rotating the $j$th MT response $- \theta_{j}$. The resultant MT responses should ideally be identical.

In practice, the rotation invariants, trajectories, and rotated MT impedance tensors are not identical between $M$ models. The deviations can arise from the difference in how the numerical grid of the mesh represents the 3D conductivity structure, the difference in the structure included near the lateral model edges, and thus the lateral boundary values (although the horizontal model dimension is generally set as large as this effect becomes negligible) and numerical errors in the forward calculation, which can be dependent on how conductivity contrast are captured in the numerical grid. Subsequently, the mean, standard deviation, and coefficient of variation of the $M$ samples for the rotational invariants or each element of the rotated MT response can be evaluated for each site (position) and frequency:

$$\mu_{X} \left( {f,{\mathbf{r}}} \right) = \frac{1}{M}\mathop \sum \limits_{j = 1}^{M} Z_{X,j} \left( {f,{\mathbf{r}}} \right),$$

(10)

$$\sigma_{X} \left( {f,{\mathbf{r}}} \right) = \sqrt {\frac{1}{M - 1}\mathop \sum \limits_{j = 1}^{M} \left| {Z_{X,j} \left( {f,{\mathbf{r}}} \right) - \mu_{X} \left( {f,{\mathbf{r}}} \right)} \right|^{2} } ,$$

(11)

$${\text{CV}}_{X} \left( {f,{\mathbf{r}}} \right) = \frac{{\sigma_{X} \left( {f,{\mathbf{r}}} \right)}}{{\left| {\mu_{X} \left( {f,{\mathbf{r}}} \right)} \right|}},$$

(12)

where the subscript $X$ denotes tr, sk, or elements of the rotated MT impedance tensor ($x_{{\text{N}}} x_{{\text{N}}}$, $x_{{\text{N}}} y_{{\text{E}}}$, $y_{{\text{E}}} x_{{\text{N}}}$, $y_{{\text{E}}} y_{{\text{E}}}$). $\sigma$ can be used as an indicator of the random component of a 3D forward calculation. ${\text{CV}}$ is useful for evaluating how $\sigma$ is significant compared to the magnitude of $\mu$.

Applications

The methods introduced in the previous section were applied to two cases based on actual seafloor MT observations (Fig. 1) and one case based on land MT observation (Additional file 1: Fig. S1). The first one was an array observation of a relatively flat ocean basin in the northwestern (NW) Pacific (Baba et al. 2017b). There were two arrays in the northwest and southeast regions of the Shatsky Rise. I chose a northwestern array, consisting of 16 available MT sites. The second one was an array observation around the Tristan da Cunha (TDC) islands in the southern (S) Atlantic (Baba et al. 2017a), consisting of 24 available MT sites. The third one was a land MT array in Hokkaido, Japan (Ichihara et al. 2021), which I intended to demonstrate the applicability of the method to more general cases. The detail is given in Additional file 1.

Fig. 1

Maps for the seafloor MT observation arrays in a NW Pacific (Baba et al. 2017b) and b S Atlantic (Baba et al. 2017a). Color indicates the bathymetry. The location of the array in the globe is indicated by a red rectangle in the inset map

The 3D structural models considered for the applications to the marine MT arrays consist of 3D topography over a one-dimensional (1D) mantle structure (Figs. 2 and 3). Topography and bathymetry are complex 3D structures that can affect seafloor MT responses significantly; therefore, they should be a good example for this application. A 1D structure was assumed beneath the seafloor for the simplicity of implementation and discussion of the topographic effect on MT responses, although the method can be applied to any 3D structure model as shown in Additional file 1. The 1D models were taken from those estimated in the two areas by Baba et al. (2017a, b).

Fig. 2

Topography and 1D mantle structure models for NW Pacific. a Regional large-scale topography model (top) and local small-scale topography model for a site NM04 (bottom), discretized in the $\left( {x_{1} ,y_{1} } \right)$ coordinate system ($\theta_{1} = - 55$°). North is up. Color indicates the bathymetry. The observation array or site locates the center of the regional or local model as indicated by a circle. b Same as a but in the $\left( {x_{6} ,y_{6} } \right)$ coordinate system ($\theta_{6} = 15$°). c 1D electrical conductivity structure model after Baba et al. (2017b)

Fig. 3

Topography and 1D mantle structure models for S Atlantic. a Regional large-scale topography model (top) and local small-scale topography model for site Tris11 (bottom), discretized in the $\left( {x_{4} ,y_{4} } \right)$ coordinate system ($\theta_{4} = - 20$°). North is up. Color indicates the bathymetry. The observation array or site locates the center of the regional or local model as indicated by a circle. b Same as a but in the $\left( {x_{8} ,y_{8} } \right)$ coordinate system ($\theta_{8} = 34$°). c 1D electrical conductivity structure model after Baba et al. (2017a)

The 3D forward modeling algorithm applied here was FS3D, which can effectively incorporate bathymetric change into a numerical model based on a finite difference method (Baba and Seama 2002), although the evaluation method of the uncertainty itself should be free from the choice of numerical algorithm, and thus any forward modeling code can be used according to the user’s preference [in the additional example described in Additional file 1, I applied a modified version of WSINV3DMT (Siripunvaraporn et al. 2005; Tada et al. 2012)]. A two-stage modeling approach has been used to effectively model the effects of large-scale regional topography and small-scale local topography (Baba et al. 2013). The large-scale regional topography was modeled for an area of $10,000 \times 10,000$ km² for NW Pacific and an area of 9356 × 9356 km² for the S Atlantic, centering the observation array. In the vicinity of the array, the horizontal mesh dimensions were $50 \times 50$ km² and $25 \times 25$ km², and the total number of meshes in the horizontal plane was $59 \times 59$ and $61 \times 61$ for the NW Pacific and S Atlantic cases, respectively. The local small-scale topography was modeled for an area of $350 \times 350$ km² around each site. The horizontal mesh dimensions were $1 \times 1$ km² in the vicinity of the site. The total number of meshes in the horizontal plane was $37 \times 37$ for both the cases. The average depth within the horizontal mesh was incorporated into the numerical model to represent the depth of the mesh area. The magnetic field calculated in the regional model at the first stage was used as the initial and boundary values for the local model in the second stage to obtain the field in which both effects were considered (Baba et al. 2013). The MT responses at 24 periods between 53.3 and 163,840 s were synthesized by forward modeling. The periods were based on those of the observed MT responses. I set $M = 10$ for these applications. The azimuth of $x$-axis was then randomly selected for each application, as listed in Table 1.

Table 1 Azimuths of the $x$-axis of the Cartesian coordinate system for which forward modeling was conducted for each application

In the following, I describe the major features of the 3D model first and then I evaluate the uncertainty of the forward calculations through three kinds of figures, respectively, for the two applications; (1) the MT sounding curves for the two rotational invariants, $Z_{{{\text{tr}}}}$ and $Z_{{{\text{sk}}}}$, and the four elements of the impedance tensor rotated back to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system, $Z_{{x_{{\text{N}}} x_{{\text{N}}} }}$, $Z_{{x_{{\text{N}}} y_{{\text{E}}} }}$, $Z_{{y_{{\text{E}}} x_{{\text{N}}} }}$, and $Z_{{y_{{\text{E}}} y_{{\text{E}}} }}$, for a selected site. These give information how the uncertainty varies with the rotational invariants or the impedance tensor elements and with the period. (2) Scatters of the coefficients of variation for the two rotational invariants for all sites. I demonstrate how the uncertainty for each invariant varies with the sites. (3) Trajectories of $xy$ and $xx$ elements of a selected site and selected periods to the azimuthal angle of the $x$-axis for the rotation of the coordinate system. It is useful to see how the uncertainty of each MT impedance tensor element varies depending on the coordinate system.

The first example is the NW Pacific seafloor. The area is ~ 1100 km away from the Japanese coastlines. Here, the local bathymetric changes are relatively small and are mainly related to the ENE–WSW trend abyssal hills and valleys associated with the paleo-spreading ridge system and a few small seamounts (Fig. 1a). Baba et al. (2017b) demonstrated that the observed MT responses were significantly affected by coastlines west of the observation array. Figure 2a, b shows examples of regional and local topography models discretized in two different coordinate systems for MT forward modeling. The regional topography model includes the Japanese islands and the Asian continent, and the local topography model includes the abyssal hills, valleys, and nearby seamounts. These topographic features were discretized differently, depending on the azimuth of the coordinate system.

The two rotational invariants calculated for the 10 models generally agreed well with each other. Figure 4a shows the invariants at site NM04 as sounding curves of their real and imaginary parts scaled by the square root of the period. ${\text{CV}}_{{{\text{sk}}}}$ was at most 1.4% and was less than 1% in most of the periods. $Z_{{{\text{tr}}}}$ was very close to zero, suggesting the three dimensionality is small, and ${\text{CV}}_{{{\text{tr}}}}$ was more than ~ 20%. These trends were common to all sites. Figure 5 shows the relationship between $\mu$ and ${\text{CV}}$ for the rotational invariants at all sites. ${\text{CV}}_{{{\text{sk}}}}$ was mostly less than 1%, and it was ~ 1.5 to ~ 3 orders smaller than ${\text{CV}}_{{{\text{tr}}}}$. ${\text{CV}}_{{{\text{tr}}}}$ tended to increase as the period decreases, and was also large at approximately 10,000 s at sites NM13, 14, and 15. These trends were related to the magnitude of $\mu_{{{\text{tr}}}}$, because $\log {\text{CV}}_{{{\text{tr}}}}$ was negatively correlated with $\log \sqrt T \left| {\mu_{{{\text{tr}}}} } \right|$. ${\text{CV}}_{{{\text{tr}}}}$ over 100% did not mean that the forward calculation was inaccurate, but that $\left| {\mu_{{{\text{tr}}}} } \right|$ was too small to distinguish from zero (i.e., three-dimensionality is negligible). In contrast, ${\text{CV}}_{{{\text{sk}}}}$ was relatively high at approximately 1000 s and was positively correlated with the magnitude of $\mu_{{{\text{sk}}}}$. There was a position-dependent trend in which ${\text{CV}}_{{{\text{tr}}}}$ was lower and ${\text{CV}}_{{{\text{sk}}}}$ was higher at the western sites.

Fig. 4

MT sounding curves scaled by the square root of the period and the coefficient of variation of a rotational invariants and b impedance elements rotated to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system for site NM04 of the NW Pacific models. Colored lines are the sounding curves for the ten models. Symbol with error bar indicates the mean and the standard deviation of the ten samples

Fig. 5

Scatter plots of ${\text{CV}}_{{{\text{sk}}}}$ and ${\text{CV}}_{{{\text{tr}}}}$ with respect to the period and the magnitude of $\mu_{{{\text{sk}}}}$ and $\mu_{{{\text{tr}}}}$ scaled by square root of the period for the NW Pacific models. Symbols filled with black indicate the site that the sounding curves and the trajectories of the MT response to the azimuth are demonstrated in Figs. 4 and 6, respectively

The synthesized MT impedance elements from the 10 models can be directly compared by rotating the responses to a common coordinate system. The trajectories by rotation demonstrate that the variance (and thus the standard deviation and the coefficient of variation) was not constant with the azimuth of the coordinate system (Fig. 6). Around the azimuths resulting in locally minimum the diagonal elements, which is approximately $- 50^\circ$ and $+ 40^\circ$ in the NW Pacific case, their magnitude was close to that of $Z_{{{\text{tr}}}}$, and the variance became larger. The off-diagonal elements show the largest split around the azimuths ($Z_{xy}$ shifted by 90° corresponds to $Z_{yx}$ in the trajectory), and the variance of either $Z_{xy}$ or $Z_{yx}$ was the smallest, while the other was the largest. In the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system, which the magnitude of the diagonal elements is close to its maximum with respect to the rotation angle in the NW Pacific case (Fig. 6b), the ${\text{CVs}}$ of the diagonal elements were mostly a few percent and less than ~ 10% in the shortest periods, while the ${\text{CVs}}$ of the off-diagonal elements were slightly higher or lower than ${\text{CV}}_{{{\text{sk}}}}$ depending on whether the magnitude was higher or lower than $\left| {\mu_{{{\text{sk}}}} } \right|$, as shown by the sounding curves for site NM04 (Fig. 4b).

Fig. 6

Trajectories (lines) of synthesized a $Z_{{x_{j} y_{j} }}$ and b $Z_{{x_{j} x_{j} }}$ (circles) with respect to the azimuth for selected three periods at site NM04 of the NW Pacific models. The values are normalized by $\left| {\mu_{{{\text{sk}}}} } \right|$ or $\left| {\mu_{{{\text{tr}}}} } \right|$. Gray shades indicate the range of $1 \pm {\text{CV}}_{{{\text{sk}}}}$ or $1 \pm {\text{CV}}_{{{\text{tr}}}}$. Colors correspond to those in Fig. 4

The second example is the S Atlantic seafloor. The observation sites were distributed around the TDC islands, which are the remotest islands ~ 2800 km away from the nearest South African coast. In contrast to the NW Pacific case, Baba et al. (2017a) demonstrated that the continental coast effect on the observed MT responses was small, but a strong topographic effect was evident for sites near the TDC islands and other large seamounts. As shown in Fig. 3a, b, the regional topography model includes major topographic changes, such as continents, the Mid-Atlantic Ridge, and the Walvis ridge, which is a volcanic chain terminating at the TDC islands. The TDC islands were incorporated into the local topography models more precisely using a finer mesh. These topographic features were discretized differently depending on the azimuth of the coordinate system in the same manner as in the NW Pacific case but for different 10 azimuths (Table 1).

Evident cusps in the MT impedance were synthesized for the Tris05, 11, 12, 18, and 26 sites. These sites were in close proximity to the TDC islands or a large seamount south of the TDC islands (Fig. 1b). Figure 7 shows the sounding curves at site Tris11 as an example of sites showing strong local topographic effects. ${\text{CV}}_{{{\text{sk}}}}$ was less than 1% in the periods longer than ~ 10,000 s, but had a cusp of approximately 30% around 400 s. ${\text{CV}}_{{{\text{tr}}}}$ was approximately 100% in the periods longer than 1000 s, and increased more in the shorter period range. For the MT impedance elements rotated to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system, the magnitude of the diagonal elements was larger than that of $Z_{{{\text{tr}}}}$, and the ${\text{CVs}}$ were a few tens of percent in the longer period range. The ${\text{CVs}}$ for all four elements showed a cusp of approximately 30%. The MT responses for the remaining sites were similar to those in the NW Pacific.

Fig. 7

MT sounding curves scaled by square root of the period and coefficient of variation of a rotational invariants and b impedance elements rotated to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system for site Tris11 of the S Atlantic models. Colored lines are the sounding curves for the 10 models. Symbol with error bar indicates the mean and the standard deviation of the 10 samples

The general trends of ${\text{CV}}_{{{\text{tr}}}}$ and ${\text{CV}}_{{{\text{sk}}}}$ for all sites of the S Atlantic were similar to those of the NW Pacific case, except for larger scatters, which are attributed to the sites showing strong local topographic effects (Fig. 8). The ${\text{CV}}_{{{\text{tr}}}}$ tended to increase with the magnitude of $\mu_{{{\text{tr}}}}$ for each site with cusps. By contrast, the general trend of ${\text{CV}}_{{{\text{tr}}}}$ over the sites tended to decrease. The trajectories by rotation demonstrate once again that the variance is not constant with the azimuth of the coordinate system, especially in the off-diagonal elements for the periods around the cusp (Fig. 9). For site Tirs11, the diagonal elements became the minimum, and the split between the off-diagonal elements reached a maximum when the $x$-axis was directed to approximately $+ 30^\circ$ and $- 60^\circ$. The variance was the smallest for only one of the off-diagonal elements and the largest for the other three elements. However, in contrast to the case of site NM04 in the NW Pacific, the off-diagonal element with a magnitude smaller than $\left| {\mu_{{{\text{sk}}}} } \right|$ showed a smaller variance.

Fig. 8

Scatter plots of ${\text{CV}}_{{{\text{sk}}}}$ and ${\text{CV}}_{{{\text{tr}}}}$ with respect to the period and the magnitude of $\mu_{{{\text{sk}}}}$ and $\mu_{{{\text{tr}}}}$ scaled by square root of the period for the S Atlantic models. Symbols filled with black indicate the site that the sounding curves and the trajectories of the MT response to the azimuth are demonstrated in Figs. 7 and 9, respectively

Fig. 9

Trajectories (lines) of synthesized a $Z_{{x_{j} y_{j} }}$ and b $Z_{{x_{j} x_{j} }}$ (circles) with respect to the azimuth for selected three periods at site Tris11 of the S Atlantic models. The values are normalized by $\left| {\mu_{{{\text{sk}}}} } \right|$ or $\left| {\mu_{{{\text{tr}}}} } \right|$. Gray shades indicate the range of $1 \pm {\text{CV}}_{{{\text{sk}}}}$ or $1 \pm {\text{CV}}_{{{\text{tr}}}}$. Colors correspond to those in Fig. 7

Discussion

Key merits and findings of the proposed method

In this study, the method proposed allows to quantitatively evaluate how stably MT responses can be synthesized using forward calculations. This method is simple and easy to implement, although it requires time to run several forward calculations. It has broad utility, as it can be applied to any numerical forward algorithm or 3D structure models as shown above and in Additional file 1. However, even if the forward calculation is confirmed to be very stable using this method, this does not guarantee the accuracy of the forward calculation, because the method does not provide information on the bias component of the uncertainty. Nevertheless, it would be useful to determine the partial uncertainty of the forward calculations for practical 3D conductivity structure models.

The two applications of the method showed that the coefficients of variation of the seafloor MT responses synthesized to a 3D topographic model using FS3D (Baba and Seama 2002; Baba et al. 2013) were between 0.1 and 10 percent for the off-diagonal elements, which is not negligible as they are relatively comparable with typical observational errors. The application to the land MT array with modified version of WSINV3DMT (Siripunvaraporn et al. 2005; Tada et al. 2012) also showed significant coefficient of variations in the MT responses synthesized to a 3D subsurface structure model (Additional file 1: Figs. S3–S5). There must be a dependence on the numerical algorithm, but there is no guarantee that other algorithms will always give better results than those presented in this study. In other words, one should not believe that the forward calculation is absolutely accurate. More importantly, the applications clearly demonstrated that the coefficients of variation vary significantly depending on the MT impedance element, period, site, structure model, and coordinate system (Figs. 4, 5, 6, 7, 8, 9 and Additional file 1: Figs. S3–S7), suggesting that presuming a constant value as a possible uncertainty in forward calculations is not reasonable. This is the key finding that affects some aspects in the following discussions.

Evaluation of data misfit

Here, the quantitative evaluation of how well a given 3D structure model explains the observed MT responses is given by considering the uncertainty of the forward calculations based on the proposed method. The standard deviation provides information on the extent to which the MT response synthesized by one of $M$ calculations can deviate from the mean. The information on the uncertainty of the mean estimated from $M$ forward calculations, that is, how much the sample mean can deviate from the population mean, can be given by the standard error defined as

$$\varepsilon^{{{\text{syn}}}} = \frac{\sigma }{\sqrt M }.$$

(13)

The standard error decreases with increasing $M$ and can, therefore, achieve a more reliable estimate of the mean with a larger $M$, in exchange for computation time. The value of $M$ should be selected based on this trade-off relationship. RMS may be calculated for the residual between the observed MT response $Z_{i}^{{{\text{obs}}}}$ and the mean of the synthesized MT responses $\mu_{i}$ normalized by the value related to the standard errors for both, $\varepsilon_{i}^{{{\text{obs}}}}$ and $\varepsilon_{i}^{{{\text{syn}}}}$, for $i$th data point:

$${\text{RMS}}_{2} = \sqrt {\frac{1}{2N}\mathop \sum \limits_{i = 1}^{N} \frac{{\left| {Z_{i}^{{{\text{obs}}}} - \mu_{i} } \right|^{2} }}{{\left( {\varepsilon_{i}^{{{\text{obs}}}} } \right)^{2} + \left( {\varepsilon_{i}^{{{\text{syn}}}} } \right)^{2} }}} .$$

(14)

Using ${\text{RMS}}_{2}$, a squared residual is less or more weighted if the mean of the synthesized MT response is less or more reliable as well as if the observed MT response is less or more reliable, respectively. The residuals should be evaluated more reasonably using ${\text{RMS}}_{2}$ than using ${\text{RMS}}_{1}$.

The impact of the normalization proposed above is demonstrated using the observed and synthesized MT responses for the S Atlantic case. The observed MT responses were obtained from Baba et al. (2017a), which include all available MT impedance tensor elements ($x_{{\text{N}}} x_{{\text{N}}}$, $x_{{\text{N}}} y_{{\text{E}}}$, $y_{{\text{E}}} x_{{\text{N}}}$, $y_{{\text{E}}} y_{{\text{E}}}$) at the 24 sites in maximum 24 (depending on the site) periods, i.e., $N = 1776$. Figure 10a shows histograms of the residuals normalized by the conventional and proposed methods. For the conventional case, I calculated the residuals between the observed MT responses that are given in the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system and the MT responses calculated for the 6th model that the $x$-axis is directed in the north direction (Table 1), and normalized them by $\varepsilon^{{{\text{obs}}}}$. The distribution showed a high concentration in the vicinity of zero and some outliers (the minimum and maximum values were $- 189.9$ and $+ 135.9$, respectively, far out of the plot range of Fig. 10a) compared to a normal distribution with a mean of zero and a variance of ${\text{RMS}}_{1}^{2}$, $N\left( {0,{\text{RMS}}_{1}^{2} } \right)$. In the new method, the residuals were calculated for the observed MT responses and the mean of the 10 synthetic MT responses rotated to $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system, and were normalized by $\sqrt {\left( {\varepsilon^{{{\text{obs}}}} } \right)^{2} + \left( {\varepsilon^{{{\text{syn}}}} } \right)^{2} }$. Although the distribution in the vicinity of zero was similar to that in the conventional case, the number of outliers was significantly reduced (the minimum and maximum values were $- 43.1$ and $+ 54.1$, respectively). As a result, the distribution was closer in shape to $N\left( {0,{\text{RMS}}_{2}^{2} } \right)$.

Fig. 10

Distribution and RMS of the residuals for the S Atlantic model. a Histograms of residuals normalized by the standard errors of the observed MT responses $\varepsilon^{{{\text{obs}}}}$ (top) and by $\sqrt {\left( {\varepsilon^{{{\text{obs}}}} } \right)^{2} + \left( {\varepsilon^{{{\text{syn}}}} } \right)^{2} }$ (bottom). Blue dashed lines indicate $\pm {\text{RMS}}_{1}$ and $\pm {\text{RMS}}_{2}$, red solid lines indicate the normal distributions that the mean is zero and the standard deviation is ${\text{RMS}}_{1}$ and ${\text{RMS}}_{2}$, respectively. b Sitewise ${\text{RMS}}_{1}$ (gray) and ${\text{RMS}}_{2}$ (red)

The uncertainty of the forward calculation varied significantly depending on the MT impedance element, period, and site, as shown in the two applications. The impact of site dependency on the model evaluation is shown in the sitewise RMS (Fig. 10b). ${\text{RMS}}_{1}$ for site Tris11 was extremely large compared to that of the other sites. Most outliers in the residuals mentioned above were residuals for Tris11. The large ${\text{RMS}}_{1}$ for Tris11 was primarily attributed to the large absolute residual, because the standard error of the observed response was not markedly different from other relatively good data. The MT response at Tris11 varied significantly with period because of the strong local topographic effect, and the synthesized response to the assumed model did not fit the data well. However, the uncertainty of the forward response was also large, especially in the periods around the cusp of the sounding curves (Fig. 7), resulting in ${\text{RMS}}_{2}$ for Tris11 being as small as for the other major sites (Fig. 10b). In the use of conventional evaluation, one must consider reducing the RMS for Tris11 to improve the total RMS; however, this is less meaningful considering the uncertainty of the forward calculation.

Perspectives of the proposed method in inversion analyses

Although the implementation of the proposed method for inversion is beyond the scope of this study, I propose some perspectives in inversion analysis. Suppose that a general iterative scheme of regularized inversion minimizes an objective function. A data misfit should be evaluated by considering the standard error of the forward calculations, as discussed above. In other words, the data covariance matrix in the data misfit term of the objective function, which is typically diagonal and consists of the squared standard errors of the observed MT responses in conventional method, should be expressed by the standard errors of both observed and synthesized MT responses in the proposed method. This means that the data covariance matrix is dependent on the model; thus, special treatment may be necessary in a precise sense. In addition, it is probably not practical in terms of computational time to conduct more forward calculations in each inversion iteration. A compromise may be to conduct forward calculations for the initial model and fix the covariance matrix during inversion, assuming that the standard error of the forward calculations does not change significantly with a small change in the conductivity model. The standard error of the forward calculations can be updated when the change in the model becomes significant and the updated objective function is minimized. I expect that the update would be necessary more frequently in the earlier stage when the change in the model is generally larger. It is important to avoid falling into the local minimum of the objective function because of overfitting to the data beyond the certainty of the forward calculations.

The error floor, which trims the observed error by a threshold value, is frequently applied in practical MT inversion analyses. ${\text{RMS}}_{2}$ may be an alternative to ${\text{RMS}}_{1}$ with an error floor. One of the basic motivations for introducing an error floor is to avoid overfitting data with unrealistically small error estimates. However, the criterion of “small” is not always evident. In many cases, it seems largely dependent on the experience of the users, without clear physical, statistical, or numerical evidence. Threshold values are rarely provided to each data point separately; rather, a common value or a relation is applied to particular MT impedance elements for all periods and sites, resulting in the loss of the relative importance of each data point, although there may be cases that are objective. Evaluating the residuals with normalization by standard errors for both the observation and forward calculation proposed above will work similarly to applying an error floor in terms of avoiding the overfitting problem. The advantage is that the evidence and meaning are clear for every data point.

The selection of the coordinate system can affect the model obtained by practical inversion analysis, although the selection is arbitrary for a 3D structure. For example, Tietze and Ritter (2013) reported an example and discussed the causes in terms of the data error interrelated between MT impedance tensor elements and sensitivity to the structure. This study suggests that the uncertainty of the forward calculation can also affect the inversion model variations, depending on the coordinate system. The application of the evaluation method proposed in this study showed that the variance of the synthesized MT responses varied with the rotation angle (Figs. 6 and 9). Therefore, overfitting beyond the certainty of the forward calculation can occur differently, depending on the coordinate system. The trajectory analysis for the rotation of the synthesized MT responses would be useful for discussing the appropriate coordinate system in terms of the uncertainty of the forward calculations.

Further merit in use of the proposed method

The proposed evaluation method is also useful for investigating the uncertainty depending on the mesh design, although the above discussion focuses on the uncertainty under a given mesh design. One of the supposed cases is to test whether known structures, such as topography, bathymetry, and/or geological setting, are appropriately incorporated into a conductivity model. Here, I present a test for local topographic effect modeling in terms of different mesh dimensions for site Tris11 in the S Atlantic. I modified the local topography model for the two-stage forward calculation (Baba et al. 2013) by applying a finer mesh, which expresses the relief of the TDC islands more precisely (Fig. 11a), and conducted second-stage modeling for the same 10 coordinate systems as in the previous modeling. Then, the means, standard deviations, and coefficients of variation of the MT responses rotated to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system were calculated. A comparison with those obtained from the forward calculations using the coarser mesh (Fig. 11b) showed that the standard deviations and coefficients of variation were mostly improved using the finer mesh except for the $x_{{\text{N}}} x_{{\text{N}}}$ element, although the improvement in the periods in which the MT response showed that the cusp was small for all elements. Furthermore, the mean responses for the coarser and finer meshes were found to agree within their standard deviations, indicating that the bias due to the difference in mesh dimension is less significant.

Fig. 11

a Local topography models for site Tris11 in S Atlantic discretized with relatively coarser (top) and finer (bottom) meshes in $\left( {x_{6} ,y_{6} } \right)$ coordinate system. b Synthesized MT response for site Tris11 from the models in 10 different coordinate systems (top and middle) and the coefficient of variation of each impedance tensor element (bottom). Symbols are the mean of the MT responses rotated to the $\left( {x_{{\text{N}}} ,y_{{\text{E}}} } \right)$ coordinate system and error bars are the standard deviations. Black and gray colors indicate those obtained from the finer and coarser mesh models, respectively. The gray symbols are slightly shifted to the right for visibility

Conclusions

A simple new method is proposed to evaluate the random component of the uncertainty of MT forward modeling to practical 3D conductivity structure models in a Cartesian coordinate system. The method is based on the notion that the horizontal coordinate system can be selected arbitrarily for a general 3D structure. Synthesized MT responses are ideally identical irrespective of the selection, but are different because of the difference in discretization angles, boundary values, and numerical errors. The mean, standard deviation, and coefficient of variation can be calculated by synthesizing the MT responses to the model in several coordinate systems. They provide quantitative information on how stably the forward calculations synthesize MT responses under the given conditions of the structure model, the position of the observation sites, the period range, the numerical algorithm for the forward modeling, and the mesh design. The advantages of this method include its easy implementation and its applicability to any 3D structure models and numerical modeling algorithms. By contrast, the disadvantage is that it is time consuming to conduct forward calculations several times.

In this study, the proposed method was applied to three practical situations of seafloor MT arrays in the NW Pacific and S Atlantic regions and a land MT array in Hokkaido, Japan. 3D models consisting of surface 3D topography and bathymetry over subsurface 1D layered structures given in previous studies were used for the marine MT arrays. A 3D structure without land topography was used for the land MT array. The results show that the uncertainty (standard deviations or coefficients of variation) was comparable with real observation errors and was significantly dependent on the MT impedance element, period, site, structural models, and horizontal coordinate system.

The uncertainty of the forward calculation should not be neglected, but should be considered for each element, period, and site to quantitatively evaluate how well a given model explains the data. In this context, a new RMS is proposed in which the residuals are normalized by both the standard errors of the observed and synthesized MT responses. This may be implemented in the inversion analysis.

This method is also useful for testing the appropriate selection of the coordinate system and mesh design. The trajectory of the MT impedance element to rotation in the horizontal coordinate demonstrates the variation in the standard deviations and coefficients of variation with the rotation angle, which will help to find a coordinate system in which the uncertainty is relatively small for all MT impedance elements, periods, and sites. A comparison of the mean, standard deviation, and coefficients of variation between different mesh designs enabled the evaluation of the difference in the stability of the forward calculations and in the bias. An advantage of this method is that one can evaluate whether the bias due to the difference in the mesh design is significant.