Effect of SC on frequency content of geomagnetic data using DWT application: SC automatic detection

Abstract

In this paper, a study is made to determine the effect of sudden commencement (SC) on the power spectrum of geomagnetic data using multiresolution analysis (MRA) of the discrete wavelet transform (DWT). The results of this study provides a guide to develop a new technique to automatically detect the SC because it could be an indicator of the onset of a geomagnetic storm. This new technique divides the original time series into different frequency sub-bands using the MRA of the DWT. Then it detects the change in a certain sub-band which shows a large change due to the SC. The geomagnetic records used in this study were 3-s resolution data collected from the Circum-Pan Pacific Magnetometer Network (CPMN). Using such high-resolution data enables us to minimize the detection error and the processing time to make a decision. The proposed algorithm is tested on one sample every three seconds of data sets collected from the CPMN. The maximum standard deviation of the algorithm detection times is observed to be fifty four seconds of the corresponding arrival times as determined by the National Geophysical Data Center (NGDC).

1. Introduction

It is known that a geomagnetic storm is one of the most prominent phenomena in the geospace environment. Moos (1910) identified a well-defined pattern in the so called “X disturbance” in the horizontal component (H-component) at Colaba, India. He found an occasional sudden rise of the horizontal component followed by a rapid decrease lasting a few hours and a slow recovery lasting 2–3 days. These disturbances are defined as “magnetic storms” and the various phases of the storm have been named by Chapman (1918) and Araki (1994) as: (i) sudden storm commencement (SC); (ii) initial phase; (iii) main phase; and (iv) the recovery phase.

The Dst index is commonly used in studies of magnetic storms as an indicator of the intensity of the ring current or magnetic storms. It is known that the SC is not a necessary condition for a storm to occur, and hence the initial phase is not an essential feature. But the essential feature of a storm is the significant development of a ring current.

The term Sudden Impulse (SI) is used for a sudden increase in the H-component without an ensuing magnetic storm. SIs were first regarded as magnetic disturbances with a different nature from SCs. However, SIs have been proved to be caused by the same physical mechanisms as SCs and should be classified as being complementary to SCs according to Curto et al. (2007).

The automatic detection of sudden commencements which are followed by magnetic storms is of great importance, as these events often affect radio and television interference and blackouts, which are hazards to orbiting astronauts and spacecrafts and power grids.

Joselyn (1985) has pointed out the need for an automatic detection of SC in real time to alert forecasters of potential geomagnetic storm conditions. Mendes et al. (2005) detected variations of the H-component of the geomagnetic field related to geomagnetic storms by means of wavelets. Shinohara et al. (2005) developed an automatic real-time detection and warning system of geomagnetic sudden commencements using real-time data from ground-based geomagnetic observations.

They developed a method based on statistical analysis to detect the SCs using the amplitude and increasing rate of the geomagnetic field. The forecasting of magnetic storms by using a moving gradient of the SYM-H index (the symmetric portion of the horizontal component of the magnetic field) as a detecting algorithm has been pointed out by Khabarova et al. (2006).

The automatic detection of the onset time of natural events is common in the literature. Hafez et al. (2009, 2010) used a spectrogram and a DWT, respectively, to detect earthquake onset arrival times. Takano et al. (1999) used lifting wavelet filters to build an SC detection algorithm for one-minute data resolution recorded at the KAG station. Hafez and Ghamry (2011) used time-frequency clusters to automatically detect SC times applied on one-minute data resolution and compared their algorithm with that of Takano et al. (1999). In this paper, we introduce a new algorithm to automatically detect SC times for three-second resolution data. The algorithm proposed by Hafez and Ghamry (2011) cannot be used for the three-second resolution data as the frequency content is totally different from the frequency content of the one-minute data.

Motivated by these facts, an algorithm for the automatic detection of geomagnetic sudden commencements is introduced using the details of multiresolution analysis (MRA) of a discrete wavelet transform (DWT). In the proposed algorithm, discrete wavelet coefficients of segments containing SCs are calculated at different scales. Using these coefficients, detailed features at each scale are determined. Among these details, we choose the second-stage detail in detecting SC because in this detail the arrival is very clear which enables us to detect the SC time correctly.

2. Multiresolution Analysis (MRA)

One of the important features extracted from DWT is the multiresolution analysis (MRA). MRA characterizes the spectrum of the signal at each DWT scale into low-frequency features, approximations (A), and high-frequency contents which are the details (D). Many algorithms are dedicated to determine approximations, Aj, and details, D _j , of the MRA of DWT. A pyramid algorithm is one of these algorithms described by Percival and Walden (2000). In this algorithm scaling, V _j,n , and wavelet, W _j,n , coefficients at each stage (j) of DWT stages are calculated using the following equations:

((1))

((2))

((3))

((4))

g _{j, l} : l = 0, 1, 2,..., L − 1 and h _{j, l} : l = 0, 1, 2,..., L − 1 are scaling and wavelet filters respectively, where j is the stage number and L is the filter width. The scaling filter g _l is a quadrature mirror filter that corresponds to the wavelet filter h _l by the equation g _l = (−1)^lȢ1h_L−1−l; the inverse relationship is . periodized to length is {h _l } periodized to length N and ‘a mod b’ stands for ‘a modulo b’.

x(n) is the input time series, .

The original time series can be reconstructed using the following equation:

3. Data Set

In this study, 93 sudden commencements followed by geomagnetic storms are checked. These events occurred between May 1992 and September 2005 based on the National Geophysical Data Center (NGDC) of the National Oceanic and Atmospheric Administration (NOAA). Checked events are 3-second resolution of H-component (northward component) data at a low latitude from the Circum-pan Pacific Magnetometer Network (CPMN). The CPMN was constructed by Kyushu University in collaboration with about 30 international organizations along the 210° magnetic meridian and magnetic equator during the international Solar Terrestrial Energy Program (STEP) period (1990–1997). The principle investigator (PI) of the CPMN project is Professor K. Yumoto. Figure 1 and Table 1 show the location of the stations used for this paper.

Fig. 1.

Map of the sites used to collect data to test the proposed algorithm.

Table 1. List of magnetic observatories used in our study.

4. Effect of SC on Geomagnetic Data Spectrum

SC, as a change in the geomagnetic record, results in an increase in the power of all spectrum bands at the time of its occurrence. The amount of the power increase in each frequency sub-band depends on the rate of variation of the SC. The traditional parameter which describes the sharpness of an SC is the maximum time variation of the SC (Shinohara et al., 2005) which is calculated over the period of SC increase as shown in Fig. 2. This period contains both high- and low-frequency regions the consequence of which is that the calculation of this parameter over the entire period does not precisely represent the rate of SC increase. The high-frequency region is the region where there is a fast rate of change of the geomagnetic field. While the low-frequency range is the region where there is a slow rate. The division of these two regions was accomplished manually, where by visual inspection we can notice the change of the rate of variation over different periods. Therefore, in this study we introduce another parameter which is the rate of variation of the SC which is the maximum rate of change of the geomagnetic field during the higher-frequency region of the SC, and without having discontinuities in the trace as shown in Fig. 3. These discontinuities are the points where there is a change from the high-frequency, to the low-frequency, region. This high-frequency region occurs within two minutes of the SC arrival. This way of calculating the SC rate of variation defines the sharpness of the SC precisely.

Fig. 2.

Maximum time variation is calculated over the total period of the SC increase.

Fig. 3.

Enlarged window of the H-component around the SC and the period in which the rate of variation is calculated. In the left subfigure, we can notice that there is no discontinuity in the trace. In the right subfigure, a discontinuity in the trace can be recognized. The rate is calculated in the period between the vertical lines drawn in each subfigure.

In Fig. 3, there are two SCs recorded at the KAG station: the first one was recorded on April 16, 1997, while the second SC was on December 17, 1992. In the first SC we can recognize that the trace has no discontinuity starting from the arrival time of this SC to the end of its rise time. The rate of variation of this SC can be calculated using the period between the two vertical lines which are drawn in the first SC. For the second SC, a discontinuity can be recognized in the trace during the rise time. Therefore, the rate of variation for the second SC is calculated over the trace between the two vertical lines which are drawn on the second SC. The rates of variation for the first and second SCs are 0.17 and 0.62 nT/sample, respectively, where the sampling rate is one sample per three seconds.

In order to investigate the effect of the SC on the power increase at different scales (frequencies), MRA details of DWT (first to third) of geomagnetic signals are calculated using the Daubechies wavelet filter of order 4 (abbreviated as db4). Each detail is squared, then the average of each five samples is calculated to reduce ripples in the record. As a result of this averaging process, the length of the averaged detail will be equal to the length of the original record divided by five. Figure 4 contains five columns and each column has four windows. The H-components of five different SCs in the time domain are located at the top of each column in the first row. The other three windows in each column, second, third and fourth row, are the corresponding averaged squared three details of the SC, first to third, respectively. A full explanation for each SC is given in the following.

Fig. 4.

Five SCs with rather small rates of variation. The first row is the H-component of the five SCs in the time domain. The second to fourth row each contains the corresponding averaged squared first to third detail of each SC, respectively.

The first window of the first column is an SC recorded at the KAG station on April 16, 1997. The arrival time of this SC in the time domain is at 13.33 UT. In the second window of the first column, we note no change in the power of the first detail around 13.33 UT. On the other hand, in the third and fourth windows of the first column, the power of the second and third details around 13.33 UT show a clear increase. The rate of variation of this SC is 0.17 nT/sample.

The SC at the top of the second column was recorded at the ONW station on May 28, 1994. The arrival timing of this SC is around 13.95 UT. Similarly to the first SC, although there is no change in the power of the first detail around 13.95 UT, a significant increase in power is observed in the second and third details around 13.95 UT. The rate of variation of this SC is 0.4 nT/sample.

In the same fashion, the third SC in the third column, which was recorded at the KAG station on May 15, 1997, the arrival time is around 01.99 UT. Although there is no change in the power of the first detail, around 01.99 UT, there is a clear increase in the power of the second and third details. The rate of variation of this SC is 0.4 nT/sample.

The SC in the fourth column was recorded at the KAG station on December 17, 1992. The arrival time is at 06.25 UT, a slight increase in power of the first detail around 06.25 UT can be recognized where as a large increase in the power of the second and third details around 06.25 UT is detected. The rate of variation of this SC is 0.62 nT/sample.

The SC in the fifth column is recorded at the ONW station on November 22, 1997. The arrival time of the SC is at 09.825 UT. A small increase in power in the first detail around 09.825 UT can be seen, and a clear increase in power in the second and third details around 09.825 UT is observed. The rate of variation of this SC is 0.92 nT/sample.

From the explanation given regarding these five SCs, we can conclude that the SCs with a low rate of variation do not result in a change in the power of the first detail, which is found in the frequency band 41.67–83 mHz. A clear change in the power of the second and third details can be detected due to the same low-rate SCs. The frequency bands of the second and third details are 20.8–41.67 mHz and 10.38–s 20.8 mHz, respectively. With the increase of the rate of variation of SCs, a slight increase in power of the first detail can be recognized as shown in the first detail of the fourth and fifth SCs of Fig. 4.

In Fig. 5 SCs with a high rate of variation are shown. The rate of variation values of the SCs in Fig. 5, starting from left to right, are 1, 1.07, 1.32 and 3.85 nT/sample, respectively. We can recognize that the level of increase of the power of the first detail during the SC is in direct proportional to the rate of variation of this SC. So that we can say that, with a higher SC rate of variation, the increase in power of the first detail will be more recognized.

Fig. 5.

Four SCs with a rather higher rate of variation values than the SCs of Fig. 4. The first row is the H-component of the geomagnetic field of the four SCs in the time domain. The second to fourth row each contains the corresponding averaged squared first to third detail of each SC, respectively.

5. SC Automatic Detector

There are advantages of building an SC automatic detection algorithm using high-resolution data instead of the one-minute data adopted by Shinohara et al. (2005), Hafez and Ghamry (2011) and others. The first advantage is a reduction of the automatic detection error. Assuming that both algorithms made an error of two samples in detection, in the former study, using a one-minute sampling data, this resulted in a 120-s error, whereas in the present study, with a 3-s sampling data, it involved a 6-s error. The second advantage is a reduction of the automatic detection delay. In the algorithm proposed by Shinohara et al. (2005) the decision is given based on three parameters, which are the amplitude, the time required for the increase and maximum time variation, which means that the delay is more than the time required for an increase which ranges from 2–10 minutes. Hafez and Ghamry (2011) used a four-samples window to search for an SC, which means a four-minute delay after receiving the data from the stations. In our proposed algorithm, the window is five samples which means 15 seconds delay after receiving the data. Therefore, it is favorable to use high-resolution data to perform fast and accurate SC detection.

The rule of MRA of DWT is to create the frequency sub-bands. Where the details represent the high-frequency components, and the approximations are the low-frequency components, of each stage. Therefore, details may be considered as the original signal filtered within a specific bandwidth, which can be calculated from the equation , where F is 166*10⁻³ Hz as the sampling rate is one sample per three seconds and j is the stage number. As is shown in the previous section, the increase in power in the second and third details is clear at the arrival of the SC regardless of how sharp the SC is, unlike the first detail for which the increase in power is only found at high-rate SCs. Using this feature, a new algorithm is introduced to automatically detect the SC arrival time. The proposed algorithm will monitor the change in power in only one of the frequency sub-bands, in which there is a detectable power increase during the SC, which is either the second or the third details. In order to choose between the second and third details, many events were checked and we found that the second detail gave better results than the third detail. So we choose the second detail to build the proposed SC detector. The second detail of the MRA of DWT of a geomagnetic signals is calculated using a db4 wavelet filter. Averages of each 5-sample of the squared second detail are calculated to reduce any ripple which may cause false detections. A trigger is declared when the amplitude of this averaged squared detail exceeds a certain threshold over a certain triggering period. The values of the triggering threshold and period are chosen after checking many SCs. One of these SCs is shown in Fig. 6.

Fig. 6.

Methodology of how the algorithm alarms for a trigger. The alarm for one record is set when the amplitudes of the averaged squared second detail exceed the triggering threshold over a period longer than the triggering period. The detailed explanation for each panel is described in the text.

In Fig. 6, a-1 and a-2 show the H-component of the geomagnetic field for an entire day and the enlarged segment of this field around SC, respectively. b-1 is the averaged squared second detail of the entire day. In (b-2), the segment around the SC in the second detail is enlarged. In this subfigure, we can notice that the condition of alarming for a trigger is satisfied when the amplitudes of the second detail exceed the triggering threshold over a period larger than the triggering period.

In (b-3), a spike during which the amplitude exceeds the threshold can be seen. This spike doesn't alarm for a trigger as the duration of such a spike is less than the triggering period. There is a possibility of the occurrence of the alarming condition when there is no SC; in such a situation, this trigger is called a false trigger. This possibility, of false triggers, is low in normal days but it is observed to increase after the occurrence of an SC. In order to confirm that this trigger is an SC and not a false trigger, the other stations are checked if they alarm for a trigger within a certain period around the time of this alarm. If a trigger is found at these stations, this trigger is declared to be an SC, otherwise this trigger is declared as a false trigger. This certain period is considered as an accepted margin, and can be set by the user. In this paper, we propose three minutes for this specific period. Examples explaining this point in detail will be described in the Section 6, and are shown in Table 2.

6. Results

The data set collected from the CPMN is chosen to satisfy different values of the SC rate of variation which is explained in Section 4. The smaller this rate of variation is, the harder it is to automatically detect the SC arrival time correctly. Figure 7 shows a plot of the rate of variation values of all SCs which have been used to test the proposed algorithm. In this figure, we can see that the values of the rate of variation range from 0.1 to 11.8 nT/sample.

Fig. 7.

The rate of variation values of the 93 SCs which are used in this study.

The proposed SC detection algorithm was tested for 93 magnetic storms, recorded by the stations listed in Table 1. The SC times calculated by the present algorithm are compared with the times listed in the web-site of the National Geophysical Data Center (NGDC) of the National Oceanic and Atmospheric Administration (NOAA). It was found that unsuccessful detections were only eight SCs, which is a small number.

Figure 8 illustrates the error probability distribution function when using a db4 wavelet filter, where the error is defined as the time difference between the algorithm arrival times and the SC times which are determined by the NGDC. The average and standard deviation of error are found to be 1 and 18 samples, respectively, where the sampling rate is one sample per three seconds.

Fig. 8.

The probability distribution function (PDF) of the error in detection times generated by the algorithm compared with NGDC times. Negative values means the detection time made by the algorithm leads the NGDC detection.

We are not aware of other algorithms which can be tested on three-second resolution geomagnetic data in order to perform a comparison with our proposed algorithm. Therefore, some examples are shown in Fig. 9 to show the capability of the algorithm to determine the SC times. In Fig. 9 four SCs are illustrated, from the data set used in this study. For each of these events, the subfigures are arranged as follows: a-1 is the H-component of the entire day which contains the SC, a-2 is an enlarged view of the SC, a-3 is the squared averaged second detail of the entire day, and a-4 is an enlarged window around SC of the averaged squared second detail. The other three SCs (b, c and d) are arranged in the same manner. In each of the four subfigures a, b, c and d-4, we can recognize that the amplitudes of the averaged squared second detail exceed the threshold over a period, which is larger than the triggering period which results in triggering for an SC at the arrival time of the SC.

Fig. 9.

Four examples from the used data set. The subfigures can be explained as follows: a-1 and a-2 are the H-component of the geomagnetic field of the entire day and the enlarged segment of this field around SC, respectively. a-3 and a-4 are the averaged squared second detail of the entire day and the enlarged segment of this detail around SC, respectively. The same sequence is valid for b, c and d.

Table 2 contains some examples of the SCs used to test the proposed algorithm. In Table 2, we can find that there are two or three stations which record the same SC. It is clear that the Algorithm Detection Times of the stations recording the same SC are very close to the actual Arrival Times calculated by NGDC times, such as in the SCs 1 to 6, 8 and 9, which ensures that the algorithm is very efficient in automatically detecting the SC times correctly. In SC number 7, although the correct proposed detection time is the third one but the first two detection times can not declare the presence of an SC, as the other two stations at 8 and 9 have no proposed detection times close to either of these two times. The same situation can be found in the SCs in 10, 11 and 12. The complete table of all the data set is available from the authors upon request.

Table 2. Arbitrarily chosen SCs from the checked data set. These examples show the capability of the algorithm to determine the SC timing correctly and its immunity against alarming false triggers. Times are expressed in UT of the respective days and in units of sample (1 sample = 3 s).

7. Conclusions

The effect of sudden commencement on geomagnetic data spectrum is investigated in this paper using the DWT. This effect provided the guide to build a new algorithm for determining geomagnetic storm sudden commencement onset times. It is proved that the change in the spectrum during the SC is directly proportional to the value of the SC rate of variation. Investigations elaborate that the second detail of the MRA of the DWT is the best detail to be used to monitor the SC timing. Case studies have shown that the algorithm is very sensitive to both low- and high-rate variation SCs. Applying the algorithm on a data set from the Circum-pan Pacific Magnetometer Network (CPMN), it is found that the time of SC arrivals computed by our proposed algorithm is very close to those observed by the NGDC.

Appendix A.

This appendix contains the pseudo code of the proposed algorithm as follows:

Read the H-component record

Divide the H-component record into segments

Calculate the number of segments

Set the detection threshold amplitude

Set the triggering period

While the segment number is less than total number of segments

Calculate the DWT squared second detail of the current segment

If the average value of the squared second detail is larger than the detection threshold over a period greater than the triggering period

Calculate the timing of the segment

Display that there is an SC in this record

else

Display that there is no SC in this record

End if

Add one to segment

End while

Read reference value of SC which is published by NGDC

Define the error which is the difference between the calculated SC timing and the reference timing published by NGDC