Empirical modeling of the storm time geomagnetic indices: a comparison between the local K and global Kp indices
Earth, Planets and Space volume 66, Article number: 95 (2014)
This paper describes a neural network-based model developed to predict geomagnetic storms time K index as measured at a magnetic observatory located in Hermanus (34°25 S; 19°13 E), South Africa. The parameters used as inputs to the neural network were the solar wind particle density N, the solar wind velocity V, the interplanetary magnetic field (IMF) total average field B t as well as the IMF B z component. Averaged hourly OMNI-2 data comprising storm periods extracted from solar cycle 23 (SC23) were used to train the neural network. The prediction performance of this model was tested on some moderate to severe storms (with K≥5) that were not included in the training data set and the results are compared to the prediction of the global geomagnetic Kp index. The model results show a good predictability of the Hermanus storm time K index with a correlation coefficient of 0.8.
Geomagnetic storms are the common features of space weather causing a threat to ground- and space-based technology systems. Most of the intense geomagnetic storms are generally caused by fast coronal mass ejections (CMEs) which induce disturbances in the solar wind (SW). Geomagnetic storms occur as a result of the energy transfer from the SW to the Earth’s magnetosphere via magnetic reconnection. Hence, changes in the SW plasma and the interplanetary magnetic field (IMF) are important factors to consider when developing magnetic storm forecast models. At present, the physics of the magnetosphere and the interplanetary medium is not completely understood and there is still no comprehensive model of the solar-terrestrial environment. The current geomagnetic storm prediction tools are dominated by empirical models, relying mostly on the observable storm precursors in the SW (Fox and Murdin 2001). There have been various functional relationships proposed for magnetic storm prediction such models to predict the disturbance storm time (Dst) index from SW parameters proposed by Burton et al. (1975) and Temerin and Li (2002). Empirical prediction models include, among others, the neural network (NN) models that are known to have the property of learning from cases and with ability to handle complex nonlinear physical phenomena. This NN capability in space weather-related predictions has been demonstrated in various studies (e.g. Uwamahoro et al. 2012; Watthanasangmechai et al. 2012). In the domain of geomagnetic field, Segarra and Curto (2012) recently applied NN for the automatic detection of sudden commencements. Other various NN-based models for predicting geomagnetic storms using SW and IMF data as inputs have also been developed and used (Lundstedt and Wintoft 1994). In particular, Elman NN-based algorithms by Wu and Lundstedt (1996), Wu and Lundstedt (1997), and Lundstedt et al. (2002) demonstrated the ability to improve the Dst forecast.
Other than the prediction of the Dst index, models for predicting geomagnetic Kp index (from SW and IMF input parameters) have also been developed (Boberg et al. 2000; Costello 1997; Wing et al. 2005). The difficulties related to the prediction of Kp index during storm periods (with K p>5) were noticed by Wing et al. (2005). A few models have also been developed for the prediction of the locally measured K index including the work by Virjanen et al. (2008) and Kutiev et al. (2009). In their study, Virjanen et al. (2008) described the problems encountered when predicting the storm time K index on the basis of only previous K index values and suggested the necessity to consider SW parameters as model inputs. The study described in this paper explored the application of Elman NN techniques for predicting the locally measured geomagnetic K index at the Hermanus Magnetic Observatory. The results obtained are compared to the prediction performance of the global Kp index.
The motivation behind this study lies on the importance of the K and related planetary Kp indices in space weather modeling. The ability to predict the K index can find application, for example, in predicting geomagnetically induced currents (GICs) (Virjanen et al. 2008). On the other hand, the K-derived planetary Kp index plays a key role in the magnetospheric and ionospheric modeling (Wing et al. 2005). Regional ionospheric models, e.g. TEC prediction models and the South African Bottomside Ionospheric Model (SABIM), take into account the local magnetic conditions by using the a index, which are directly derived from the locally recorded K index (Habarulema 2010; McKinnell 2002). An accurate model to predict the local storm time K index might, therefore, make a significant contribution towards improving ionospheric and other regional space weather models that consider magnetic activity as input.
The data sets
Geomagnetic K and K p indices
Geomagnetic K index is a quasi-logarithmic local index of geomagnetic activity. The K index quantifies disturbances in the H component of the Earth’s magnetic field with an integer in the range of 0 to 9, with 1 indicating calm conditions and 5 or more indicating a storm. The K index is derived from the maximum fluctuations of the H component observed on a magnetometer during a 3-h interval. Ground stations (magnetometers) throughout the world monitor geomagnetic activity providing a local logarithmic K index. There is a close link between the local K and global Kp indices. The planetary-scale Kp index (Menvielle and Berthelier 1991) is derived from the average of fractional K indices at 13 subauroral observatories. The Kp index is based primarily on data from magnetic observatories at middle latitudes and its values are generated with a time resolution of 3 h. This index represents a quasi-logarithmic measure of the disturbance range, also having values between 0 (very quiet) and 9 (very disturbed). While the K is a measure of the local magnetic disturbance, the Kp index is a good measure of the global magnetic activity (Prölss 2004).
This paper mainly focusses on the predictability of the storm time K index recorded at the Hermanus Magnetic Observatory (34° 25.5′ S; 19° 13.5′ E) in South Africa. The observatory is part of the South African National Space Agency (SANSA) at Hermanus and is also an active participant in the worldwide network of magnetic observatories (INTERMAGNET), monitoring and modelling variations of the Earth’s magnetic field. The K index represents a measure of the local geomagnetic activity response to solar and associated SW disturbances (http://spaceweather.sansa.org.za/). Figure 1 illustrates an intense magnetic storm on 3 to 5 August 2010 (K=6 recorded at Hermanus) following a coronal mass ejection launched from the Sun on the 1 August 2010 at 13:42 UT.
Solar wind input parameters
Geomagnetic disturbances are closely linked to the IMF fluctuations, both in magnitude and direction (Schwenn et al. 2005). An interconnection between the long-duration southward IMF B z component and the Earth’s magnetic field allows SW energy transport into the Earth’s magnetosphere (Gonzalez et al. 1994). Several studies including a recent work by Kissinger et al. (2011) have indicated the role of the SW speed in magnetic storms generation. Indeed, sustained and enhanced SW speed and southward and northward IMF B z components are commonly associated with interplanetary shocks and ejecta known to be important causes of storms (Gosling et al. 1990). On the other hand, enhanced SW number density N is also an important parameter which often affects the storm strength (Crooker 2000). An increase in the SW density can cause the compression of the dayside magnetopause which drives the increase of the magnetopause current, field-aligned currents and cross-tail currents. Many papers, e.g. Wang et al. (2003) and Xie et al. (2008), have described the link between the high SW dynamic pressure and geomagnetic storms. Correlations between the Kp index and various SW parameters were previously established (e.g., Papitashvili et al. 2000). Figure 2 indicates the correlation between various SW parameters and the Hermanus (Her) K index. From the figure, it is clear that the IMF B z is more correlated with the K index more than any other SW parameter.
For the model described in this paper, the input parameters used were the SW speed V, the IMF B t and B z components as well as the SW particle number density N. The B z used here is in the Geocentric Solar Magnetospheric (GSM) system because it maximizes the correlation with geomagnetic activity (Kivelson and Russell 1995). Figure 3 illustrates the disturbances in the SW parameters and associated geomagnetic response as measured by the local K and the global Kp indices during a storm period. The figure clearly indicates how higher values of magnetic indices are directly associated with the abrupt changes in the SW parameters.
The model was developed using hourly OMNI-2 SW and IMF parameters [B t ,B z ,V and N] data for both network training and testing sets. These data are from various spacecraft and are provided by the National Space Science Data Center available online on its OMNIWEB http://omniweb.gsfc.nasa.gov/html. The Kp index data used are provided by the National Geophysical Data center (NGDC) and are also available online on the website ftp://ftp.ngdc.noaa.gov/STP/GEOMANGETIC_DATA/.
An introduction to neural network prediction techniques
A neural network is an information processing system consisting of a large number of simple processing elements called neurons. NNs are characterised by (1) the pattern of connection between the neurons, (2) the method of determining the weights on the connections (training or learning algorithm) and (3) the activation function (Fausett 1994). For the NN models used for predictions, three types of neurons (or units) are defined: (i) input units, which are set to represent values within the time series, (ii) output units, which store the output values corresponding to a given set of input values and produce the results of the NN processing and (iii) hidden units, which keep the internal representation of the mapping.
Units in layers are connected by weights which keep the knowledge of the network and govern the influence of each input has on each output. Weights are adjusted by a learning process which involves the comparison of network calculations with input-output data for known cases. The process of adjusting weights is known as network training. During the training, weights are determined so that the network properly relates inputs to desired outputs. Hence, the network learns to predict outcomes from experience rather than from using causal laws (Macpherson et al. 1995).
A unique feature of NNs lies in their ability not only to learn the training data but also to generalise by predicting unseen patterns within the boundaries given by the training set. In general, solving a nonlinear problem with the NN technique requires (1) choosing a convenient network architecture, (2) selecting a large database of input-output pairs (patterns) that contains sufficient historical information about the time series, and (3) training the network to relate the inputs to the corresponding outputs. Several available NN training algorithms have been proposed (Bishop 1995; Fausett 1994; Haykin 1994) including the feed forward NN (FFNN) and the Elman neural network. To develop the local K and global index prediction model, the Elman neural network algorithm was used. The Elman NN (Elman 1990) is a type of network that belongs to the class of recurrent NNs commonly known as the Elman recurrent network (ERN). This consists of an input layer, a hidden layer and an output layer. It also has an additional context layer that always stores the output from the hidden layer and relays this information in the next iteration. Therefore, context neurons form a sort of short-term memory, very useful for improving prediction of sequences. This means that the state of the whole network at a given time depends on an aggregate of the previous states, as well as on the current inputs (Pallocchia et al. 2006). A simplified mathematical description of ERN can be found in various literature including a recent paper by (Cai et al. 2010).
Development of the NN model
The training and testing data sets consist of storm periods selected within SC 23 [1996-2006]. The database was constructed based on storm events with K p K≥5. Each storm period was defined as having a K≥5 (K p ) at least once, each preceded and followed by a quiet magnetic period of at least 12 h. The quiet time data included variations from one storm event to the other since it depended on the storm behaviour. However, for each storm event, there was at least a day (eight data points) of K p K≤5 included before and after the storm time. Based on this criteria, the training database (1996 to 2003 and 2006) consisted of 4,930 data points. Selected storm periods during years 2004 and 2005 (688 data points) were excluded from the training process and were used to test the performance of the model. Note that both the observed and predicted local K and global K p indices are three-hourly indices. Therefore, an input row (pattern) is made up of four SW parameter values (V,B t,B z,N), each one being the average of the three preceding hourly values. Figure 4 shows a schematic illustration of the network architecture used in this study. The m values were 5 and 6 (indicating that there were five and six hidden nodes in the hidden layer) for the K index and Kp index models, respectively. The output of the NN is a three-hourly K(K p ) index. During the training process, the optimal number of hidden nodes was systematically determined by varying the number of hidden nodes. At the start of the training process, weights are chosen randomly for the ERN within both the input and context layers. Training is done iteratively and the mean square errors for training and testing patterns were monitored. As long as the error on the testing pattern decreased, the training process was allowed to proceed and terminated only when the error started increasing since at this point, the network is believed to have achieved convergence/generalisation. The root mean square (RMSE) and the correlation coefficient (CC) were the statistical measures used to characterise the prediction performance of the model. The network with the optimum performance was reached with NN structure configurations 4:5:1 for K index and 4:6:1 for Kp index. Numbers in configuration 4:5:1 represent input, hidden and output nodes, respectively. Table 1 shows various network configurations that were tried and the corresponding RMSE or CC.
Results and discussion
Table 1 shows different NN configurations that were investigated with the corresponding prediction performance, evaluated by calculating RMSE and CC over the whole validation data set. From Table 1, it is clear that the developed model performs better when predicting the global Kp index than it does for the prediction of the local Hermanus K index. One among other possible reasons of this difference in prediction performance might be due to the fact that the global Kp index is derived from various K indices averaged and corrected to their respective magnetic latitudes observatories. The results from the developed model indicate a CC of 0.76 between the predicted and observed Hermanus K index and a CC of 0.88 for Kp index. Even though the data set used is not the same, this model prediction is comparable with the previous Kp index prediction by Wing et al. (2005) and Boberg et al. (2000). However, it is important to note that the latter models considered all the Kp s as input data, while the current model was developed using the Kp input data for only selected storm events.
The prediction performance of this model was tested on four selected intense storms which were part of the validation data set (not included in the training data set). Two of the four storms occurred in 2004 and the two others in 2005. The two selected storms in 2004 were both long-duration intense storms, behaving like two storms in one with two peak maxima. Figure 5 shows the performance of the model on predicting the local Hermanus K index during the selected storm periods. The storm in Figure 5a lasted for 5 days (from 7 to 12 of November 2004) reaching two peak maxima of K=7 within the storm period. The global Kp index reached a peak maximum of K p=8.7 on 8 November at 03:00 UT and on 9 November at 18:00 UT as shown on Figure 6a. The storm in July 2004 (Figure 5b) also had two peak K index maxima, one on 25 July at 6:00 UT (K=6) and another on 27 July at 21:00 UT with K=7. The two test storms in 2005 represented in Figure 5c,d represent the most recent and greatest magnetic storms of SC 23. Both the storms reached the peak Hermanus K index value of 8. Figure 6 is similar to Figure 5, for the global K p , and indicates that there is a slightly improved performance when the model is applied to the prediction of the global Kp index. Table 2 presents a statistical summary of the model’s K index prediction performance on the selected individual storms and clearly shows that the selected storms are well predicted with an average CC of 0.80 for the K index and 0.90 for the Kp index.
A NN-based model for predicting the storm time local K and global K p indices using SW and IMF input parameters has been developed. Many previous studies focused on developing models to predict magnetic storms as measured by the Dst index. However, some findings (e.g. Borovsky and Denton 2006) suggest that the Dst alone can, in some cases, be a poor indicator of the properties of a storm. The primary aim of this study was to explore the NN predictability of the locally measured storm time K index from SW and IMF parameters. The results obtained compare well with previous Kp (closely related to K index) predictions by Wing et al. (2005) and Boberg et al. (2000) noting however that contrary to these previous models, the current model involved Kp data for only selected storm events. The results obtained from the developed NN models are in line with what is already known about the SW control of geomagnetic activity. With a knowledge of the SW velocity, density, as well as the IMF strength and orientation, it is possible to predict well the energisation of the ring current and reproduce accurately the magnetic measurements recorded by ground-based magnetometers (Russell 1986). The developed model constitutes a step towards achieving real-time forecasts of the locally (Hermanus) measured K index. If achieved, the real-time prediction of the K index will contribute significantly to improving regional ionospheric modelling as well as other regional space weather models that consider the locally measured magnetic activity as input.
Bishop CM: Neural networks for pattern recognition. Oxford University Press, New York; 1995.
Boberg FP, Wintoft P, Lundstedt H: Real time Kp prediction from solar wind data using neural networks. Phys Chem Earth 2000, 25(A04203):275–280.
Borovsky JE, Denton MH: Differences between CME-driven storms and CIR-driven storms. J Geophys Res 2006, 111: A07S08. doi:10.1029/2005 JA011447
Burton RK, McPherron RL, Russell CT: An empirical relationship between interplanetary conditions and Dst. J Geophys Res 1975, 80: 4204–4214. 10.1029/JA080i031p04204
Cai L, Ma SY, Zhou YL: Prediction of SYM-H index during large storms by NARX neural network from IMF and solar wind data. Annales Geophys 2010, 28: 381–393. 10.5194/angeo-28-381-2010
Costello KA: Moving the rice MSFM into a realtime forecast mode using solar wind driven forecast models. Ph.D. thesis, Rice University, Houston, Texas; 1997.
Crooker NU: Solar and heliospheric geoeffective disturbances. J Atmos Solar-Terrestrial Phys 2000, 62: 1071–1085. 10.1016/S1364-6826(00)00098-5
Elman JL: Finding structure in time. Cogn Sci 1990, 14: 179–211. 10.1207/s15516709cog1402_1
Fausett LV: Fundamentals of neural networks: architecture, algorithms and applications. Prentice-Hall, New Jersey; 1994.
Fox N, Murdin P: Solar-terrestrial connection: space weather predictions. In Encyclopedia of astronomy and astrophysics. No. 2416. Edited by: Murdin P. Bristol: Institute of Physics; 2001.
Gonzalez WD, Joselyn JA, Kamide Y, Kroehl HW, Tsurutani BT, Vasyliunas VM, Rostoker G: What is a geomagnetic storm? J Geophys Res 1994, 99: 5771–5792. 10.1029/93JA02867
Gosling JT, Bame SJ, McComas DJ, Philips JL: Coronal mass ejections and large geomagnetic storms. Geophys Res Lett 1990, 17: 901–904. 10.1029/GL017i007p00901
Habarulema J: A contribution to TEC modelling over Southern Africa using GPS data. Ph.D. thesis, Rhodes University, Grahamstown, South Africa; 2010.
Haykin S: Neural networks: a comprehensive foundation. Macmillan, New York; 1994.
Kissinger J, McPherron RL, Hsu T-S, Angelopoulos V: Steady magnetospheric convection and stream interfaces: relationship over a solar cycle. J Geophys Res 2011, 116: A00I19. doi:10.1029/2010JA015763
Kivelson GM, Russell CT: Introduction to space physics. Cambridge University Press, UK; 1995.
Kutiev I, Muhtarov P, Andonov B, Warant R: Hybrid model for nowcasting and forecasting the K index. J Atmos Solar-Terrestrial Phys 2009, 71: 589–596. 10.1016/j.jastp.2009.01.005
Lundstedt H, Gleisner H, Wintoft P: Operational forecasts of the geomagnetic Dst index. Geophys Res Lett 2002, 29(24):2181. doi:10.1029/2002GL016151,2002
Lundstedt H, Wintoft P: Prediction of geomagnetic storms from solar wind data with the use of a neural network. Ann Geophys 1994, 12: 19–24. 10.1007/s00585-994-0019-2
Macpherson KP, Conway AJ, Brown JC: Prediction of solar and geomagnetic activity data using neural networks. J Geophys Res 1995, 100(A11):735–744.
McKinnell LA: A neural network based ionospheric model for predicting the bottomside electron density profile over Grahamstown, South Africa. Ph.D thesis, Rhodes University, Grahamstown; 2002.
Menvielle M, Berthelier A: The K-derived planetary indices - description and availability. Rev Geophys 1991, 29: 415–432. 10.1029/91RG00994
Pallocchia G, Amata E, Consolini G, Marccucci MF, Bertello I: Geomagnetic Dst index forecast based on IMF data only. Annales Geophys 2006, 24: 989–999. 10.5194/angeo-24-989-2006
Papitashvili VO, Papitashvili NE, King JH: Solar cycle effects in planetary geomagnetic activity: analysis of 36-year long OMNI dataset. Geophys Res Lett 2000, 27: 2797–2800. 10.1029/2000GL000064
Prölss GW: Physics of the Earth’s space environment: an introduction. Springer, Berlin Heidelberg; 2004.
Russell CT: Solar wind control of magnetospheric configuration. In Solar wind-magnetosphere coupling. Edited by: Kamide Y, Salvin JA. Tokyo: Terrapub; 1986:209–231.
Schwenn R, Lago AD, Huttunen E, Gonzalez WD: The association of coronal mass ejection with their effects near the Earth. Annales Geophys 2005, 23: 1033–1059. 10.5194/angeo-23-1033-2005
Segarra A, Curto JJ: Automatic detection of sudden commencements using neural networks. Earth Planets Space 2012, 65: 791–797.
Temerin M, Li X: A new model for prediction of Dst on the basis of the solar wind. J Geophys Res 2002., 107(A12, 1472): doi:10.1029/2001JA007532,2002 doi:10.1029/2001JA007532,2002
Uwamahoro J, McKinnell LA, Habarulema JB: Estimating the geoeffectiveness of halo CMEs from associated solar and interplanetary parameters using neural networks. Annales Geophys 2012, 30: 963–972. 10.5194/angeo-30-963-2012
Virjanen A, Pulkkinen A, Pirjola R: Predictions of the geomagnetic K index based on its previous value. Geophysica 2008, 44(1–2):3–13.
Wang CB, Chao JK, Lin C-H: Influence of the solar wind dynamic pressure on the decay and injection of the ring current. J Geophys Res 2003, 108: A9. doi:10.1029/2003JA009851
Watthanasangmechai K, Supnithi P, Lerkvaranyu S, Tsugawa T, Nagatsuma T, Maruyama T: TEC prediction with neural network for equatorial latitude station in Thailand. Earth Planets Space 2012, 64: 473–483. 10.5047/eps.2011.05.025
Wing S, Johnson JR, Jen J, Meng C-I, Sibeck DG, Bechtold K, Freeman J, Costello K, Balikhin M, Takahashi K: Kp forecast models. J Geophys Res 2005., 110(A04203): doi:10.1029/2004JA010500 doi:10.1029/2004JA010500
Wu JG, Lundstedt H: Prediction of geomagnetic storms from solar wind data using Elman recurrent networks. Geophys Res Lett 1996, 23: 319–322. 10.1029/96GL00259
Wu JG, Lundstedt H: Neural network modeling of solar wind-magnetosphere interaction. J Geophys Res 1997, 102(A7):457–466.
Xie HN, Gopalswamy N, Cyr OCS, Yashiro S: Effects of solar wind dynamic pressure and preconditioning on large geomagnetic storms. Geophys Res Lett 2008, 35: L06S08. doi:10.1029/2007GL032298
The authors would like to thank both the National Space Science Data Center and the National Geophysical Data Center (USA) for making available their data to us via the following websites: http://omniweb.gsfc.nasa/htmland ftp://ftp.ngdc.noaa.gov/STP/GEOMAGNETIC_DATA/. This work was facilitated by a logistic support from both the University of Rwanda and the Space Science Directorate of the South African National Space Agency (SANSA), in Hermanus, South Africa.
The authors declare that they have no competing interests.
JU contributed in the conception, design, drafting of the manuscript and coordination of the study. JBH participated in the design of the manuscript with data processing and proofreading of the manuscript. Both authors read and approved the final manuscript.
Jean Uwamahoro and John Bosco Habarulema contributed equally to this work.
Authors’ original submitted files for images
About this article
Cite this article
Uwamahoro, J., Habarulema, J.B. Empirical modeling of the storm time geomagnetic indices: a comparison between the local K and global Kp indices. Earth Planet Sp 66, 95 (2014). https://doi.org/10.1186/1880-5981-66-95