Figure 2 shows the F10.7 (input) and TEC (output) variations derived from the GAIA simulation run on January 6, 2020, example of an ionospheric storm. The F10.7 value was ~ 72 × 10−22 Wm−2 Hz−1 during the interval (Fig. 2a). Figure 2b shows the variation of TEC averaged over 35–39° latitude and 125–147° longitude, for 5 days from January 4 to January 8. The TECs obtained under the WJRA setting (red line) and WOJRA settings (orange line) show a diurnal variation with enhancements during the local daytime. We refer to a 27-day median at the same local time as a reference profile. Compared with this reference profile (black line in Fig. 2b), several deviations are apparent, the largest one occurring in early 8 January. The grey hatches correspond to a scale indicating a significance of the deviation of the TEC profile from the reference profile. We measure the significance and detect ionospheric storms using the “I-scale” index (Nishioka et al. 2017) derived for GAIA (see Appendix). The shade of grey represents the deviation normalized by a standard deviation derived using long-term (1996–2016) simulation outputs. The significance levels are shown by the different darkness levels. The TEC variation (red or orange line) within the dark grey region is not significant, while that extending to the light grey or white region is a significant disturbance above the regular variation. From this index, two significant ionospheric storms are detected when the TEC values increased to the light grey region from the end of January 7 to UT 3:00 on January 8 and UT 8:00–18:00 on 8 January, as indicated by IP2 labels in Fig. 2b. According to results based on observations using the Global Navigation Satellite System (GNSS) (Nishioka et al. 2017), the IP2 positive storm was seen at UT 17:00–22:00 on January 7, which is slightly earlier than the first event.
Figure 2c–e shows TEC global maps related to the large enhancement seen at LT midnight on January 7–8 at ~ 37° in Japan. The difference between the instantaneous profile (Fig. 2c) and the reference TEC, i.e., 27-day median at the same local time (Fig. 2d), provides a deviation TEC map (Fig. 2e). For this event, our simulation suggests that localized TEC enhancements around Japan at the west edge of a northern equatorial anomaly create this event.
To understand the cause of the TEC enhancements, we also output electric field, neutral wind, and atmospheric composition. The east electric field lifts the ionospheric plasma by ExB drift. Southward (northward) neutral wind drags up the plasma along the slanted magnetic field in the middle latitudes in the northern (southern) hemisphere. Fewer recombinations take place at high altitude with fewer molecular ions. The O/N2 ratio, a measure of ion production versus ion loss processes, also varies with season and magnetospheric activity caused by atmospheric dynamics.
The above parameters during this ionospheric event on January 8, 2020 are shown in Fig. 3. The time variation of the latitudinal profile of TEC at 135° longitude from the same GAIA run shows the extension of the equatorial anomaly with 30 TECU (1 TEC unit = 1016 m−2) at 30° latitude and 20 TECU beyond 47° latitude in early January 8 (Fig. 3a). All parameters are dominated by diurnal variations, and some variations are added. The eastward electric field is dominant in the dayside at high latitudes owing to magnetospheric convection, as well as at mid-low latitudes (Fig. 3b). Further enhancement of the eastward electric field over a wide range of latitude is seen from late January 7 and continues LT daytime and evening time on January 8. The meridional neutral wind is northward at daytime and southward at night time at the northern mid-latitude of our interest (Fig. 3c). The O/N2 ratio at 300 km altitude is high in the northern winter hemisphere (Fig. 3d). The ratio rather decreases to < 5 at UT 0:00–8:00 on January 8 compared with O/N2 of ~ 7 at the same time the previous day. Therefore, the TEC enhancement on January 8 would be mainly related to the enhancement of the east electric field, originated from the neutral wind dynamics and/or polarization field.
Next, we test the predictability of ionospheric storms using the WJRA simulation and WOJRA simulations from July 2019 to March 2020. Figure 4 shows the GAIA simulation results under these five settings with results based on observations using the GNSS (Nishioka et al. 2017). Here, flag = 1 is assigned when a strong positive (IP2) or negative (IN2) storm is detected in at least one of the TEC I-scales at the five latitudes in Japan (see Appendix), and flag = 2 is assigned when a severe positive (IP3) or negative (IN3) storm is detected in at least one of the TEC I-scales at the five latitudes. Other days are assigned flag = 0. Figure 4 also shows the daily maximum of the Kakioka K index for a reference of the magnetospheric activity (blue line). The observation shows flag enhancements sometimes over the entire year, and some of them are associated with the large K index, e.g., on day of year (DOY) ~ 216 and ~ 243. On the other hand, the current GAIA simulation assumes a quiet magnetospheric condition. The GAIA flag values appear slightly concentrated on days 270–300 and 340–400 from January 1, 2019, which would mainly reflect a seasonal variation. This more sensitive seasonal dependence of the model might be partly affected by the constant quiet magnetospheric setting. These concentrations of flag appearance are more significant under WOJRA-2/3/4 and seem to be improved under WOJRA-1 and WJRA settings.
To evaluate the estimation, we use accuracy, threat score (TS), and Heidke’s skill score (HSS), which are calculated as
$${\text{Accuracy }} = \, \left( {{\text{TP}} + {\text{TN}}} \right)/N,$$
(1)
$${\text{Threat score }} = {\text{ TP}}/\left( {{\text{TP}} + {\text{FP}} + {\text{FN}}} \right),$$
(2)
$${\text{Heidke's skill score}} = {\mkern 1mu} \left( {{\text{TP}} + {\text{TN}} - {\text{Sc}}} \right)/{\mkern 1mu} \left( {N - {\text{Sc}}} \right),$$
(3)
where TP (true positive) is the number of “hits”, i.e., both the observation and model report events (flag ≥ 1). FP (false positive) is the number of “false alarms”, i.e., the model forecasts events while the observation does not (flag = 0). FN (false negative) is the number of “misses”, i.e., the model forecasts quiet while the observation shows events. TN (true negative) is the number of events that both model and observation forecasts as quiet. N = TP + FP + FN + TN. HSS represents the score excluding the random prediction. Sc is the hit rate by random prediction defined as follows.
$${\text{Sc }} = \, \left( {{\text{TP}} + {\text{FN}}} \right) \, /N*\left( {{\text{TP}} + {\text{FP}}} \right) + \left( {{\text{FP}} + {\text{TN}}} \right)/N*\left( {{\text{FN}} + {\text{TN}}} \right).$$
(4)
These scores for the WJRA and WOJRA settings are shown in Fig. 5. All scores decrease with the lead time. The accuracy decreases from 0.80 to 0.72 (Fig. 5a). The TS decreases with time after the cessation of WJRA input: TS = 0.11 (0.096) for the WJRA setting, TS = 0.10 (0.090) for WOJRA-2, and TS = 0.061 (0.044) for WOJRA-4 of all case of K (limited daily maximum K < 4). Both the decreasing trend and values are similar to those in the analysis using magnetospheric quiet days with daily maximum K of less than 4 (blue dashed line in Fig. 5). Since the occurrence number of ionospheric storm events is small, the TS is considered to be a better evaluation than the accuracy for this case. The results of this analysis confirmed that the input of realistic low-altitude information improves the model prediction accuracy.
As mentioned above, HSS represents the score excluding the random prediction, with positive value means better prediction compared to the random prediction. The derived score shows the positive value for the WJRA and WOJRA-1 ~ 3 settings, while it becomes negative for the WOJRA-4 setting (Fig. 6c). This indicates that the prediction ability is better than the random prediction up to the WOJRA-3 setting, while the accuracy is worse for the WOJRA-4 setting. Therefore, the maximum lead time is evaluated to be ~ 1-day.
On the other hand, if the prediction for tomorrow is assumed to be the same as today, the accuracy, TS, and HSS are 0.78, 0.19, and 0.20, respectively. The neural network method provides the prediction of ionospheric storms using the critical ionospheric frequency foF2 from the ionosonde at Tokyo over 1985–1996 (Nakamura et al. 2007) with the accuracy of 0.63, TS of 0.22, and HSS of 0.11 based on their Table 15. Although prediction ability of our numerical model is less than their neural net approaches except for the accuracy value, we can obtain spatial and temporal variation with physical backgrounds, as described in Fig. 3. To contribute to space weather forecast efficiently, the improvements of our model and system improvements described in the last paragraph of “Summary and future works” section are required.