Assessment of International GNSS Service Global Ionosphere Map products over China region based on measurements from the Crustal Movement Observation Network of China

1.   Introduction

Total electron content (TEC) is an important physical parameter used in describing ionospheric properties. Highly accurate TEC models play a vital role in forecasting and elucidating ionospheric variability. Substantial efforts have been made to recreate the ionospheric morphology and evolution as realistically as possible through empirical modeling and numerical simulations (e.g., Chiu, 1975; Roble et al., 1988; Huba et al., 2000; Schunk et al., 2003, 2004, 2005; Aa et al., 2015; Lin CY et al., 2020; Tang RX et al., 2020; Dang T et al., 2020). A systematic assessment of model specifications benefits both practical applications and scientific research.

In recent decades, several empirical ionospheric models have been established (Hernández-Pajares et al., 2009; Goss et al., 2019; Calabia and Jin SG, 2020; Li ZS et al., 2020). The empirical model of the International Reference Ionosphere (IRI), developed by using ionosonde, incoherent scatter, rocket, topside sounder, and various satellite observations (Bilitza et al., 2016), provides global-scale ionospheric parameters from the lower boundary to a user-specified upper boundary and is widely utilized in the ionospheric community. The three-dimensional electron density is also mapped by combining fully polarimetric synthetic aperture radar (SAR) images and the IRI model, with good performance (Zhu W et al., 2020). Nevertheless, various efforts have illustrated that the IRI model is still subjected to in-depth study. Wang C et al. (2016) noted that significant discrepancies exist between the IRI data and the observed values, particularly during geomagnetic storms. Studies evaluating IRI-2012 (Akala et al., 2015) and IRI-2016 (Rajana et al., 2022) against the global positioning system (GPS) TEC also highlight the need for continued IRI improvement. Similar GPS TEC comparisons have quantified accuracy limitations in the NeQuick model (Jodogne et al., 2005; Coïsson et al., 2006; Bidaine and Warnant, 2010; Wang H and Zhang KD, 2017). Notably, empirical models provide well-parameterized climatological behaviors instead of their dynamic evolution.

The International GNSS (Global Navigation Satellite System) Service (IGS) assimilates diverse observations to generate and provide real-time TEC at a global scale (e.g., Wang L et al., 2018; Schmölter et al., 2021; Chen Z et al., 2022), which are widely used in practical applications and scientific research. Yang JL et al. (2023) analyzed the performance of the global dilution of precision and positioning for the BeiDou Navigation Satellite System Phase III (BDS-3). Alkan et al. (2022) examined the performance in real-time Precise Point Positioning (RT-PPP) applications in Antarctica. Moreover, the IGS products are used in understanding and monitoring ionospheric behaviors, including variability investigations, earthquake responses, and empirical model development (Stankov et al., 2010; Chakraborty et al., 2015; Şentürk, 2020; Şentürk et al., 2021). However, the more than 400 IGS stations display a nonuniform global distribution, with a disproportionate concentration in the Northern Hemisphere and Europe/North America (Zhang Q and Zhao QL, 2019). The limited observational coverage with only 7 stations across China results in considerable challenges in validating extant models or constructing regional Chinese ionospheric models when leveraging IGS Global Ionosphere Map (GIM) products.

In this work, a systematic validation of IGS-GIM products in China is conducted by using independent Crustal Movement Observation Network of China (CMONOC) observations from 2013 to 2014. The mean error (ME), root mean square error (RMSE), and normalized root mean square error (NRMSE) are selected to quantify the suitability of IGS-GIM products. Section 2 briefly details the observations and their data preprocessing. Section 3 presents a comparison between the IGS-GIM and CMONOC-TEC. The results reveal a significant local time and latitudinal dependence and seasonal variation in discrepancies of the IGS-GIM products. Section 4 provides a summary of the main findings.

2.   Data and Method

The IGS is a collaborative organization founded by a consortium of universities, research centers, and geodetic and space agencies worldwide. Its primary mission is to provide the highest quality GNSS data and related products on a freely accessible basis for scientific advancement and public benefit (Dow et al., 2009; Montenbruck et al., 2017). The GIMs used in this study are among the IGS products released by the Center for Orbit Determination of Europe from IGS products based on the expression of spherical harmonic functions, and they are commonly used for ionospheric studies. The time interval of GIMs within each day is 2 h, which produces 2-hourly snapshots of the global ionosphere. The spatial resolution of each TEC map is 5° × 2.5° in the geographic longitude and latitude, respectively. The red circles in Figure 1 show the distribution of IGS stations over China region, which illustrates an uneven distribution and results in sparse data coverage.

Figure  1.  Distribution of ground stations of the IGS (red circles) and CMONOC (black triangles), respectively, over China region.

DownLoad: Full-Size Img PowerPoint

The TEC measurements from approximately 260 stations of the CMONOC are used to quantify the accuracy of IGS-GIM products over China region in the years 2013–2014. The CMONOC stations are shown by the black triangles in Figure 1, with the region covering from 70°E to 140°E in longitude and 15°N to 55°N in latitude. Slant TEC (sTEC) represents the integrated electron density of the ionosphere along the line of sight from the satellite to receiver, which is estimated by using the carrier-phase signal observations from ground-based GPS stations (Thomas et al., 2018; Ren XD et al., 2019). The equation is expressed as

where ${f_1}$ and ${f_2}$ are the carrier-phase frequencies of GPS signals, ${\lambda _1}$ and ${\lambda _2}$ represent the corresponding wavelengths, and ${L_1}$ and ${L_2}$ stand for the carrier phases. Furthermore, $N$ is the ambiguity of the ray path, $b$ and $d$ are the instrumental biases of the carrier phase and pseudo-range of the derived signal, and $\varepsilon$ is the random residual of the signal. The GPS sTEC is converted to vertical TEC (VTEC) by using a geometry mapping function (Huang FQ et al., 2016). The equation is expressed as

where R is the Earth radius, H represents the height of the ionosphere, and Z stands for the satellite elevation angle for the point of observation.

Given the spatial distribution of CMONOC stations, measurements in the region within the longitudes of 90°E–120°E and latitudes of 20°N–45°N are utilized in the following investigation. Because the time resolution of the CMONOC-TEC is 30 s, CMONOC measurements within 6 min centered at the current time of GIM products are first collected, and these time-filtered data are then interpolated into the GIM grids, that is, spatial bins with a resolution of 5° (longitude) × 2.5° (latitude). As a consequence, the ME, RMSE, and NRMSE are utilized to quantify the uncertainties in IGS-GIM data over China region. The following equations express the computation (Lin CH et al., 2010; Wang H and Zhang KD, 2017):

where n is the sample number at each spatial grid, $\mathrm{T}\mathrm{E}{\mathrm{C}}_{\mathrm{I}\mathrm{G}\mathrm{S}}$ represents the IGS-GIM, and $\mathrm{T}\mathrm{E}{\mathrm{C}}_{\mathrm{C}\mathrm{M}\mathrm{O}\mathrm{N}\mathrm{O}\mathrm{C}}$ is the CMONOC-TEC. In this study, given that ME and RMSE are both susceptible to the influence of background values, the NRMSE has been designated the primary error assessment metric.

3.   Results and Discussion

Figure 2 gives the monthly average of the IGS-GIM and the CMONOC-TEC over China region, associated with their deviations, at 12:00 universal time (UT) for June and December in the year 2013. As shown in Figure 2a, the IGS-GIM decreases with an increase in latitude. In addition, a clear longitudinal variation exists at low latitudes, which could be a reference to the local time dependence. Figure 2b presents the situation of the CMONOC-TEC. As shown, the CMONOC-TEC also displays a similar significant latitudinal characteristic. However, the TEC is much weaker at high latitudes in Figure 2b than in Figure 2a, and the longitudinal structure at low latitudes is smoothed. The $\Delta \mathrm{T}\mathrm{E}\mathrm{C}$, defined as $\Delta\mathrm{T}\mathrm{E}\mathrm{C}=\mathrm{I}\mathrm{G}{\mathrm{S}}_{\mathrm{T}\mathrm{E}\mathrm{C}}-\mathrm{C}\mathrm{M}\mathrm{O}\mathrm{N}\mathrm{O}{\mathrm{C}}_{\mathrm{T}\mathrm{E}\mathrm{C}}$, is introduced to describe the deviation between these two TEC datasets. As shown in Figure 2c, the positive peak of $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$ appears at 45° latitudes with an amplitude of more than 6 TEC units (TECu), implying that the IGS-TEC is overestimated compared with the CMONOC-TEC. Notably, the peak of $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$ does not occur where the background values are the largest. However, the situation is opposite in the 20°–25° latitude region, where the $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$ is negative, suggesting an underestimation of the IGS-TEC there. Additionally, a significant longitudinal variation occurs at different latitudes, indicating a significant local time dependence of the error distribution. Figures 2d–2f are similar to Figures 2a–2c except for the situation in December of 2013. The latitudinal trends are similar to the ones in June, which also exhibit a significant latitudinal dependence. Moreover, a comparison of the situations in June and December suggests a remarkable seasonal dependence in the uncertainty of the IGS-GIM products over China region.

Figure  2.  Variation in the monthly averaged TEC (or $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$) at 12:00 UT in 2013 as a function of the geographic longitude and geographic latitude. The top panel is for June and the bottom panel is for December. From left to right are the IGS-TEC, CMONOC-TEC, and their differences.

DownLoad: Full-Size Img PowerPoint

Since the longitudinal variation refers to the local time dependence in this work, the dependence of the variables of ME, RMSE, and NRMSE on the local time, latitude, and seasons are further investigated. Figure 3 depicts the variations in ME, RMSE, and NRMSE as a function of latitude and local time in different months. As shown in Figure 3a, the ME is significantly larger at higher latitudes before 10:00 local time (LT), and the maximum is approximately 8 TECu. This situation is quite different after 10:00 LT, when the ME peaks at a latitude of ~30° with an amplitude of approximately 8 TECu. In addition, the ME is negative with a minimum of −8 TECu in the 20°–25° latitude region, and the amplitude of the negative peak at a latitude of ~35° is approximately −2 TECu. Figure 3b illustrates the situation in June, which depicts a similar pattern as the one in Figure 3a, except that the magnitudes in Figure 3b are relatively small. Figure 3c is similar to Figure 3a except for the situation in September. However, the entire pattern is close to the one in March, with a comparable magnitude. The situation in December is shown in Figure 3d, which again depicts a similar variation, except that the 30° peak at 15:00 LT in Figure 3a shifts to a lower latitude, and the peak amplitude is approximately 7 TECu.

Figure  3.  Variations in $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$ as a function of the local time and geographic latitude. From top to bottom are the ME (a–d), RMSE (e–h), and NRMSE (i–l), respectively.

DownLoad: Full-Size Img PowerPoint

Figures 3e–3h show the local time and latitudinal variations of the RMSE. The RMSE exhibits an equatorward-extending trend that the peak time is generally earlier at higher latitudes. In March (Figure 3e), the peak of the RMSE in the 20°–30° latitudinal region occurs at 20:00 LT with an amplitude of 14 TECu, whereas the maximum of 35°–45° latitudes appears around 8:00 LT, and the peak value is approximately 7 TECu. The RMSE is comparably low in June (Figure 3f). The noticeable maximum in June occurs at 15:00 LT in the low latitudinal region of 20°–30°, and the magnitude is approximately 6–8 TECu. Figures 3g and 3h depict the situations in September and December, respectively, which are quite similar to the ones in March.

Figures 3i–3l illustrate variations in the NRMSE. The most significant phenomenon is that the maximum NRMSE is generally located at high latitudes during midnight. Besides these noticeable local time and latitudinal dependencies, the NRMSE also shows an obvious seasonal variation. The midnight maximum NRMSE is significantly larger in December (Figure 3l), with a magnitude greater than 100%, compared with those in other seasons, and it is the smallest in June (Figure 3j; ~60%). A secondary peak also occurs at the latitudes of 25°–30° before midnight except for the situation in June.

Figure 4 exhibits the index structures evaluated at different local times, which is more beneficial for quantifying latitudinal errors in the IGS-GIM products. As shown in Figure 4a, the ME becomes higher with latitude, increasing from 2:00 to 10:00 LT, and the magnitude achieves ~6 TECu at a latitude of 45°. However, the latitudinal variations from 12:00 to 24:00 LT are significantly different at another local time. The maximum ME is more than 8 TECu at a latitude of 45°, and the amplitude of the secondary peak at ~25° is approximately 6 TECu. Figure 4b presents the situation for the RMSE. The minimum RMSE from 2:00 to 10:00 LT appears around the 25°–30° latitudinal region, whereas the one at 12:00 LT occurs around 35° latitude. It is noteworthy that this minimum shifts toward lower latitudes in the post-noon period. The variations in the NRMSE (Figure 4c) exhibit a local time dependence similar to that observed for the ME. The peaks of structures increase with latitude, whereas the secondary peaks from 14:00 to 24:00 LT occur around 25°–30° latitude.

Figure  4.  Latitudinal variations in $\Delta \mathrm{T}\mathrm{E}\mathrm{C}$ under different local times. From left to right are the ME, RMSE, and NRMSE, respectively.

DownLoad: Full-Size Img PowerPoint

Figure 5 presents variations in the ME, RMSE, and NRMSE as a function of the month and latitude at different local times. As depicted in Figure 5a, the ME is markedly larger at higher latitudes, peaking in March and October with an amplitude of approximately 7 TECu. In addition, the ME is negative in summer at low latitudes, with a minimum of −6 TECu. The pattern at 8:00 LT (Figure 5b) is quite similar but with more pronounced magnitudes. In contrast, the variation in ME at 14:00 LT (Figure 5c) exhibits significant differences from the morphology at 2:00 LT. Compared with 2:00 LT, the two peaks shift to middle–low latitudes with a maximum of ~7 TECu. Moreover, a negative ME occurs in the 20°–25° latitude region. The morphology at 20:00 LT (Figure 5d) resembles that at 14:00 LT except that the peaks extend into the winter months. Additionally, the peaks at 20:00 LT are the highest compared with those at other local times, with an amplitude exceeding 8 TECu.

Figure  5.  Variations in $\Delta \mathrm{T}\mathrm{E}\mathrm{C}$ as a function of the month and geographic latitude. From top to bottom are the ME (a–d), RMSE (e–h), and NRMSE (i–l), respectively.

DownLoad: Full-Size Img PowerPoint

The seasonal and latitudinal variations in the RMSE are shown in Figures 5e–5h. At 2:00 LT (Figure 5e), the RMSE maximum of ~7 TECu emerges at ~45° latitude in April and October, whereas the minimum of 3 TECu occurs in July at 30° latitude. The overall structure at 8:00 LT (Figure 5f) resembles that at 2:00 LT, with some differences. Notably, the RMSE at 8:00 LT is comparatively higher, with a peak of 8 TECu in October and a slightly weaker maximum of 7 TECu in April. In contrast, the morphology at 14:00 LT (Figure 5g) diverges substantially from that at 2:00 LT. Compared with 2:00 LT, distinct peaks in the RMSE materialize around the equator with greater magnitudes, exceeding 11 TECu, in March and November. The distribution at 20:00 LT (Figure 5h) mirrors that at 14:00 LT but with relatively larger amplitudes.

Figures 5i–5l exhibit variations in the NRMSE. As shown, the NRMSE maxima materialize in the 40°–45° latitude region, with markedly higher peak values at nighttime (Figures 5i and 5l) compared with daytime (Figures 5j and 5k). Taking 20:00 LT as an illustrative example, Figure 5l shows the NRMSE peaking in the winter months, with magnitudes exceeding 80%, whereas the minima occur in July. An additional secondary peak emerges in winter at 30° latitude.

The ME, RMSE, and NRMSE at different local times are presented in Figure 6. For more direct comparisons, representative local times (i.e., 6:00, 12:00, 18:00, and 24:00 LT) are selected to quantitatively depict the seasonal variation. As shown in Figure 6a, the ME has two peaks in the first half and second half of the year, with the November maximum exceeding that in the first half year. The RMSE pattern (Figure 6b) resembles that of the ME aside from some key differences. Notably, the RMSE is higher during the daytime, whereas the ME exhibits the opposite dependence. Regarding the NRMSE (Figure 6c), the nighttime values are again markedly higher, with maxima concentrated in the winter months.

Figure  6.  Average seasonal variations in $\Delta\mathrm{T}\mathrm{E}\mathrm{C}$. From left to right are the ME (a), RMSE (b), and NRMSE (c).

DownLoad: Full-Size Img PowerPoint

As discussed previously, the errors between the IGS-GIM product and the CMONOC-TEC exhibit local time and latitudinal dependencies, along with significant seasonal variations. Notably, the IGS-GIM dataset analyzed coincides with periods of high solar activity, potentially contributing to the observed deviations. Apart from solar influences, the modeling approaches used by IGS institutions further affect the accuracy of the IGS-GIM. Figure 7 shows the distribution of geomagnetic declination and geomagnetic latitude in geographic coordinates. The data are derived from the International Geomagnetic Reference Field. As depicted in Figure 7, the discrepancies between geographic (gray lines) and magnetic coordinates (blue and red lines) also introduce a considerable modulation of the ionospheric plasma owing to the ion drag along the geomagnetic field lines (Zhang SR et al., 2011; Hu J et al., 2022). The GIM products are superposed by the spherical harmonic functions requiring substantial measurements, whereas this is insufficient for IGS station coverage over China. This implies that interpolation of the TEC using sparse observations and spherical harmonics is ill-posed, yielding an unrealistic overestimated or negative TEC, degrading the model reliability. Additionally, intense atmospheric disturbances induce pronounced, intricate electron density fluctuations that cannot be captured rapidly by IGS models, inducing deviations from the CMONOC-TEC. However, the reliability of IGS-GIM products in other years and regions warrants further investigation.

Figure  7.  Discrepancy between the geography and magnetism in China and adjacent areas. The gray lines indicate geographic coordinates, and the red and blue lines represent magnetic latitude and declination, respectively.

DownLoad: Full-Size Img PowerPoint

4.   Conclusions

This study evaluates the accuracy of IGS-GIM products from 2013 to 2014 in China by comparing them with CMONOC measurements. Discrepancies between the CMONOC-TEC and IGS-GIM are quantified by using the ME, RMSE, and NRMSE metrics. The key findings are as follows:

(1) A distinct local time dependence is evident in IGS-GIM deviations, with a higher average NRMSE (39%) at nighttime compared with other times (19%).

(2) After averaging by the local time, notable latitudinal variations emerge in IGS-GIM errors, with a peak NRMSE of 49% occurring above 40° latitude, whereas a minimum NRMSE of 18% is observed within the 20°–25° latitudinal band.

(3) Significant seasonal differences manifest in IGS-GIM deviations, with larger biases in spring (NRMSE: 30%) and autumn (37%) versus summer (20%).

(4) Discrepancies in the IGS-GIM likely stem from the geomagnetic environment and sparse observations in China.

These results elucidate the characteristics of IGS-GIM error variations over China. However, the limited measurements herein necessitate further investigation into the mechanisms underlying the identified biases.