The forecasting of solar flares is critical given the range of their terrestrial impacts - from power outages (such as occurred in 1989 in Ottawa) to disturbance of aircraft navigation. The Brier P-Score is one of the first methods - applied to statistical flare forecast evaluation. It was developed in 1950 as a “proper” assessment technique for flare prediction. By way of comparison, an “improper” method would be illustrated if a forecaster were to issue a ‘no flare’ forecast (say for major flares) every day of the year and only 5 events occurred. Then, by improperly counting the ‘no flare’ days as actual events a 99 percent success rate could be arrived at.
The standard Brier P-Score is defined (as I showed in my first statistical flare forecasting paper published in The Journal of the Royal Astronomical Society of Canada):
where P is the verification score, M is the number of forecasts made, k is the number of categories for each forecast occasion, and f is the forecast probability with range 0- 1 in each category.
The
observation is denoted by the letter O and may be zero (0) (event i in category
k does not occur) or 1 (event does occur).
Mathematically, the smaller the forecaster’s score the greater his skill
– since the less difference between what is forecast and observed (the squared
term at the end)
A more refined modification due to Saunders (1963) takes into account more factors than the simplified version above, but we will focus on the simpler version.
Now, as to a specific application. Consider the interval April 5 – 11, 1980 when I actually made ex post facto predictions that were later checked using the P-Score. The results are tabulated as follows and these are for “major SID flares”. E.g. flares that produced an SID event or sudden ionospheric disturbance, of at least importance ‘2’ on a 0-2 scale.
Date 4/5 4/6 4/7 4/8 4/9 4/10 4/11
Obs. 2 1 2 0 0 2 0
Pred. 0 1 2 1 0 1 0
f_ik 0 0.2 0.5 0.2 0 0.1 0
= 1.0
A P-score of 0.48 resulted from this example. Again, this is raw and just to show how the basic score works. As I noted there are ways to refine it. More recently (1979) Simon and Smith (Solar –Terrestrial Predictions Proceedings, Vol. II, p. 311) have noted that forecast accuracy can be fundamentally limited by Poisson statistics, e.g. the type that yield the Posson distribution:
Where P N is the probability corresponding to N flare days of the observed magnetic class (N = 0, 1, 2 etc.) and l is the mean number of flares per day per magnetic class. Then the expected frequency of N flare days is found from:
E( d N ) = P N å d N
Where E( d N ) is the expected number of N flare days, and the summation refers to the total number of recorded flare days for the particular magnetic class. For any given magnetic class the extent of agreement between observed and expected flare days is calculated from:
c2 = å [ O(d N ) - E( d N ) ] 2/ E( d N )
It is possible, if such considerations had been applied to the example above, the P-score would have been significantly improved – since fewer predicted flares would have been assigned on those days when fewer occurredMy second paper (published in Solar Physics, 1984 ) examined the specific statistics pertaining to frequency of occurrence and associated intensity. This began with using the Poisson equation for probability:
In this paper I applied a further index of goodness of fit, obtained by comparing the statistical moments M n with the predicted values for the Poisson theoretical distribution. The moments about the mean ( l)are then given by:
Where n = 2, 3. 4 etc. and f j (j= 1, 2, ...k) = f (No) denotes the observed distribution of N flare days for the observed magnetic class. For n = 2, for example, we obtain M 2 = s2 or the mean squared deviation from the mean (variance) which is a measure of the spread of f (No) ; for n = 3 we obtain M 3 = d3 or the cubed deviation from the mean, i.e. the skewness of f (No)
.For a theoretical Poisson distribution of form:
We expect: M 2 = l , and a 3 = M 3 /(M 2 )1/2
But if these are appreciably different from the observed values a modified form of the theoretical Poisson distribution must be used, i.e.
Where x /h = l and ( l + l/h ) = M 2 . As with the theoretical Poisson form the goodness of fit may be assessed by using the c2 distribution.
Suffice it to say, the preceding statistical aspects were critical in disclosing the need to incorporate a flare trigger to account for the different SID effects. One of the major findings on analysis was that: i) Subflares - with typical energy 1029 erg, were the major producers of SID flares, and (ii) 35% of the major SID flares (greatest geo-effective impacts) were optical subflares.
These results in turn disclosed the basis for a Poisson-based "delay time" and magnetic free energy (MFE) buildup preceding geo-effective solar flares, paving the way for a flare trigger. Thereby it was shown how the flare distribution actually corresponds to a time-dependent Poisson process of the form:
P(t) = e -l lt / t!,
where theoretically the Poisson mean rate of occurrence is: lm = l Dt, with Dt = t, assuming the time interval Dt = 1d. Since magnetogram measurements referred to solar active regions -sunspot groups will not generally be made at the exact same time each day this ensures Dt ¹ 1d, so Dt ¹ t thereby introducing a selection effect variability. It is this inherent variability which opens the door as it were to the need for the modified Poisson distribution.
If MFE buildup was large, but the energy release (triggering) 'premature' (t <<t', time of prediction) a subflare could then occur but with terrestrial effects (e.g. short wave fadeouts or SWFs). If the MFE buildup was large and triggering delayed enough to discharge most of all of it, then major impacts occurred, such as powerful magnetic (auroral) substorms.
These consequences were first postulated by me (Proceedings of the Second Caribbean Physics Conference, Ed. L.L. Moseley, pp. 1-11.) to account for the intermittent release of magnetic free energy in large area sunspots, using:
1/m ò V div ( v X B) X B )dV - ò V han Jms 2]
Where han is the anomalous resistivity given by Chen (1974)[i]:
h an = 4pneff/ we
where neff is the effective collision frequency and we is the electron plasma frequency. And Jms the current density at marginal stability of the magnetically unstable region. Bear in mind that ( v X B) X B reference relative footpoint motion within the large active region.
The plasma response to the rotary motion is accounted for by a (-J·E) term (or the E·J term, since -J·E = E·J). The change in total energy over a defined volume V may then be written (using appropriate identities of curl, div):
òv [¶ e /¶t] dV = òv [E curl H – H curl E] dV - òv [J·E] dV
This work led directly to one of the first semi-successful uses of the Brier P-score to predict flare occurrence [ii] followed by publication of the key statistical results in the Meudon Solar-Terrestrial Predictions Proceedings [iii].
See Also:
Why Space Weather Is Still "Something of a Black Box"
And:
New Solar Research Confirms Why Delta Sunspots Are More Flare Worthy Than Other Magnetic Classes
And:
Analysis of Helicity Variation Via Collision of 2 Solar Loops In Relative Proximity (Pt. 1)
And:
