This is the presentation Dr. Lucy Hancock gave at a session on space weather at the Annual Meeting of the American Meteorological Society’s session on Space Weather, in January, 2022.
Dr. Hancock presented an anomaly in solar radiation data, and suggested it be explained as the effect of space dust shading the line of sight Sun to Earth.
The significance of space dust shading Earth is that it would be a climate driver not explicitly known to atmospheric models, although it may be implicitly known as a calibration estimated by machine learning. Machine learning would incorporate it in short-term forecasts but not in climate forecasts.
Therefore, if this driver exists, we can improve medium and long-term forecasting by knowing of it.
There is hardly anything more helpful we can do to raise farming income in the tropics, than improve forecasts in medium and long-term time frames.
So let’s see… Figure 1
Figure 1. CWOP stations
The dataset I used here was the archive of solar radiation measurements collected by the Citizen Weather Observer Program (CWOP).
The CWOP sensor array is global, as shown Figure 1. The network has about eight thousand sensors reporting now. Besides these, about two thousand more contributed in the past. The sensor array is global although it is denser in the Northern Hemisphere.
Figure 2. Number of reports contributed (by day)
Figure 2 shows how many observations were contributed to the CWOP archive each day, day by day since 2009. If you add it all up, the database now comprises about a billion observations
Figure 3. The most common radiation sensors and software used in the CWOP network.
As Figure 3 shows, stations in the CWOP network use a variety of sensors and software. That variety is important, to clarify that the anomalies we find are not attributable to some manufacturer’s quirks.
The analyses offered today were originally aimed at automating quality management for the CWOP database by comparing contributed observations to model values.
Figure 4. Comparing observations from NOAA Table Mountain station to modeled solar radiation. Data is blue, model is yellow.
Figure 4 shows a few days of data from NOAA’s Table Mountain pyranometer (blue) compared to the Bird clear sky model (in yellow). The model values are obtained using meteorological data from the sensor, and parameterizations for unmeasured variables that are set at average values. The model value incorporates only a minimal correction for atmospheric aerosols (specifically, we assume atmospheric visibility is 90 km).
Figure 5. Comparing observations to model values – several climates, multi-year
Figure 5 presents a comparison of contributed observations from several different climates, to the respective model values. Data is blue, model is yellow. We see that, by and large, there is a fit but there are differences – what is the best way to summarize and analyze them?
Figure 6. Definition of optical depth
The conventional way to automate the comparison between observation and clear-sky model is to calculate optical depth. (See Figure 6.) Optical depth 𝜏 (tau) is going to be everything not included in the model, or that the model mis-assessed. So it will be the sum of
- ● shading by aerosols … because we did not include it in the model,
- ● optical depth of clouds … ditto,
- ● atmospheric glinting … ditto,
- ● model issues if any, such as misparameterizations, and,
- ● quirks of individual sites – that somehow do not conform to model assumptions (sensor was tarnished or tilted or miscalibrated or values were outside calibrated bounds, or site was shaded, etc.).
Figure 7. Left: all the optical depth 𝜏 values for one station plotted at respective sun position. Right: a photo of the southern sky at the same station.
Figure 7 shows that optical depth 𝜏 values accumulate a “sunprint” of environmental shading as seen by the solar radiation sensor. On the left are all the optical depth values for several years from one station (near Philadelphia, in case you want to check seasonality), each plotted at the respective altitude and azimuth of the sun at the moment of the observation. On the right is a photo of the southern sky taken at the same station. The optical depth 𝜏 values have accumulated a “sunprint” of the sky: the same trees, the same buildings. The sunprint is different from the photo, however, as it is a kind of Four Seasons image: the deciduous trees have leaves on them during summer (higher part of the image) but are bare branches in winter (lower part).
Figure 8. “Sunprints” for two other stations in the CWOP network, each made the same way as Figure 7, by accumulating values of optical depth 𝜏.
Figure 8 presents similar “sunprints” from two other stations. We see something like a telephone pole on the left, fir trees on the right.
These three images give us a feeling for the optical depth calculation. Namely, optical depth 𝜏 is showing us how much sunlight has been somehow scattered away, additional to the scattering that the model expects. This value also includes various types of error, but to first order,
𝜏, or “optical depth” is just shading, scattering, extinction of sunlight.
These figures deducing environmental shading from 𝜏 have helped us make progress toward quality management, but … not enough progress. Our goal is to automate quality management. We have eight thousand stations to assess! It is nice to see and understand the shading issues at individual stations, but we can not automate quality management for eight thousand stations by examining images one by one. Furthermore, we would want to distinguish year from year at each station. Sometimes a station has a problem (rain gauge shading it, for example) and then after a year or two the station’s managers realize the effect of the shade and move the rain gauge. We’d like to distinguish those differences automatically. Visual inspection of these sunprints is not a practical approach to quality management.
Top left: Sunprint as in Figure 7. Top right, Photo as in Figure 7.
Bottom left: The sunprint divided into six sections.
Bottom right, probability density function (PDF) of optical depth (𝜏) values for each section.
In order to automate station inspection, we decided to plot each station’s distribution of optical depth (𝜏) values – namely, the probability density function. By inspecting the PDF of 𝜏, we can automate a hunt for anomalies in our stations.
So we need to establish a reasonable expectation for the PDF of 𝜏.
(And hang on, because this step is where the anomaly turns up…)
In Figure 9, the sunprint is divided into six areas. The PDF of 𝜏 is shown for each area, so that for this one sunprint we have six sections to look at.
The density functions in these sections vary, naturally, since the six areas represent different times of the day, shading from different trees and buildings, and different proportions of the seasons and their different clouds.
But the PDFs have similarities from section to section. The most striking similarity is the nearly ubiquitous presence of two peaks, one of which has a constant value. Each PDF section usually has two peaks, one narrow peak centered somewhere around 7.5 percent, and one wide peak, centered at some larger value. Just one PDF, from the far left, is missing the narrow peak at 7.5 percent. That area is mostly in shade. So the absence of the “spike” here seems to be the result of all that shade from trees and buildings.
Figure 10. PDF of values of optical depth, for a million randomly selected values over the whole database.
Now we’d like to go beyond analyzing just one station. We’d like to look at the database as a whole. Figure 10 is the PDF of 𝜏 for a million randomly selected observations from the whole database. This figure shows the narrow peak at around 7.5 percent that we saw at the Philadelphia station, and a wide peak centered among large values.
This PDF highlights discrepancies between model and observations. Let’s notice that if the PDF were presented at low resolution, we would not see two separate peaks – we would see one peak near zero, trailing off to the right. It would appear to confirm that the model is about right – a lot of observations with discrepancy distributed around zero, and a long tail to the right representing the ever decreasing likelihood of larger discrepancies.
Although verifications are not carried out quite so crudely, the point persists: It is the large and varied CWOP database that will enable us to make the statistical distinctions which establish that there is some universal and persistent anomaly relative to the solar radiation model.
Figure 11. PDF of 𝜏 values at different sun zenith angles
Figure 11 shows the different distribution of 𝜏 values that you get if you vary sun zenith angle.
~*~[If you imagine a pole going straight up from your head to the sky that is an approximation of the zenith. The sun zenith angle is that angle between that imaginary, straight-up pole, and your line of sight to the sun. If the sun were straight ahead, rising or setting near the horizon, the number would be large.]~*~
The figure on the left shows the distribution of all the extinction (𝜏) values for observations collected such that the sun was 50 degrees down from the zenith.
The middle figure is the distribution of the extinction (𝜏) values for observations collected when the sun was about 70 degrees down from zenith.
And the far right figure is the distribution of extinction (𝜏) values for observations collected such that the Sun was about 80 degrees down from the zenith – ten degrees above the horizon.
So from left to right, you are looking at the sun through more and more atmosphere.
We see the same two peaks in each of these figures, but the narrow peak is a smaller share near the horizon. There is more and more sunlight being knocked out of the observations. And we would expect that. What is surprising is that the centroid is not changing. We are using a statistical analysis to find the centroid of the two peaks, and the centroid of the left-side peak is not moving.
The amazing thing is that you would expect more and more extinction as you get near the horizon, ~*~[When you look straight up into the atmosphere you are looking through about five miles of atmosphere but when you look almost horizontally, the atmosphere is well over 100 miles thick.]~*~ and yet, there is this spike of values (on the left side of the figures) where that is not happening. It is ~7.5% off. The peak is always just about that. No more. Same as if you were looking straight up at the zenith. How can that be?
Is this a software problem, some glitch that puts out 7.5% routinely? Remember the station figure, where that spike does NOT appear in one distribution (Fig. 9), the shaded distribution. So the software can produce a distribution without this spike. Due to that station, we understand what are the conditions for producing it.
Figure 12. Centroids of three contributing distributions, as a function of sun zenith angle
Figure 12 quantifies the steadiness of the left-side (spike), which is the lowest line of symbols. Its centroid is something like 7.5 percent shading, even for data taken nearer and nearer to the horizon. This quantifies the visual observation just made, namely that the centroid of the spike distribution is near 7.5 percent and doesn’t change much with sun zenith angle. The peak moves but the centroid which takes the other peak into account does not move.
Figure 13. Centroids of contributing distributions for each day’s data.
Figure 13 analyzes the centroids in daily data slices, so here we are looking for a dependence on time. The lower line shows the daily values of the centroid of the “spike distribution” somewhere near 7.5 percent. In this figure, we see that the centroid does vary with time – it is oscillating. Its value is still quite steady, but it is oscillating.
Figure 14. Centroid of the “spike” distribution day by day, in expanded scale.
In Figure 14, where we increase the scale to magnify that oscillation, we see that it is an approximately annual oscillation. The 7.5 percent centroid varies from 5 to 10 percent. There is more extinction in the summer; less in the winter.
Figure 15. Standard deviation of the distributions, day by day.
Figure 15 shows the standard deviation of the distributions, and as usual the spike distribution is at the bottom. This is to quantify what we saw before, which is that this distribution is very narrow. In this day-by-day presentation, it is even narrower. Once the annual variation is backed out, this distribution is very narrow, possibly comprising a few yet-narrower component distributions, sometimes but not always distinct.
Figure 16. All the Gaussian components identified in the day-by-day analysis, plotted as mean value on the X-axis, standard deviation on the Y axis, and proportion of all observations (that day) as color.
Figure 16 summarizes all the distributions presented in Figures 14 and 15, each as one dot. These dots have a strong clustering. The physical meaning of the clustering is that all the lines of sight from Sun to Earth can be categorized as just a few statistical types.
About 30 percent of all observations are represented by that tadpole on the lower left, with shading about 7.5 percent. All the other distributions are components of the wide peak.
The existence of this fat dot, the head of the tadpole, means that the sum total effect of shading by clouds, aerosols, and everything else … has a favored value of 7.5 percent shading (extinction). Such observations comprise a cluster, statistically distinct from the rest of the database, apparently about 30 percent of the whole [all observations].
This cluster of extinction values is characterized by
- – Seasonal variation in mean value, from 5 percent to 10 percent,
- – Where larger values occur in the summer,
- – Independence from sun zenith angle,
- – Statistical isolation
What source of extinction would have these statistical characteristics?
*The lack of dependence on sun zenith angle suggests the extinction is not due to an atmospheric component. It is above the atmosphere. *The seasonal variation suggests the structure is oriented to cause more shade in summer. *The statistical isolation of this cluster, and the fact it is the smallest possible value of extinction anywhere in the database, could be explained if it is always present. All other causes comprise distributions whose centroid and standard deviation are added to this one. *The structure is large enough relative to any internal unevenness that its net effect is steady.
Figure 17. Path of solar radiation from Sun to Jupiter, reflected to Earth, would encounter a Sun- orbiting dust structure three times.
Let’s consider possible dust structures persisting in the line of sight from Earth to Sun. Would the structure be orbiting Earth or Sun? The likelier hypothesis seems to be a dust cloud orbiting Earth. A Sun-orbiting structure causing extinction of this magnitude along the line of sight Earth to Sun, would [also] cause at least double or triple this magnitude of extinction for solar radiation reflected from the outer planets. It would vary with on solar elongation, a strong dependence that is not observed.
A ring of little rocks imposing this much extinction would have been observed, but a ring of dust particles (for example, size 100 microns or smaller) could have a significant optical effect without much mass and without being detected by debris surveillance.
The ring of dust that causes the “spike” can’t lie in the plane of the Earth’s equator, because such a ring would shade the winter half of the planet, not the summer half. But planetary rings can also lie in the plane of a moon’s orbit. An example of this is Saturn’s Phoebe ring. Consider that Earth’s Moon lies within a few degrees of the ecliptic. and so an Earth ring in the plane of our Moon would be tilted the same way, just a few degrees from the ecliptic. It would, indeed, shade Earth more in summer than in winter.
The ring must be fuzzy because a sharp ring would have been identified (as a density jump in particle counts in the near-Earth Environment). But fuzziness is to be expected! The effects that flatten a planetary ring depend on orientation in the planet’s equator, and on oblateness [flattening at the poles] of the planet. Neither condition applies here; the Earth is much more nearly round than Saturn or the gas giants.
The ring of dust must be old, since the dust collection program has not identified very much “exotic” dust in the near-Earth space environment. In that case the dust must have fallen to Earth over a time frame long enough to be identified as “terrestrial”, although it is not.
The only source of dust that would persistently replenish the near-Earth environment is the Moon. It’s possible – there is no longer a consensus that the Moon is dead.
Going forward, the next step would be to search for the infall of such dust among the weather records of the tropics – perhaps driving wind, rain, lightning, wildfires and drought (particles cause rainfall, too many particles cause drought).
Even a faint, light, fuzzy ring, may have consequences for weather. If so let us find them.
~*~[For a helpful diagram of the ecliptic try this article from Sky and Telescope.]~*~
What Is the Ecliptic?
Updated Saturday with Dr. Hancock’s corrections.
Dr. Hancock has a Ph.D in Physics from UNC Chapel Hill and has consulted for the World Bank on meteorological questions.
(Full disclosure. She is my sister.)