US nighttime lights data, 1994 (DMSP, composite from 29 orbits)

US nighttime lights data, 1995 (DMSP, composite from 236 orbits)

Overview

Nearly all production and consumption after sundown requires (and emits) light

  1. Household appliances
  2. Office equipment
  3. Transportation
  4. Sporting events
  5. Street markets
  6. Restaurants
  7. Factory operations
  8. Roadworks
  9. Construction
  10. Studying and homework

 

Remote sensing of nighttime lights allows us to observe human activity from space


Emitting light

Applications of Luminosity Data

Economic activity, income, growth

  1. Persistent economic data challenges
    1. in most countries, GDP stats not available at the sub-national level
    2. national accounts & surveys are sometimes unreliable
    3. some locations too costly, dangerous for field & survey work
  2. Luminosity has some advantages
    1. more spatially fine-grained than standard income accounts
    2. regular, frequent time intervals
    3. recorded homogeneously across national borders
    4. available in hard-to-reach areas
  3. Not a bad proxy measure
    1. nighttime lights found to be positively correlated with GDP, GRP at multiple spatial scales


 

 

Luminosity and GDP/capita

 

 

Urbanization

  1. First applications of luminosity data:
    1. delineate urban extents
    2. track electrification, street lighting
  2. Luminosity captures things surveys, census can’t
    1. growth of informal, illegal settlements
    2. communities with inadequate infrastructure
    3. emergence of new local urban centers
    4. characterize urban growth over time


South Asia, 1994

South Asia, 2010

 

 

Emergency management, recovery

  1. Luminosity captures impact of emergencies
    1. disasters cause damage to power lines, disrupt electric utility services
    2. luminosity data can detect power outages, also fires, flares, other sudden increases in light
  2. But not everything is visible from space
    1. cloud cover prevents real-time storm damage tracking and assessment
    2. luminosity better suited for monitoring non-meteorological events (e.g. earthquakes, accidents), long-term tracking of recovery efforts


Before landfall

After landfall

 

Armed conflict

  1. War creates darkness
    1. direct: power grid damage
    2. indirect: population displacement, curfews, intentional dimming to avoid detection
  2. Danger lurks in darkness
    1. power disruptions constrain economic development, harm public health
    2. fall in light emissions is warning that civilians, hospitals lack stable electricity supply
  3. Data can illuminate the physical impact of war
    1. luminosity data allow us to track impact of war on public services, utilities, urban economic activity
    2. this analysis can be near-real time


October 2021

October 2022

 

 

Technological change

  1. Outdoor lighting technology evolves
    1. wood, dung \(\to\) oil
    2. oil \(\to\) gas
    3. gas \(\to\) incandescent
    4. incandescent \(\to\) LED, HID
  2. Ecological and social impacts
    1. each change increases emissions, changes spectrum of visible light
    2. spectral change affects perception of artificial lights by animals
    3. LEDs are more easily dimmed, lighting becomes more temporally dynamic
  3. This is also a measurement challenge
    1. change in spectrum can be mistaken for change in light intensity


Alles ist erleuchtet

Varieties of Luminosity Data

Space-based sensors for night-lights (partial list)

Sensor Spatial resolution Temporal resolution Availability Free? On-board calibration
DMSP/OLS 3km Monthly 1992-2013 \(\checkmark\)
Landsat 8 30m Irregular 2013- \(\checkmark\)
VIIRS/DNB 740m Daily 2012- \(\checkmark\) \(\checkmark\)
EROS-B <1m Daily 2013- \(\checkmark\)
Jilin-1 <1m Daily 2017- \(\checkmark\)

Defense Meteorological Satellite Program (DMSP)

  1. History
    1. 1960s: starts as DoD’s meteorological program
      • goal: prevent spy satellites from wasting resources when targets are cloud-covered
    2. 1992: images available in digital form
    3. 1994: NOAA launches global nightlights program
  2. Limitations
    1. data from different years not directly comparable due to different atmospheric conditions, sensor settings and degradation
    2. temporal analysis requires intercalibration
      • use invariant pixels as training data
      • calibrate raw images to same radiometric level as base image
    3. images blurred due to scattering, saturation

 

Link: eogdata.mines.edu/products/dmsp/


DMSP-5D2

Visible Infrared Imaging Radiometer Suite (VIIRS)

  1. History
    1. 2011: sensor launches onboard Suomi NPP satellite
    2. quickly becomes industry standard
    3. main product: Day and Night Band (DNB)
  2. Advantages over DMSP
    • on-board calibration enables temporal comparisons
    • higher spatial precision
    • sensitive to lower light levels
    • no saturation in urban areas
    • cloud-free, stray light corrections
    • free daily images \(+\) monthly, annual composites

 

Link: eogdata.mines.edu/products/vnl/


Suomi NPP

Not all orbits make good data

Requirements for nighttime light observation:

  1. Center half of orbital swath
    • (i.e. area being imaged by satellite)
  2. No sunlight
    • (no high-latitude zones in summer)
  3. No moonlight
    • (within-month variation)
  4. No solar glare
    • (e.g. from solar panels)
  5. Cloud-free
    • (sorry, Seattle. hello, Dubai)
  6. No auroral emissions
    • (why Santa has evaded detection)
  7. No forest fires
    • (no intense sources of natural light)


Swath path

Illustration of cloud cover contamination

Daily VIIRS data for Ukraine, 2021-2022

Daily VIIRS data for Ukraine, 2021-2022

Vignettes

Overview of lab exercise

 

  1. Vignettes on differences in luminosity across borders
    1. Korean peninsula
    2. Island of Hispaniola
  2. Vignette on differences in luminosity over time
    1. Syrian Civil War

Vignette 1 / Korea

Vignette 2 / Hispaniola

Vignette 3 / Syria

We can obtain VIIRS nighttime luminosity (vnl) data from eogdata.mines.edu/products/vnl/

Scroll down to the “Annual VNL V2” section

Click on the “Go to Download V2.2” button for the most recent year’s data

Download file ending with .average_masked.dat.tif.gz for most recent year

Let’s also grab data for the “oldest” year available.
Navigate to the parent directory and find the annual data in the v21/ folder

There are two files ending in .average_masked.dat.tif.gz here.
Download the first one (April through December)

Let’s get some country and administrative boundaries from geoboundaries.org

Navigate to the “Individual Country Files” section

Download country-level (ADM0) data for South and North Korea (KOR, PRK)

When you unzip, the only file you need to extract is one ending in ADM0.geojson

Repeat this process for country-level (ADM0) data for Haiti (HTI)…

and country-level (ADM0) data for Dominican Republic (DOM)…

and district-level (ADM2) data for Syria (SYR)

We will be using the same event data on violence as in the last lab:
UCDP GED version 25.1, in csv format

Here is the full list of data sources and links:

Category Type Format Data source
Nighttime luminosity Raster .tif VIIRS
Administrative units Vector (polygons) .geojson geoBoundaries
Political violence Table (non-geo) .csv UCDP GED

 

These are all in the Lab10WT02.zip file posted on Canvas.

Differences in Luminosity Across Countries (Korea and Hispaniola)

Always save your progress!
Go to Project \(\to\) Save As...

Vignette 1. Load the 2024 VNL data (Layer \(\to\) Add Layer \(\to\) Add Raster Layer). VNL_npp_2024_....tif file in Data/VIIRS folder

The default color scheme is too dark. Let’s see if we can add some contrast

In the layer’s Properties, change Render type to Singleband pseudocolor and set Mode to Quantile. Click Classify and OK

This is probably too much contrast, but at least we can see the distribution

Load country boundaries for the two Koreas (Layer \(\to\) Add Layer \(\to\) Add Vector Layer). 2 files: geoBoundaries-PRK-ADM0.geojson and geoBoundaries-KOR-ADM0.geojson from Data/geoBoundaries folder.

Let’s merge the two Koreas into a single layer (Vector menu \(\to\) Data Management Tools \(\to\) Merge Vector Layers...)

Input layers \(=\) geoBoundaries-KOR-ADM0 and geoBoundaries-PRK-ADM0

Save the merged file as koreas.geojson

Change the symbology of the new layer, to make all but the borders transparent

Let’s extract the part of the global VNL raster that overlaps with the Koreas.
Go to Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...

Set parameters

  • Input layer \(=\) VNL_npp_2024...
  • Mask layer \(=\) koreas
  • Match the extent of the clipped raster to the extent of the mask layer
  • Save file as koreas_nl2024_mask.tif


The clipped raster should look something like this

You can zoom in to see Seoul in greater detail

Change color scheme to Singleband pseudocolor, Quantile again for contrast

General tip: set Interpolation \(=\) Discrete, Mode \(=\) Equal interval and manually edit the cutpoints like this

This way, you can customize the appearance of the map for your needs

This map is ready to be exported (you know how to do this)

Now for Vignette 2, let’s repeat this process for the island of Hispaniola.
Load country boundaries for Haiti and the Dominican Republic (2 files: geoBoundaries-HTI-ADM0.geojson and geoBoundaries-DOM-ADM0.geojson from Data/geoBoundaries).

Once loaded, let’s merge the two countries into a single layer again (Vector menu \(\to\) Data Management Tools \(\to\) Merge Vector Layers...)

Input layers \(=\) geoBoundaries-DOM-ADM0 and geoBoundaries-HTI-ADM0

Save the merged file as hispaniola.geojson

Let’s extract the part of the global VNL raster that overlaps with the island.
Go to Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...

Set Input layer \(=\) VNL_npp_2024..., Mask layer \(=\) hispaniola. Save file as hispaniola_nl2024_mask.tif

The clipped raster should look something like this

Customize the color ramp and export the map (just like last time)

Differences in Luminosity Over Time (Syrian Civil War)

Vignette 3! Load administrative boundaries for Syria. File is geoBoundaries-SYR-ADM2.geojson from Data/geoBoundaries folder.

You may see a “Select Items to Add” screen after clicking Add. Click Add Layers

To compare current luminosity to a period earlier in the Syrian Civil War, load the 2012 VNL data (VNL_v21_npp_201204....tif file in Data/VIIRS folder)

Let’s extract the parts of both the 2024 and 2012 global VNL rasters that overlap with Syria. Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...

Set Input layer \(=\) VNL_npp_2024..., Mask layer \(=\) geoBoundaries-SYR-ADM2. Save file as syria_nl2024_mask.tif

Repeat with Input layer \(=\) VNL_v21_npp_2012..., Mask layer \(=\) geoBoundaries-SYR-ADM2. Save file as syria_nl2012_mask.tif

Let’s now calculate the difference between 2024 and 2012. Go to Raster menu \(\to\) Raster Calculator...

Set the expression to "syria_nl2024_mask@1" - "syria_nl2012_mask@1", save output layer as syria_nldiff.tif

Explore the distribution of this new raster by modifying the color scheme
(here, purple areas lost luminosity, orange areas gained luminosity)

We can now calculate average differences in luminosity per district, using Zonal statistics (in the Processing Toolbox)

Set the Input layer \(=\) geoBoundaries-SYR-ADM2, Raster layer \(=\) syria_nldiff, Output column prefix \(=\) nldiff_. Save the output as syria_nl_1.geojson

Adjust the symbology in the new syria_nl_1 layer to visualize the nldiff_mean variable with graduated colors, Equal Interval mode and Symmetric Classification around 0.00

The district-level luminosity differences should look something like this.
Now let’s see if places hardest-hit by violence saw the biggest declines…

Add the Syrian Civil War violence data to the project, using Add Delimited Text Layer.... Load the GEDEvent_v25_1.csv file in Data/GED folder. Set X field \(=\) longitude and Y field \(=\) latitude. Check box \(\checkmark\) Use spatial index

Highlight GED layer and go to Edit \(\to\) Select \(\to\) Select by Expression...

Expression: year>=2012 AND country='Syria' AND where_prec<4
Click Select Features

This procedure should have selected about 74.5 thousand events within Syria.
Now let’s calculate the number of violent events per district

Open the Count Points in Polygon tool

Select Polygons \(=\) syria_nl_1, Points \(=\) GEDEvent_v25_1. Make sure the box is checked next to Selected Features Only for the points. Name the count field all_violence, and save the output file as syria_nl_2.geojson. Click Run

The new layer syria_nl_2 should appear in your project window.
Now, let’s see if there is a relationship between violence and change in luminosity

Let’s run a simple regression model in R to see how violence impacted luminosity.

 

This code chunk imports the syria_nl_2.geojson file we created into an object called syr, and then lists the variable names:

syr = sf::read_sf("Output/syria_nl_2.geojson")
names(syr)
## [1] "shapeName"    "shapeISO"     "shapeID"      "shapeGroup"   "shapeType"   
## [6] "nldiff_mean"  "all_violence" "geometry"

This code chunk estimates an Ordinary Least Squares model that regresses dependent variable nldiff_mean on explanatory variable all_violence

mod = lm(nldiff_mean~all_violence,data=syr)
summary(mod)
## 
## Call:
## lm(formula = nldiff_mean ~ all_violence, data = syr)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -18.0902  -0.1057   0.3550   0.8098   2.4571 
## 
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)  
## (Intercept)  -0.3274582  0.4213761  -0.777    0.440  
## all_violence -0.0004381  0.0002107  -2.079    0.042 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.61 on 59 degrees of freedom
## Multiple R-squared:  0.06824,    Adjusted R-squared:  0.05245 
## F-statistic: 4.321 on 1 and 59 DF,  p-value: 0.04199

Let’s rescale the variable all_violence to make the coefficient more interpretable.

mod = lm(nldiff_mean~I(all_violence/100),data=syr)
summary(mod)
## 
## Call:
## lm(formula = nldiff_mean ~ I(all_violence/100), data = syr)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -18.0902  -0.1057   0.3550   0.8098   2.4571 
## 
## Coefficients:
##                     Estimate Std. Error t value Pr(>|t|)  
## (Intercept)         -0.32746    0.42138  -0.777    0.440  
## I(all_violence/100) -0.04381    0.02107  -2.079    0.042 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 2.61 on 59 degrees of freedom
## Multiple R-squared:  0.06824,    Adjusted R-squared:  0.05245 
## F-statistic: 4.321 on 1 and 59 DF,  p-value: 0.04199

For every 100 violent events, the change in luminosity falls by 0.044.

You can perform all these steps in R
(see replication code wt02_demo.R in Lab10WT02.zip)

Vignette 1

Vignette 2

 

Vignette 3

… or in Python.
(see replication code wt02_demo.py in Lab10WT02.zip)

Vignette 1

Vignette 2

 

Vignette 3