US nighttime lights data, 1994 (DMSP, composite from 29 orbits)


US nighttime lights data, 1995 (DMSP, composite from 236 orbits)

Overview


Nearly all production and consumption after sundown requires (and emits) light

  1. Household appliances
  2. Office equipment
  3. Transportation
  4. Sporting events
  5. Street markets
  6. Restaurants
  7. Factory operations
  8. Roadworks
  9. Construction
  10. Studying and homework

 

Remote sensing of nighttime lights allows us to observe human activity from space


Emitting light

Applications of Luminosity Data


Economic activity, income, growth

  1. Persistent economic data challenges
    1. in most countries, GDP stats not available at the sub-national level
    2. national accounts & surveys are sometimes unreliable
    3. some locations too costly, dangerous for field & survey work
  2. Luminosity has some advantages
    1. more spatially fine-grained than standard income accounts
    2. regular, frequent time intervals
    3. recorded homogeneously across national borders
    4. available in hard-to-reach areas
  3. Not a bad proxy measure
    1. nighttime lights found to be positively correlated with GDP, GRP at multiple spatial scales


 

 

Luminosity and GDP/capita


 

 

Urbanization

  1. First applications of luminosity data:
    1. delineate urban extents
    2. track electrification, street lighting
  2. Luminosity captures things surveys, census can’t
    1. growth of informal, illegal settlements
    2. communities with inadequate infrastructure
    3. emergence of new local urban centers
    4. characterize urban growth over time


South Asia, 1994

South Asia, 2010


 

 

Emergency management, recovery

  1. Luminosity captures impact of emergencies
    1. disasters cause damage to power lines, disrupt electric utility services
    2. luminosity data can detect power outages, also fires, flares, other sudden increases in light
  2. But not everything is visible from space
    1. cloud cover prevents real-time storm damage tracking and assessment
    2. luminosity better suited for monitoring non-meteorological events (e.g. earthquakes, accidents), long-term tracking of recovery efforts


Before landfall

After landfall


 

Armed conflict

  1. War creates darkness
    1. direct: power grid damage
    2. indirect: population displacement, curfews, intentional dimming to avoid detection
  2. Danger lurks in darkness
    1. power disruptions constrain economic development, harm public health
    2. fall in light emissions is warning that civilians, hospitals lack stable electricity supply
  3. Data can illuminate the physical impact of war
    1. luminosity data allow us to track impact of war on public services, utilities, urban economic activity
    2. this analysis can be near-real time


October 2021

October 2022


 

 

Technological change

  1. Outdoor lighting technology evolves
    1. wood, dung \(\to\) oil
    2. oil \(\to\) gas
    3. gas \(\to\) incandescent
    4. incandescent \(\to\) LED, HID
  2. Ecological and social impacts
    1. each change increases emissions, changes spectrum of visible light
    2. spectral change affects perception of artificial lights by animals
    3. LEDs are more easily dimmed, lighting becomes more temporally dynamic
  3. This is also a measurement challenge
    1. change in spectrum can be mistaken for change in light intensity


Alles ist erleuchtet

Varieties of Luminosity Data


Space-based sensors for night-lights (partial list)

Sensor Spatial resolution Temporal resolution Availability Free? On-board calibration
DMSP/OLS 3km Monthly 1992-2013 \(\checkmark\)
Landsat 8 30m Irregular 2013- \(\checkmark\)
VIIRS/DNB 740m Daily 2012- \(\checkmark\) \(\checkmark\)
EROS-B <1m Daily 2013- \(\checkmark\)
Jilin-1 <1m Daily 2017- \(\checkmark\)

Defense Meteorological Satellite Program (DMSP)

  1. History
    1. 1960s: starts as DoD’s meteorological program
      • goal: prevent spy satellites from wasting resources when targets are cloud-covered
    2. 1992: images available in digital form
    3. 1994: NOAA launches global nightlights program
  2. Limitations
    1. data from different years not directly comparable due to different atmospheric conditions, sensor settings and degradation
    2. temporal analysis requires intercalibration
      • use invariant pixels as training data
      • calibrate raw images to same radiometric level as base image
    3. images blurred due to scattering, saturation

 

Link: eogdata.mines.edu/products/dmsp/


DMSP-5D2


Visible Infrared Imaging Radiometer Suite (VIIRS)

  1. History
    1. 2011: sensor launches onboard Suomi NPP satellite
    2. quickly becomes industry standard
    3. main product: Day and Night Band (DNB)
  2. Advantages over DMSP
    • on-board calibration enables temporal comparisons
    • higher spatial precision
    • sensitive to lower light levels
    • no saturation in urban areas
    • cloud-free, stray light corrections
    • free daily images \(+\) monthly, annual composites

 

Link: eogdata.mines.edu/products/vnl/


Suomi NPP


Not all orbits make good data

Requirements for nighttime light observation:

  1. Center half of orbital swath
    • (i.e. area being imaged by satellite)
  2. No sunlight
    • (no high-latitude zones in summer)
  3. No moonlight
    • (within-month variation)
  4. No solar glare
    • (e.g. from solar panels)
  5. Cloud-free
    • (sorry, Seattle. hello, Dubai)
  6. No auroral emissions
    • (why Santa has evaded detection)
  7. No forest fires
    • (no intense sources of natural light)


Swath path


Illustration of cloud cover contamination

Daily VIIRS data for Ukraine, 2021-2022

Vignettes


Overview of lab exercise

 

  1. Vignettes on differences in luminosity across borders
    1. Korean peninsula
    2. Island of Hispaniola
  2. Vignette on differences in luminosity over time
    1. Syrian Civil War

Vignette 1 / Korea


Vignette 2 / Hispaniola


Vignette 3 / Syria


We can obtain VIIRS nighttime luminosity (vnl) data from eogdata.mines.edu/products/vnl/


Scroll down to the “Annual VNL V2” section


Click on the “Go to Download V2.2” button for the most recent year’s data


Download the file ending with .average_masked.dat.tif.gz


Let’s also grab data for the “oldest” year available.
Navigate to the parent directory and find the annual data in the v21/ folder


There are two files ending in .average_masked.dat.tif.gz here.
Download the first one (April through December)


Let’s get some country and administrative boundaries from geoboundaries.org


Navigate to the “Individual Country Files” section


Download country-level (ADM0) data for South and North Korea (KOR, PRK)


When you unzip, the only file you need to extract is one ending in ADM0.geojson


Repeat this process for country-level (ADM0) data for Haiti (HTI)…


and country-level (ADM0) data for Dominican Republic (DOM)…


and district-level (ADM2) data for Syria (SYR)


We will be using the same event data on violence as in the last lab:
UCDP GED version 23.1, in csv format


Here is the full list of data sources and links:

Category Type Format Data source
Nighttime luminosity Raster .tif VIIRS
Administrative units Vector (polygons) .geojson geoBoundaries
Political violence Table (non-geo) .csv UCDP GED

 

These are all in the WT02.zip file posted on Canvas.

Differences in Luminosity Across Countries (Korea and Hispaniola)


Always save your progress!
Go to Project \(\to\) Save As...


Vignette 1. Load the 2023 VNL data (Layer \(\to\) Add Layer \(\to\) Add Raster Layer). VNL_npp_2023_....tif file in Data/VIIRS folder


The default color scheme is too dark. Let’s see if we can add some contrast


In the layer’s Properties, change Render type to Singleband pseudocolor and set Mode to Quantile. Click Classify and OK


This is probably too much contrast, but at least we can see the distribution


Load country boundaries for the two Koreas (Layer \(\to\) Add Layer \(\to\) Add Vector Layer). 2 files: geoBoundaries-PRK-ADM0.geojson and geoBoundaries-KOR-ADM0.geojson from Data/geoBoundaries folder.


Let’s merge the two Koreas into a single layer (Vector menu \(\to\) Data Management Tools \(\to\) Merge Vector Layers...)


Input layers \(=\) geoBoundaries-KOR-ADM0 and geoBoundaries-PRK-ADM0


Save the merged file as koreas.geojson


Change the symbology of the new layer, to make all but the borders transparent


Let’s extract the part of the global VNL raster that overlaps with the Koreas.
Go to Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...


Set parameters

  • Input layer \(=\) VNL_npp_2023...
  • Mask layer \(=\) koreas
  • Match the extent of the clipped raster to the extent of the mask layer
  • Save file as koreas_nl2023_mask.tif



The clipped raster should look something like this


You can zoom in to see Seoul in greater detail


Change color scheme to Singleband pseudocolor, Quantile again for contrast


General tip: set Interpolation \(=\) Discrete, Mode \(=\) Equal interval and manually edit the cutpoints like this


This way, you can customize the appearance of the map for your needs


This map is ready to be exported (you know how to do this)


Now for Vignette 2, let’s repeat this process for the island of Hispaniola.
Load country boundaries for Haiti and the Dominican Republic (2 files: geoBoundaries-HTI-ADM0.geojson and geoBoundaries-DOM-ADM0.geojson from Data/geoBoundaries).


Once loaded, let’s merge the two countries into a single layer again (Vector menu \(\to\) Data Management Tools \(\to\) Merge Vector Layers...)


Input layers \(=\) geoBoundaries-DOM-ADM0 and geoBoundaries-HTI-ADM0


Save the merged file as hispaniola.geojson


Let’s extract the part of the global VNL raster that overlaps with the island.
Go to Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...


Set Input layer \(=\) VNL_npp_2023..., Mask layer \(=\) hispaniola. Save file as hispaniola_nl2023_mask.tif


The clipped raster should look something like this


Customize the color ramp and export the map (just like last time)

Differences in Luminosity Over Time (Syrian Civil War)


Vignette 3! Load administrative boundaries for Syria. File is geoBoundaries-SYR-ADM2.geojson from Data/geoBoundaries folder.


You may see a “Select Items to Add” screen after clicking Add. Click Add Layers


To compare current luminosity to a period earlier in the Syrian Civil War, load the 2012 VNL data (VNL_v21_npp_201204....tif file in Data/VIIRS folder)


Let’s extract the parts of both the 2023 and 2012 global VNL rasters that overlap with Syria. Raster menu \(\to\) Extraction \(\to\) Clip Raster by Mask Layer...


Set Input layer \(=\) VNL_npp_2023..., Mask layer \(=\) geoBoundaries-SYR-ADM2. Save file as syria_nl2023_mask.tif


Repeat with Input layer \(=\) VNL_v21_npp_2012..., Mask layer \(=\) geoBoundaries-SYR-ADM2. Save file as syria_nl2012_mask.tif


Let’s now calculate the difference between 2023 and 2012. Go to Raster menu \(\to\) Raster Calculator...


Set the expression to "syria_nl2023_mask@1" - "syria_nl2012_mask@1", save output layer as syria_nldiff.tif


Explore the distribution of this new raster by modifying the color scheme
(here, purple areas lost luminosity, orange areas gained luminosity)


We can now calculate average differences in luminosity per district, using Zonal statistics (in the Processing Toolbox)


Set the Input layer \(=\) geoBoundaries-SYR-ADM2, Raster layer \(=\) syria_nldiff, Output column prefix \(=\) nldiff_. Save the output as syria_nl_1.geojson


Adjust the symbology in the new syria_nl_1 layer to visualize the nldiff_mean variable with graduated colors, Equal Interval mode and Symmetric Classification around 0.00


The district-level luminosity differences should look something like this.
Now let’s see if places hardest-hit by violence saw the biggest declines…


Add the Syrian Civil War violence data to the project, using Add Delimited Text Layer.... Load the GEDEvent_v23.csv file in Data/GED folder. Set X field \(=\) longitude and Y field \(=\) latitude. Check box \(\checkmark\) Use spatial index


Highlight GED layer and go to Edit \(\to\) Select \(\to\) Select by Expression...


Expression: year>=2012 AND country='Syria'. Click Select Features


This procedure should have selected about 85 thousand events within Syria.
Now let’s calculate the number of violent events per district


Open the Count Points in Polygon tool


Select Polygons \(=\) syria_nl_1, Points \(=\) GEDEvent_v23_1. Make sure the box is checked next to Selected Features Only for the points. Name the count field all_violence, and save the output file as syria_nl_2.geojson. Click Run


The new layer syria_nl_2 should appear in your project window.
Now, let’s see if there is a correlation between violence and change in luminosity


Open the Plotly tool (View menu \(\to\) Panels \(\to\) DataPlotly)


Set Plot type \(=\) Scatter Plot, Layer \(=\) syria_nl_2, X field \(=\) all_violence, Y field \(=\) nldiff_mean


In the “Layout Options” tab, uncheck the box next to Show legend and edit the title and axis labels. Change X axis mode to Logarithmic. Click Create Plot


From the scatterplot, we can see that the steepest drops in luminosity happened in some of the districts most heavily exposed to violence


If we click on some of the points in the lower-right corner, we can see some familiar places, like Damascus…


… or Aleppo (the administrative unit most heavily exposed to violence)


You can export the scatterplot to a png file, if you like


You can also perform all these steps in R
(see replication code wt02_demo.R in WT02.zip)

Vignette 1

Vignette 2

 

Vignette 3