Objectives

Our immediate goal is to pre-process (i.e. clean up) declassified WWI-era maps of trenches in Northern France and Belgium, preparing them for statistical analysis. The down-stream analytical goal is to better understand the long-term legacies of warfare for local communities. Proximity to WWI trench warfare had profound implications for local political dynamics, economic recovery, and ecological health. Communities near the front lines endured significant physical devastation but also became focal points for long-term political and cultural transformations. The declassified military maps in this collection will allow us to precisely measure each locality’s exposure and proximity to sustained trench warfare in WWI, and to estimate its long-term effects on voting, economic activity, agricultural productivity, and other outcomes in subsequent decades.

Example of WWI trench data, Aubigny

These are GIS data in raster format (.tif). Rasters are grid-based structures of equally sized cells (pixels), each representing a geographic location with a value for continuous or categorical attributes. In our case, the cells represent 4 \(\times\) 4 meter plots of land (resolution of 4m), and their values represent the presence or absence of trenches or other fortifications in WWI. The cells take a non-zero value if trenches were present (usually a 2 or 3, depending on whether the positions belong to allies or Germans), and are empty (NA, missing value) elsewhere. The above image shows an example of what these data look like.

The main problem we need to address right now is that some of the raster data are contaminated with “false positives”: raster values that falsely indicate the presence of trenches where there were none. The most common of these are street names, hill numbers, and other annotations that a geoprocessing algorithm might mistake for trenchlines, likely because these features were drawn in a similar color on the original map. If these errors remain, we will systematically over-estimate exposure to trenches in some locations, potentially biasing statistical analyses. The image below shows an example of these false positives: the black cells include street names and other info, not just the physical location of trenches.

Example of WWI trench data with “false positives” in need of removal, Amiens

Your goal is to remove as many of these “false positives” as possible. The instructions below tell you how to do this in QGIS. QGIS is a free, open-source, cross-platform geographic information system (GIS) software for creating, editing, visualizing, analyzing, and publishing geospatial data. Get it at qgis.org.

Instructions for Manually Editing Raster Cell Values in QGIS Using the Serval Plugin

0. Create a QGIS Project File to Back Up Your Work

  • Open QGIS.
  • Before starting, create a QGIS project file to save your workspace:
    • Go to Project > Save As in the main menu.
    • Choose a location and name for your project file, ensuring the file is saved as a .qgz (default) or .qgs format.
    • This project file will save your layer configurations, symbology, and other settings, allowing you to restore your workspace later if needed.
  • Save your work periodically by clicking Project > Save or using the keyboard shortcut:
    • Windows/Linux: Ctrl+S
    • macOS: Cmd+S

1. Install the Serval Plugin

  • Open QGIS and navigate to Plugins > Manage and Install Plugins.
  • Search for “Serval” in the plugin repository and install it.
  • Restart QGIS if required to activate the plugin.

2. Activate the Serval Toolbar

  • Go to Plugins > Serval > Show Serval Plugin to enable the toolbar.
  • The Serval toolbar will appear as a floating panel in your QGIS interface. You can dock it by dragging it to your preferred location, such as below the main toolbar or near the Layers Panel.

3. Load a Raster File into QGIS

  • In QGIS, go to Layer > Add Layer > Add Raster Layer or click the Add Raster Layer button (a small icon resembling a grid with a plus sign) on the Data Source Manager toolbar.
  • In the dialog box that appears, click the three dots (...) next to the “Source” field, navigate to your raster file (e.g., .tif), and select it.
    • The raw data tiles are in the Dropbox folder Dropbox/YZRA/Data/WWI/Raw. They are all named something like reres_PARIS.tif
    • The default path to the Dropbox folder varies by operating system: ~/Dropbox (Mac and Linux), C:\Users\YourUsername\Dropbox (Windows).
  • Click Add to load the raster into your project. The raster will now appear in the Layers Panel and on the map canvas.

4. Create a Working Copy of Your Raster

  • To preserve your original raster file, create a working copy that you will edit:
    • Right-click on your raster layer in the Layers Panel and select Export > Save As.
    • In the export dialog:
      1. Click the three dots (...) next to the “File name” field to open a file browser. This allows you to specify the exact path and file name for your working copy.
      2. Navigate to the Working directory in the WWI folder (Dropbox/YZRA/Data/WWI/Working) and enter a descriptive name for your working copy (e.g., copy_PARIS.tif).
      3. Avoid saving to a temporary file location (e.g., /tmp/ or similar paths), as these files may be deleted when QGIS is closed.
      4. Ensure that you select an appropriate format (i.e., GeoTIFF, .tif file extension) and maintain consistent projection settings with your original raster.
    • Click OK to save your working copy.

5. Set the Render Type to Single Color

  • Right-click on the loaded working copy raster layer in the Layers Panel and select Properties.
  • In the Layer Properties dialog, go to the Symbology tab.
  • Under “Render Type,” select Single Color from the dropdown menu.
    • This option renders all raster cells with a uniform color, making it easier to visually identify individual cells during editing.
  • Adjust any additional settings (e.g., color or transparency), if needed.
  • Click OK to apply changes and close the dialog.

6. Activate and Select Your Raster Layer

  • Before using any tools in Serval, ensure that your working copy raster layer is selected in the Layers Panel by clicking on it.
  • Only one raster layer can be edited at a time using Serval. If you have multiple rasters loaded, confirm that you have activated (highlighted) the correct one.

7. Select Raster Cells Using Polygon Selection Tool

  • Locate and activate the Polygon Selection Tool on the Serval toolbar. The button is represented by an icon resembling a polygon shape or outline.

Polygon Selection Tool

  • Once activated, use this tool to draw a polygon over your raster by clicking points on your map canvas with:
    • Windows/Linux/macOS: Left mouse button (LMB) for each vertex of your polygon.

Drawing Polygon

  • Complete drawing by right-clicking (RMB) after placing all vertices of your polygon. This will finalize your selection of underlying raster cells.

8. Apply NoData Values to Selection

  • After completing your selection, click on the Apply NoData to Selection button on the Serval toolbar (represented by an icon with a dashed rectangle or similar symbol).

Apply NoData Button

  • All selected cells within your polygon will be updated with NoData values, masking them from further analysis.

NoData Applied

9. Undo/Redo Changes

  • Standard keyboard shortcuts like Ctrl+Z (Windows/Linux) or Cmd+Z (macOS) and Edit > Undo are not available while working within Serval.
  • Instead, use Serval’s own Undo and Redo buttons:
    • The Undo button is represented by a leftward curved arrow; Redo is represented by a rightward curved arrow.
  • The number of undo/redo steps available is governed by Serval settings:
    1. Open Serval’s settings by clicking its gear icon in the toolbar.
    2. Adjust “Number of Undo Steps” as needed.

10. Save Changes to Your Working Copy

  • Save your working copy regularly, including making backup copies as you go.
  • Once editing is complete, and you are ready to commit all changes, export your edited raster as a new file:
    • To export: Right-click on your edited raster layer and select Export > Save As.
    • Choose a filename (e.g. clean_PARIS.tif) and location (use the Processed folder in YZRA/Data/WWI/Processed to store all completed files).

Additional Tips

Tip 0: Keep Everything Updated

  1. Keep track of your and the team’s progress with the spreadsheet raster_tracker.csv in the YZRA/Data/WWI folder. When you begin working on a file (and have created a working copy), change the value in the working column for that file from N to Y. Save and close. Then do the same for the processing column when you’re done.
  2. Also keep things up to date on our Trello board. Move the card for each file from To Do to Doing and Done as you go.

Tip 1: Do the Easier Maps First

  1. You find that some map tiles are already pretty “clean” (like the Aubigny example above). A lucky few maps are already 100% clean. Some look like they will be more of a struggle (like the Amiens example above).
  2. You should try to find all of the “low-hanging fruit” (cleaner maps) first. Every now and then, you’ll be able to put a map in the Processed folder without having to do any editing at all. But before you do that, be sure to closely inspect every inch of the map to make sure it’s really clean!

Tip 2: Zooming In and Out While Selecting

  1. Use mouse wheel:
    • Roll forward/backward for zooming in/out.
    • On macOS, hold down Cmd while scrolling if needed.
  2. Use keyboard shortcuts:
    • Press Ctrl + + (Windows/Linux) or Cmd + + (macOS) for zooming in.
    • Press Ctrl + – (Windows/Linux) or Cmd + – (macOS) for zooming out.
  3. Temporarily pan across large rasters by holding down spacebar while dragging with your mouse.

Tip 3: Preventing QGIS from Crashing

  1. Save frequently (Ctrl+S/Cmd+S) after major changes.
  2. Disable unnecessary plugins via Plugins > Manage Plugins.
  3. Monitor memory usage; close other applications if needed.
  4. Use Long Term Release (LTR) versions of QGIS for stability.
  5. Keep data layers minimal:
    • Remove unused layers via right-click > Remove Layer.
    • Turn off visibility for inactive layers via Layers Panel checkboxes.
  6. If crashes persist, create new profiles via Settings > User Profiles > New Profile.