How to use the parenx module scripts to simplify linear network features

Authors

Affiliations

Will Deakin

Digital, Data and Technology services, Network Rail, UK

Robin Lovelace

Leeds Institute for Transport Studies, University of Leeds, UK

Zhao Wang

Leeds Institute for Transport Studies, University of Leeds, UK

Josiah Parry

Environmental Systems Research Institute, Redlands, CA, USA

Reproducibility

To reproduce this paper you need quarto installed.

After installing the dependencies, you can reproduce the paper by running the following command in the terminal:

quarto render cookbook.qmd

1 Introduction

This cookbook is an addendum to the network merge paper¹, and consists of a number of examples based on real datasets used in the development and testing. As part of this for the paper the standalone parenx module² was developed and deployed to pypy. However, noting this is beta software, the hope is that by sharing these examples will make using these scripts a bit less painful.

This was developed and tested on a Debian Linux variant³ and run on the command-line interface with various versions of python3. While not tested on other environments they should work on under environments and operating systems.

2 Set up and run

In this example create a working project directory, say elegant-tern, and set a virtual-environment and simplify data there. This assumes you have bash and a working base python3 installation.

2.1 Create a working directory

Open a shell command line prompt and type:

mkdir elegant-tern
cd elegant-tern

2.2 Create and activate python3 virtual-environment

Create a python virtual environment under venv in the working directory:

python3 -m venv venv
source venv/bin/activate

Activation then means that the scripts and modules installed in the virtual environment are added to the shell execution path.

2.3 Upgrade the base pip and wheel modules (optional)

This is to make sure you are working with the lastest version of the python3 package management and installation tools:

pip install --upgrade pip
pip install --upgrade wheel

This becomes more important if you are maintaining a pypi project.

2.4 Install the parenx project:

This installs the latest release of the parenx scripts from pypi and prints the module version to stdout:

pip install parenx
python3 -c "import parenx; \
            from importlib.metadata import version; \
            print(version('parenx'))"
0.5.6

2.5 Simplify a linear-network

To simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

URL=https://github.com/anisotropi4/parenx/blob/main/data
parenx skeletonize ${URL}/rnet_doncaster_rail.geojson?raw=true \
    sk-doncaster.gpkg --buffer 30

Let’s look at the before and after network:

2.6 Notes

While there are many different ways to manage packages, our experiences suggest that package management in a virtual environment with python3 works well. Another recommended option is to use a Docker container such as the python image from the Geocompx project, which can be downloaded with the command docker pull ghcr.io/geocompx/python (or pythonr for a variant with R packages needed to reproduce all the images in the paper installed) or using the ‘Devcontainer’ in the networkmerge repo for full reproducibility.

3 Simplification: TL;DR

The remaining “Too Long; Do not Read” sections explains more about how the parenx scripts work, and introduces a helper function used in testing. The parenx wrapper to the skeletonize.py and voronoi.py simplification scripts simplify linear geometry with example road and rail data for Leeds, Edinburgh and Doncaster used during development is available⁴. Both allow the maximum displacement (simplify)⁵ and buffer size (buffer) to be set.

3.1 Skeletonize simplification

As the script take both a filepath or URL, to simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

URL="https://github.com/anisotropi4/parenx/blob/main/data"
parenx skeletonize ${URL}/rnet_doncaster_rail.geojson?raw=true sk-doncaster.gpkg

The base skeletonize.py also takes parameters to split buffered line segments or preserve knots. These parameters:

skeletonize.py
start       0:00:00.000266
usage: skeletonize.py [-h] [--simplify SIMPLIFY] [--buffer BUFFER] [--scale SCALE]
  [--knot] [--segment] inpath [outpath]
skeletonize.py: error: the following arguments are required: inpath

3.2 Voronoi simplification

As the script take a filepath or URL, simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

URL=https://github.com/anisotropi4/parenx/blob/main/data
voronoi.py ${URL}/rnet_doncaster_rail.geojson?raw=true sk-doncaster.gpkg

The base voronoi.py also takes a tolerance parameter⁶ to sets to the Voronoi polygon overlap:

start       0:00:00.000154
usage: voronoi.py [-h] [--simplify SIMPLIFY] [--scale SCALE] [--buffer BUFFER]
  [--tolerance TOLERANCE] inpath [outpath]
voronoi.py: error: the following arguments are required: inpath

4 Helper script

The run.sh helper script sets and installs parenx in a virtual environment, and runs the skeletonize and voronoi simplification scripts with a number of different simplification parameters, and converts the output into a sanitized GeoJSON format if ogr2ogr is installed.

4.1 Find the helper script

To find it:

find . -name run.sh -o parenx
./venv/lib/python3.12/site-packages/parenx/run.sh

4.2 Copy the helper scripts

Copy the helper script to the working directory. If you want to check for its existence copy it to the working directory type:

ls run.sh
ls: cannot access 'run.sh': No such file or directory
find . -name run.sh -o -name parenx -type f \
    -exec cp {} . \;
ls run.sh
run.sh

Otherwise the following will suffice:

find . -name run.sh -exec cp {} . \;

4.3 Run the helper script

The default runs the skeletonize and voronoi simplification scripts with a number of parameters, against an OpenStreetMap⁷ Edinburgh Princes Street road network file (net_princes_street.geojson). It creates a GeoPKG⁸ with three output layers and, if ogr2ogr is installed, GeoJSON⁹.

./run.sh
simplify ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
skeletonize ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
start       0:00:00.000270
read geojson    0:00:00.093758
...
write simple    0:00:03.403231
write primal    0:00:03.446779
stop        0:00:03.498828
voronoi ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
start       0:00:00.000165
read geojson    0:00:00.079670
...
stop        0:00:19.703512

4.4 What does the helper run.sh do?

The helper script creates the environment and runs the skeletonize and voronoi simplification scripts with different simplification parameters, and converts the output into a sanitized GeoJSON format if ogr2ogr is installed. The path and output filename can also be specified.

4.4.1 Set up environment

As above this checks to see if a venv directory exists, creates and populates it if not, and activates the environment:

#!/usr/bin/env bash

if [ ! -d venv ]; then
    python3 -m venv venv
    source venv/bin/activate
    pip install --upgrade pip
    pip install --upgrade wheel
    pip install parenx
fi

source venv/bin/activate

It also see if there are any command line parameters set and creates an archive directory, if it is absent:

LIBPATH=$(find . -name data | fgrep parenx)
INPATH=${1:-"${LIBPATH}/rnet_princes_street.geojson"}
OUTPUT=${2:-"output"}

echo simplify ${INPATH}

if [ ! -d archive ]; then
    mkdir archive
fi

This sets the LIBPATH shell variable to the location of parenx library.

It also sets the INPATH and OUTPATH shell variables to command line values, or defaults.

4.4.2 Archive previous files

Create an archive directory, and archive any existing output files.

if [ ! -d archive ]; then
    mkdir archive
fi

for k in sk vr
do
    if [ -s ${k}-${OUTPUT}.gpkg ]; then
        mv ${k}-${OUTPUT}.gpkg archive
    fi
    if [ -s ${k}-${OUTPUT}.geojson ]; then
        mv ${k}-${OUTPUT}.geojson archive
    fi
done

4.4.3 Simplify using skeletonization

This creates three skeletonization outputs with varying simplify and segment parameters:

echo skeletonize ${INPATH}
skeletonize.py ${INPATH} sk-${OUTPUT}.gpkg
skeletonize.py ${INPATH} sk-${OUTPUT}-simple.gpkg --simplify 1.0
skeletonize.py ${INPATH} sk-${OUTPUT}-segment.gpkg --segment

4.4.4 Simplify using Voronoi

This creates two Voronoi outputs with varying simplify parameters:

echo voronoi ${INPATH}
voronoi.py ${INPATH} vr-${OUTPUT}.gpkg
voronoi.py ${INPATH} vr-${OUTPUT}-simple.gpkg --simplify 1.0

4.4.5 What does this all look like?

Remembering less is more:

less run.sh
#!/usr/bin/env bash

if [ ! -d venv ]; then
    python3 -m venv venv
    source venv/bin/activate
    pip install --upgrade pip
    pip install --upgrade wheel
    pip install parenx
fi

source venv/bin/activate

LIBPATH=$(find . -name data | fgrep parenx | head -1)
INPATH=${1:-"${LIBPATH}/rnet_princes_street.geojson"}
OUTPUT=${2:-"output"}

echo simplify ${INPATH}

if [ ! -d archive ]; then
    mkdir archive
fi

for k in sk vr
do
    if [ -s ${k}-${OUTPUT}.gpkg ]; then
        mv ${k}-${OUTPUT}.gpkg archive
    fi
    if [ -s ${k}-${OUTPUT}.geojson ]; then
        mv ${k}-${OUTPUT}.geojson archive
    fi
done

echo skeletonize ${INPATH}
skeletonize.py ${INPATH} sk-${OUTPUT}.gpkg
skeletonize.py ${INPATH} sk-${OUTPUT}-simple.gpkg --simplify 1.0
skeletonize.py ${INPATH} sk-${OUTPUT}-segment.gpkg --segment
echo voronoi ${INPATH}
voronoi.py ${INPATH} vr-${OUTPUT}.gpkg
voronoi.py ${INPATH} vr-${OUTPUT}-simple.gpkg --simplify 1.0

OGR2OGR=$(which ogr2ogr)
if [ x"${OGR2OGR}" != x ]; then
    for k in sk vr
    do
        rm -f ${k}-${OUTPUT}.geojson
        ogr2ogr -f GeoJSON ${k}-${OUTPUT}.geojson ${k}-${OUTPUT}.gpkg \
           line
        sed -i 's/00000[0-9]*//g' ${k}-${OUTPUT}.geojson
    done
    for k in sk vr
    do
        rm -f ${k}-${OUTPUT}-simple.geojson
        ogr2ogr -f GeoJSON ${k}-${OUTPUT}-simple.geojson \
            ${k}-${OUTPUT}-simple.gpkg line
        sed -i 's/00000[0-9]*//g' ${k}-${OUTPUT}-simple.geojson
    done
fi

5 Acknowledgement

Many thanks to everyone who has helped and supported in making this work possible, and making the data available.

• The centre-line track-model is an extract of the Network Rail centre-line track-model hosted by OpenRailData under the Open Government License, retrieved 2023-07-11.
• The road network data are static extracts from OpenStreetMap under CC-BY-SA 2.0 through the OverPassAPI Turbo service.

Footnotes

1. The network merge paper here.

2. See the online project.

3. The Debian Linux distribution

4. GitHub data used in development.

5. The simplify parameter sets shapely simplify function tolerance value.

6. The tolerance parameter set shapely voronoi_diagram function Voronoi snapping tolerance.

7. The OpenStreetMap map of Edinburgh.

8. The OSC GeoPKG specification.

9. The GeoJSON Specification (RFC 7946).