How to use the parenx module scripts to simplify linear network features

Authors

Affiliations

Will Deakin

Digital, Data and Technology services, Network Rail, UK

Robin Lovelace

Leeds Institute for Transport Studies, University of Leeds, UK

Zhao Wang

Leeds Institute for Transport Studies, University of Leeds, UK

Josiah Parry

Environmental Systems Research Institute, Redlands, CA, USA

Reproducibility

To reproduce this paper you need quarto installed.

After installing the dependencies, you can reproduce the paper by running the following command in the terminal:

quarto render cookbook.qmd

1 Introduction

This cookbook is an addendum of the network merge paper¹, and consists of a number of examples based on real datasets used in the development and testing. As part of this for the paper the standalone parenx module² was developed and deployed to pypy. However, noting this is beta software, this module is neither well documentmed nor intuitive to use. So the hope is that by sharing these examples will make using these scripts a bit less painful.

This was developed and tested on a Debian Linux variant³ and run on the command-line interface with various versions of python3. While not tested on other environments they should work on under environments and operating systems.

2 Set up and run

In this example create a working project directory, say elegant-tern, and set a virtual-environment and simplify data there. This assumes you have bash and a working base python3 installation.

2.1 Create a working directory

Open a shell command line prompt and type:

mkdir elegant-tern
cd elegant-tern

2.2 Create and activate python3 virtual-environment

Create a python virtual environment under venv in the working directory:

python3 -m venv venv
source venv/bin/activate

Activation then means that the scripts and modules installed in the virtual environment are added to the shell execution path.

2.3 Upgrade the base pip and wheel modules (optional)

This is to make sure you are working with the lastest version of the python3 package management and installation tools:

pip install --upgrade pip
pip install --upgrade wheel

This becomes more important if you are maintaining a pypi project.

2.4 Install the parenx project:

This installs the latest release of the parenx scripts from pypi and prints the module version to stdout:

pip install parenx
python3 -c "import parenx; from importlib.metadata import version; print(version('parenx'))"
0.5.6

2.5 Simplify a linear-network

To simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

skeletonize.py https://github.com/anisotropi4/parenx/blob/main/data/rnet_doncaster_rail.geojson?raw=true sk-doncaster.gpkg --buffer 30

Let’s look at the before and after network:

2.6 Notes

While there are many different takes and system to manage python packages, my experience is that package management in a virtual environment with python3 just works.

3 Simplification: TL;DR

The remaining “Too Long; Do not Read” sections explains more about how the parenx scripts work, and introduces a helper function used in testing. The skeletonize.py and voronoi.py simplication scripts simplify linear geometry with example road and rail data for Leeds, Edinburgh and Doncaster used during development is available⁴. Both allow the maximum displacement (simplify)⁵ and buffer size (buffer) to be set.

3.1 Skeletonize simplification

As the script take both a filepath or URL, to simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

skeletonize.py https://github.com/anisotropi4/parenx/blob/main/data/rnet_doncaster_rail.geojson?raw=true sk-doncaster.gpkg

The base skeletonize.py also takes parameters to split buffered line segments or preserve knots. These parameters:

skeletonize.py
start       0:00:00.000266
usage: skeletonize.py [-h] [--simplify SIMPLIFY] [--buffer BUFFER] [--scale SCALE] [--knot] [--segment] inpath [outpath]
skeletonize.py: error: the following arguments are required: inpath

3.2 Voronoi simplification

As the script take a filepath or URL, simplify the rail centre-line track-model near Doncaster to the GeoPKG file sk-doncaster.gpkg:

skeletonize.py https://github.com/anisotropi4/parenx/blob/main/data/rnet_doncaster_rail.geojson?raw=true sk-doncaster.gpkg

The base voronoi.py also takes a tolerance parameter⁶ to sets to the Voronoi polygon overlap:

start       0:00:00.000154
usage: voronoi.py [-h] [--simplify SIMPLIFY] [--scale SCALE] [--buffer BUFFER] [--tolerance TOLERANCE] inpath [outpath]
voronoi.py: error: the following arguments are required: inpath

4 Helper scripts

The run.sh helper script sets and installs parenx in a virtual environment, and runs the skeletonize and voronoi simplification scripts with a number of different simplification parameters, and converts the output into a sanitized GeoJSON format if ogr2ogr is installed. A second parenx helper script has also been developed which allows the simplification algorithm to be passed as a command line parameter.

4.1 Find the run.sh helper script

To find it:

find . -name run.sh
./venv/lib/python3.12/site-packages/parenx/run.sh

4.2 Copy the run.sh helper script

Copy the helper script to the working directory. If you want to check for its existence copy it to the working directory type:

ls run.sh
ls: cannot access 'run.sh': No such file or directory
find . -name run.sh -exec cp {} . \;
ls run.sh
run.sh

Otherwise the following will suffice:

find . -name run.sh -exec cp {} . \;

4.3 Run the run.sh helper script

The default runs the skeletonize and voronoi simplification scripts with a number of parameters, against an OpenStreetMap⁷ Edinburgh Princes Street road network file (net_princes_street.geojson). It creates a GeoPKG⁸ with three output layers and, if ogr2ogr is installed, GeoJSON⁹.

./run.sh
simplify ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
skeletonize ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
start       0:00:00.000270
read geojson    0:00:00.093758
...
write simple    0:00:03.403231
write primal    0:00:03.446779
stop        0:00:03.498828
voronoi ./venv/lib/python3.12/site-packages/parenx/data/rnet_princes_street.geojson
start       0:00:00.000165
read geojson    0:00:00.079670
...
stop        0:00:19.703512

4.4 What does the helper run.sh do?

The helper script creates the enviroment and runs the skeletonize and voronoi simplification scripts with different simplification parameters, and converts the output into a sanitized GeoJSON format if ogr2ogr is installed. The path and output filename can also be specified.

4.4.1 Set up environment

As above this checks to see if a venv directory exists, creates and populates it if not, and activates the environment:

#!/usr/bin/env bash

if [ ! -d venv ]; then
    python3 -m venv venv
    source venv/bin/activate
    pip install --upgrade pip
    pip install --upgrade wheel
    pip install parenx
fi

source venv/bin/activate

It also see if there are any command line parameters set and creates an archive directory, if it is absent:

LIBPATH=$(find . -name data | fgrep parenx)
INPATH=${1:-"${LIBPATH}/rnet_princes_street.geojson"}
OUTPUT=${2:-"output"}

echo simplify ${INPATH}

if [ ! -d archive ]; then
    mkdir archive
fi

This sets the LIBPATH shell variable to the location of parenx library.

It also sets the INPATH and OUTPATH shell variables to command line values, or defaults.

4.4.2 Archive previous files

Create an archive directory, and archive any existing output files.

if [ ! -d archive ]; then
    mkdir archive
fi

for k in sk vr
do
    if [ -s ${k}-${OUTPUT}.gpkg ]; then
        mv ${k}-${OUTPUT}.gpkg archive
    fi
    if [ -s ${k}-${OUTPUT}.geojson ]; then
        mv ${k}-${OUTPUT}.geojson archive
    fi
done

4.4.3 Simplify using skeletonization

This creates three skeletonization outputs with varying simplify and segment parameters:

echo skeletonize ${INPATH}
skeletonize.py ${INPATH} sk-${OUTPUT}.gpkg
skeletonize.py ${INPATH} sk-${OUTPUT}-simple.gpkg --simplify 1.0
skeletonize.py ${INPATH} sk-${OUTPUT}-segment.gpkg --segment

4.4.4 Simplify using Voronoi

This creates two Voronoi outputs with varying simplify parameters:

echo voronoi ${INPATH}
voronoi.py ${INPATH} vr-${OUTPUT}.gpkg
voronoi.py ${INPATH} vr-${OUTPUT}-simple.gpkg --simplify 1.0

4.4.5 What does the full run.sh script look like?

Remembering less is more:

less run.sh
#!/usr/bin/env bash

if [ ! -d venv ]; then
    python3 -m venv venv
    source venv/bin/activate
    pip install --upgrade pip
    pip install --upgrade wheel
    pip install parenx
fi

source venv/bin/activate

LIBPATH=$(find . -name data | fgrep parenx | head -1)
INPATH=${1:-"${LIBPATH}/rnet_princes_street.geojson"}
OUTPUT=${2:-"output"}

echo simplify ${INPATH}

if [ ! -d archive ]; then
    mkdir archive
fi

for k in sk vr
do
    if [ -s ${k}-${OUTPUT}.gpkg ]; then
        mv ${k}-${OUTPUT}.gpkg archive
    fi
    if [ -s ${k}-${OUTPUT}.geojson ]; then
        mv ${k}-${OUTPUT}.geojson archive
    fi
done

echo skeletonize ${INPATH}
skeletonize.py ${INPATH} sk-${OUTPUT}.gpkg
skeletonize.py ${INPATH} sk-${OUTPUT}-simple.gpkg --simplify 1.0
skeletonize.py ${INPATH} sk-${OUTPUT}-segment.gpkg --segment
echo voronoi ${INPATH}
voronoi.py ${INPATH} vr-${OUTPUT}.gpkg
voronoi.py ${INPATH} vr-${OUTPUT}-simple.gpkg --simplify 1.0

OGR2OGR=$(which ogr2ogr)
if [ x"${OGR2OGR}" != x ]; then
    for k in sk vr
    do
        rm -f ${k}-${OUTPUT}.geojson
        ogr2ogr -f GeoJSON ${k}-${OUTPUT}.geojson ${k}-${OUTPUT}.gpkg line
        sed -i 's/00000[0-9]*//g' ${k}-${OUTPUT}.geojson
    done
    for k in sk vr
    do
        rm -f ${k}-${OUTPUT}-simple.geojson
        ogr2ogr -f GeoJSON ${k}-${OUTPUT}-simple.geojson ${k}-${OUTPUT}-simple.gpkg line
        sed -i 's/00000[0-9]*//g' ${k}-${OUTPUT}-simple.geojson
    done
fi

4.5 Copy the parenx helper script

Copy the parenx helper script to the working directory and make an output directory

find . -name parenx -type f -exec cp {} . \;
mkdir output

4.6 Run the parenx helper script

The following runs the skeletonize and voronoi simplification scripts where the algorithm is selected on the command line. It assumes that the environment has been set with a data directory containing the rnet_princes_street.geojson. The simplified network is then created under the output directory as specified on the command line.

./parenx skeletonize ./data/rnet_princes_street.geojson output/rnet_princes_street_skeltonize.gpkg
./parenx voronoi ./data/rnet_princes_street.geojson output/rnet_princes_street_voronoi.gpkg

All additional parameters supported by the python scripts are also supported. For example, the following will pass the value 4.0 to skeletonize script and retain knots

./parenx skeletonize data/rnet_princes_street.geojson output/rnet_princes_street_sk.gpkg --scale 4 --knot`


## Application Programming Interface (API) Example

The `skeletonize_frame`, `voronoi_frame`, `primal_frame` and `tile_skeletonize_frame` functions are exposed via a simple API.

```python
#!/usr/bin/env python3

import geopandas as gp
from parenx import skeletonize_frame, voronoi_frame, skeletonize_tiles, get_primal

CRS = "EPSG:27700"
filepath = "data/rnet_princes_street.geojson"
frame = gp.read_file(filepath).to_crs(CRS)

parameter = {"simplify": 0.0, "buffer": 8.0, "scale": 1.0, "knot": False, "segment": False}
r = skeletonize_frame(frame["geometry"], parameter)

parameter = {"simplify": 0.0, "scale": 5.0, "buffer": 8.0, "tolerance": 1.0}
r = voronoi_frame(frame["geometry"], parameter)

primal = get_primal(r)

5 Acknowledgement

Many thanks to everyone who has helped and supported in making this work possible, and making the data available.

• The centre-line track-model is an extract of the Network Rail centre-line track-model hosted by OpenRailData under the Open Government License, retrieved 2023-07-11.
• The road network data are static extracts from OpenStreeMap under CC-BY-SA 2.0 through the OverPassAPI Turbo service.

Footnotes

1. The network merge paper here.

2. See the online project.

3. The Debian Linux distribution

4. GitHub data used in development.

5. The simplify parameter sets shapely simplify function tolerance value.

6. The tolerance parameter set shapely voronoi_diagram function Voronoi snapping tolerance.

7. The OpenStreetMap map of Edinburgh.

8. The OSC GeoPKG specification.

9. The GeoJSON Specification (RFC 7946).