POOPy = Pollution discharge monitoring with Object Oriented Python

• Description
• Installation
  • Dependencies
  • Testing
• Usage
  • Examples
  • Gallery
• Contributing
• Credit
• Disclaimer

This is a Python package for interfacing with live data from Event Duration Monitoring (EDM) devices maintained by English and Welsh Water Companies. This package was ostensibly developed to provide the back-end for SewageMap.co.uk but may be generically useful for those exploring the impact of sewage discharges on rivers. More information can be found in our preprint.

Currently, POOPy supports data from all of the major water and sewerage companies:

Different water companies share their live EDM data via APIs with different formats. This is obviously confusing and means that it is hard to access national data simultaneously and ultimately understand their potential impact on the environment. POOPy solves this problem by encapsulating relevant information about EDM monitors maintained by different water companies into a standardised interface. This interface (represented by the WaterCompany and Monitor classes) makes it very easy to, for instance, quickly identify monitors that are discharging, have discharged in the last 48 hours or are offline. POOPy combines this information with key meta-data about the monitor such as location and the watercourse it discharges into. Additionally, POOPy provides a basic approache to explore the 'impact' of discharges on the environment, using a simple hydrological model to identify river sections downstream of sewage discharges in real-time. POOPy could easily be extended to consider more complicated ways of exploring the 'impact' of sewage spills (e.g., dynamic river flow).

Where historical information on CSO discharges are available (currently only provided as an API by Thames Water), POOPy processes this information making it very easy to query the spill history of a particular monitor. For instance, to calculate the total hours of sewage discharge from a given monitor over a given timeframe. Experimentally, POOPy also has capabilities to 'build' histories of sewage spills from repeated queries to the current status of a monitor, even if (in the case of most water companies) this information is not made readily accessible.

POOPy is designed to be installed using the conda package manager. The following instructions were designed for UNIX systems (Linux, mac) but should also work on a Windows operating system by using Anaconda prompt.

First, download the code-base from this repository to a sensible location on your computer. This is easiest done using the following git command but you can also download the repository manually if you prefer.

git clone https://github.com/AlexLipp/POOPy.git [LOCAL DIRECTORY]

Navigate into the directory which contains the source code (it should contain a setup.py file). Next, we will create a conda environment with the required dependencies installed using the provided .yaml file. Run the following code from your base conda environment.

conda env create -f environment.yaml

This creates an environment called poopy that automatically installs POOPy's dependencies. Now, activate that environment:

conda activate poopy

Finally, we can actually install POOPy by running:

pip install .

If install has happened correctly you should see something like Successfully built poopy printed to your terminal. If you'd like to install an editable version of the package simply run: pip install -e . instead.

A test script is provided in the tests folder. To run the tests, you will need to install the pytest package. If installed, the tests can be run from the command line by navigating to the folder in which the package is installed and simply running the command:

pytest --disable-warnings

This will run the tests and provide a summary of the results. If all tests pass, the package has been installed correctly and behaving as expected. Note that the --disable-warnings flag is used to suppress the many warnings that POOPy generates, these are mostly informative rather than disastrous (e.g., indicating when an input data-stream is ambiguous), but can be overwhelming en masse.

Once installed, the package can be imported into Python scripts using standard import commands, such as:

import poopy

or

from poopy.companies import ThamesWater

Examples of how to use the package (using the ThamesWater class as an example) are given in the examples folder in the form of interactive python Jupyter noteboooks. Note that whilst ThamesWater is used as an example, the same operations apply to all of the water companies supported by POOPy (with the exception of the historical data operations which are currently only supported by Thames Water):

• Investigating the current status of sewer overflow spilling
• Investigating the historical status of sewer overflow spilling

POOPy can be used to be really easily make figures like...

... this one showing the stretches of the Thames downstream of active sewage discharges at the shown time...

from poopy.companies import ThamesWater
tw=ThamesWater()
tw.plot_current_status()

...or this one which shows the discharge history of a specific monitor...

from poopy.companies import ThamesWater
tw=ThamesWater()
monitor=tw.active_monitors["Bourton-On-The_water"]
monitor.plot_history()

...or this one which shows the number of live monitors deployed by Thames Water through time and whether they were discharging...

Contributions are very welcome! Take a look at the issues tab, or reach to us if you have suggestions or features. Or, just dive straight in.

This repository uses automatic code formatting and linting. To ensure consistency in the codebase, tools are run automatically on pre-commit. Before you commit, the following hooks will automatically run:

• Black: Automatically formats your code to adhere to the Black code style.
• Ruff: Automatically runs a linter on your code to catch style issues, errors, and potential bugs.

To set up the pre-commit hooks and ensure your code is formatted and linted before committing, run the following:

1. Install pre-commit if you haven’t already: pip install pre-commit
2. Install the hooks: pre-commit install Now the checks will automatically run when you commit, but you can:
3. Test them manually with: pre-commit run --all-files

The settings for these linters is detailed in .pre-commit-config.yaml.

If you use these scripts, or the data, please reference its source. For instance:

"Data generated using the POOPy software (github.com/AlexLipp/POOPy)"

Whilst every effort has been made to ensure accuracy, this is experimental software and errors may occur and I assume no responsibility or liability for any such errors. If you see any issues please contact me, or raise an `Issue' above. This code is licensed under the GNU General Public License 3.0