Python ISNOBAL Interface

The Image Processing Workbench (IPW) software package is the standard on which we base our work here. We need to extend this functionality a bit, if for no other reason to improve the usability and documentation, which seem pretty lacking.

IPW is a pretty large project of which the ISNOBAL model is one part. The groundbreaking paper that established ISNOBAL as one of the premier snowmelt water input models is Marks, et al., 1999. The ISNOBAL documentation, including what each band is in each type of file, can be found here.

The best way to find a file is by first searching through this grouped list of Standard IPW User Commands.

Every IPW file has some lines dedicated to its header, which describes the image, and a single binary line that must be parsed according to the information in the header. The line of binary data represents a data cube, visualized here. There are \(N_l\) lines of data, \(N_s\) geographic samples, and \(N_b\) bands. Transformed to human-readable data, the binary data looks something like this made up example:

Temp Pressure Dew Point
23.4 101.0 21.0
24.5 100.0 20.0
24.1 102.3 22.0
... ... ...

You can see what the binary data actually looks like in Python by doing the following:

filePath = "path/to/in.0000"

with open(filePath, 'rb') as f:
    lines = f.readlines()

header = lines[:-1]
binaryPart = lines[-1]

binaryPart[:100]

Only looking at the first 100 characters will keep your screen from getting filled with hexadecimal digits.

Before we read any of that data to be integers, which we can then convert to floating point like in the table above, we need to know the scheme for doing so. That is given to us in the rest of the basic image header, which tells us

  1. The number of bits used to store a sample in the band
  2. The number of bytes used to store these bits
  3. The floating-point range of each band
  4. (Optional) Annotative text describing the band

Basic IPW Operations

We can get a summary of this information using the IPW command ipwfile:

$ ipwfile in.0000
File: "in.00"  Bands: 5  Lines: 148  Samples: 170  Bytes/Pixel: 1 1 2 2 1

Note: some input files have five bands and some have six. The last one may be omitted if the sun is down.

We can get a view of the floating point values stored in an image file using the primg command:

$ primg -a -i in.0000
292.157 22.4 468.743 0.84229 0
296.078 22.4 468.743 0.84229 0
296.078 22.4 468.743 0.84229 0
294.118 22.4 468.743 0.84229 0
296.078 22.4 468.743 0.84229 0
296.078 22.4 468.743 0.84229 0
300 22.4 468.743 0.84229 0
300 22.4 468.743 0.84229 0
294.118 22.4 468.743 0.84229 0
296.078 22.4 468.743 0.84229 0

Note that

$ primg -a -iin.0000 | wc -l
25160

is equal to the number of lines times the number of samples, which makes “lines” seem a bit of a misnomer.

Make Watershed Metadata

Reading IPW with Python

The IPW class takes care of reading, modifying, and writing modified IPW files. To use it, simply provide the file name you wish to load into Python.

Insert IPW file or directory of files to Virtual Watershed

If we have some iSNOBAL data in the folders data/inputs and data/outputs, we can either upload all the files in thosy directories or indivdual files. Providing a description is required. If the parent or model run IDs are provided, they will be used. If not provided, upsert will create new ones. If the model run UUID is provided, its parent must also be provided, though this is may be changed in the near future. Some examples are shown below.

# set base data directory and description for this model run
data_dir = "data/"
description = "iSNOBAL model run with Treeline observed data"

# upload the entire directory, creating a new parent and model run uuid
input_dir = data_dir + "inputs"
parent_uuid, uuid = upsert(input_dir, description)

# use the existing parent_uuid and uuid to insert the output data
input_dir = data_dir + "outputs"
parent_uuid, uuid = upsert(output_dir, description,
                           parent_model_run_uuid=parent_uuid,
                           model_run_uuid=uuid)

upsert uses a config file and watershed.default_vw_client to connect to the virtual watershed. You can set which configuration file upser uses using the last keyword argument, config_file.

Run ISNOBAL from Python

The ISNOBAL interface is a simple, straight-forward wrapper for ISNOBAL.