1.3.6. Creating hillslope datasets

Creating hillslope datasets for use in CTSM is a multi-step process:

Perform the Height Above Nearest Drainage (HAND) analysis, which processes the DEM data for each gridcell, breaking the work into a number of “chunks.”
Combine the per-gridcell files from the HAND analysis into one file per chunk.
Combine the per-chunk files into a hillslope_file for use in CTSM.

1.3.6.1. HAND analysis

The HAND analysis uses the optional Representative_Hillslopes submodule. Before following these instructions, do bin/git-fleximod update -o to make sure that’s been downloaded. Then the instructions assume:

You’ve installed the Representative_Hillslopes Conda environment, if needed (conda env create -f tools/external/representative-hillslopes/conda_yml/casper.yml).
You’ve activated that environment (conda activate Representative_Hillslopes).
You’ve changed your working directory to tools/external/representative-hillslopes/shell_scripts/.

1.3.6.1.1. Front matter: README for `Representative_Hillslopes` shell scripts

Sam S. Rabin (samrabin@ucar.edu) (2024-10-08)

1.3.6.1.2. `hand_analysis_loop.sh`

This script performs the Height Above Nearest Drainage (HAND) analysis for each gridcell. It breaks the analysis into a number of chunks—by default, 36—to enable parallel processing using batch submission scripts. Do

./hand_analysis_loop.sh --help

for more information.

Note that if you want to submit each chunk’s analysis to a separate batch job, you must provide batch_submit_preamble.sh (which will not be tracked by git). This script must echo everything except the actual call of call_hand_analysis.sh. It will be called in a subshell of hand_analysis_loop.sh, so you can refer to that script’s variables $log_out and $log_err to save logs of STDOUT and STDERR, respectively.

1.3.6.1.3. `hand_analysis_1chunk.sh`

This script calls the Python script that does the analysis for one “chunk”. hand_analysis_loop.sh calls this script in every iteration of its loop.

./hand_analysis_1chunk.sh --help

Your machine may need some additional code to be called before calling the HAND analysis Python script. You can put this in 1chunk_preamble.sh, which will not be tracked by git.

1.3.6.1.4. Example

These scripts were tested on NSF NCAR’s Casper machine like so (bash). The analysis takes a long time—indeed, even with 12 hours of walltime, some of the chunks may not complete. (You can check if a chunk has completed by checking for its done_ file in $outdir.) If that’s the case, you can rerun this code as many times as needed, and the chunks that haven’t completed will start where they left off.

# Preamble for submitting a PBS job. Note "echo echo".
echo echo qsub -V -q casper -A $PROJECT -l walltime=12:00:00 -l select=1:ncpus=1:mem=32GB -o \${log_out} -e \${log_err} -- > batch_submit_preamble.sh

# Preamble for calling the HAND analysis Python script for a single chunk. Note "echo echo".
echo -e "module load ncarenv \n echo conda run -n Representative_Hillslopes" > 1chunk_preamble.sh

# Arguments
fsurdat="/glade/campaign/cesm/cesmdata/cseg/inputdata/lnd/clm2/surfdata_map/surfdata_0.9x1.25_78pfts_CMIP6_simyr2000_c170824.nc"
demdata="/glade/campaign/cgd/tss/projects/hillslopes/MERIT/data"
outdir="$SCRATCH/hillslope_script_testing/hand_analysis_1deg"

# Call script
./hand_analysis_loop.sh $fsurdat $demdata $outdir

1.3.6.2. Gridcell files to chunk files

(Instructions here assume you have activated the ctsm_pylib Conda environment.)

This is performed by the tools/hillslopes/combine_gridcell_files script; do tools/hillslopes/combine_gridcell_files --help for usage help. Briefly, you need to provide three things:

-i, input surface dataset: The CTSM surface dataset (fsurdat) you used in the HAND analysis above.
-d, input directory: The directory the HAND analysis above saved files to (outdir in those instructions).
-o, output directory: The directory where you want the chunk files generated by combine_gridcell_files to be saved.

1.3.6.2.1. Example

This example can be run on the NSF NCAR Casper machine; it assumes you’ve activated the ctsm_pylib Conda environment.

fsurdat=python/ctsm/test/testinputs/surfdata_5x5_amazon_hist_16pfts_CMIP6_2000_c231031.nc
indir=$SCRATCH/hillslopes_5x5_amazon/hand_analysis_global
outdir=$SCRATCH/hillslopes_5x5_amazon/hand_analysis_global/combined

python tools/hillslopes/combine_gridcell_files \
    -i $fsurdat \
    -d $indir \
    -o $outdir

1.3.6.3. Chunk files to `hillslope_file`

(Instructions here assume you have activated the ctsm_pylib Conda environment.)

This is performed by the tools/hillslopes/combine_chunk_files script; do tools/hillslopes/combine_chunk_files --help for usage help. Briefly, you need to provide three things:

-i, input surface dataset: The CTSM surface dataset (fsurdat) you used in the steps above.
-d, input directory: The directory where combine_gridcell_files saved the chunk files (-o in those instructions above).
-o, output file: The file path where you want the output hillslope_file to be saved.

1.3.6.3.1. Example