00_preliminaries.sh - Prepare a parallel scanner step

Description¶

First command of each parallel scanner step in 1-populate-from-icpsr. The Download step hands the deposit ZIP to later steps through the cache/ artifact; this script moves cache/<projectID>.zip back into the working directory, so that 00_unpack_zip.sh can unpack it.

Usage¶

./automations/00_preliminaries.sh <yes|no> <projectID>

• First argument - The step’s Process* switch (for example $ProcessStata). When it is no, the script exits without doing anything.

• projectID - Deposit directory name; the ZIP is expected at cache/<projectID>.zip.

Notes¶

• It lists the contents of cache/ for the pipeline log.

• It also sets a pythonbin variable, but because the script runs in its own shell, that variable is not visible to the commands that follow it.