28_sivacor_partb.sh - Apply the SIVACOR-generated Part B during ingest

Description¶

Ingest-pipeline step that checks whether the repository contains a SIVACOR artifact (a */tro/tro-*.jsonld TRO file). If one is present, it runs 18_summarize_sivacor.sh to produce generated/REPLICATION-PartB-SIVACOR.md, and then either replaces REPLICATION-PartB.md with it (first run) or leaves it in generated/ and notifies the Jira ticket (later runs).

It runs in both ingest pipelines (1-populate-from-icpsr and w-big-populate-from-icpsr) right after 25_replace_report.sh, and in tools/pipeline-steps1-4.sh. When no SIVACOR artifact is present, or SkipProcessing=yes, it exits without doing anything.

Usage¶

./automations/28_sivacor_partb.sh [pipeline-name]

• pipeline-name - Optional. Name of the Bitbucket pipeline, passed to 70_publish_comment.sh so the Jira comment says which pipeline produced it.

Environment Variables¶

• jiraticket - Jira ticket used as the commit-message prefix and, via 70_publish_comment.sh, as the comment target. Falls back to config.yml for the comment.

• SkipProcessing - When yes, the script exits immediately.

• PYTHON - Passed through to 18_summarize_sivacor.sh.

Behavior¶

1. Find the first */tro/tro-*.jsonld file (excluding .git/). None found: print a message and exit 0.

2. Decide whether REPLICATION-PartB.md is still pristine. It is pristine when it exists, is tracked by git, and its current content is identical to the version in the commit that created it (the pipeline’s “Added split report” commit). Any later edit, committed or not, marks it as modified. A missing REPLICATION-PartB.md (unsplit or revision report) also counts as modified.

3. Run ./automations/18_summarize_sivacor.sh -j <jsonld> to write the four generated/sivacor-partb-*.md snippets and generated/REPLICATION-PartB-SIVACOR.md.

4. Pristine Part B: copy the generated file over REPLICATION-PartB.md, commit it together with the generated files ([skip ci]), and post a Jira comment saying the SIVACOR Part B was applied.

5. Modified Part B: commit only the generated files and post a Jira comment saying the SIVACOR Part B is waiting in generated/REPLICATION-PartB-SIVACOR.md, with the command to apply it (./automations/18_summarize_sivacor.sh --replace-report).

Jira comment failures are non-fatal. Because the pristine check compares against the file’s creation commit, re-running the pipeline after the script has replaced Part B once will not overwrite it again.

Notes¶

• The TRO can arrive in two ways. openICPSR deposits may include the SIVACOR tro/ directory directly, in which case the artifact is part of the downloaded deposit and this step runs during the initial ingest with no extra action. Alternatively, tools/download_sivacor.py (pipeline 9-download-sivacor) fetches the artifacts from SIVACOR and commits them to a sivacor-<id> branch; the ingest pipeline then only sees the artifact once that branch is merged into the branch it runs on.

• Only the highest arrangement recorded in the TRO is summarized; see get_sivacor_info.py for what the generated snippets contain.