get_sivacor_info.py - Extract information from SIVACOR JSONLD files

Description¶

Parses SIVACOR JSONLD (TRO - Transparent Research Object) files to extract computing environment and execution timing information. Can output to stdout or automatically update replication reports with the extracted information in the appropriate sections.

Usage¶

# Output to stdout
python3.12 tools/get_sivacor_info.py <jsonld_file> <keyword>
python3.12 tools/get_sivacor_info.py --jsonld <file> --key <keyword>
python3.12 tools/get_sivacor_info.py --jobid <job_id> --key <keyword>

# Update report file
python3.12 tools/get_sivacor_info.py --jobid <job_id> --key <keyword> --report <report_file>

# Dry-run (preview without updating)
python3.12 tools/get_sivacor_info.py --jobid <job_id> --key <keyword> --report <report_file> --dry-run

Arguments¶

Positional Arguments:

• jsonld_file - Path to JSONLD file (e.g., tro-69cede1db3a6af67b1c01c3d.jsonld)

• keyword - Information keyword to extract (computing or time)

Named Options:

• --jsonld <file> - Path to JSONLD file (alternative to positional)

• --jobid <id> - SIVACOR job ID (searches for tro-{jobid}.jsonld)

• --key <keyword> - Information keyword (alternative to positional)

• --report <file> - Report file to update with information

• --dry-run - Preview changes without modifying files

Keywords¶

computing¶

Extracts and displays computing environment information:

• SIVACOR Job ID

• Processor type

• Number of CPUs

• Total memory

• Operating system and version

• Kernel version

• Docker image

• Max CPU usage percentage

• Max memory usage

• OS type

When --report is specified, adds information to the “Computing Environment of the Replicator” section under a “SIVACOR” heading.

time¶

Extracts and displays execution timing information:

• SIVACOR Job ID

• Start timestamp

• Finish timestamp

• Calculated duration (formatted as hours/minutes/seconds)

When --report is specified, adds information to the “Findings” section under a “SIVACOR Execution Time” heading.

Examples¶

# Extract computing info and print to stdout
python3.12 tools/get_sivacor_info.py --jobid 69cede1db3a6af67b1c01c3d --key computing

# Extract timing info and print to stdout
python3.12 tools/get_sivacor_info.py --jobid 69cede1db3a6af67b1c01c3d --key time

# Preview what would be added to report (dry-run)
python3.12 tools/get_sivacor_info.py --jobid 69cede1db3a6af67b1c01c3d --key computing --report REPLICATION-PartB.md --dry-run

# Add computing info to report
python3.12 tools/get_sivacor_info.py --jobid 69cede1db3a6af67b1c01c3d --key computing --report REPLICATION-PartB.md

# Add timing info to report
python3.12 tools/get_sivacor_info.py --jobid 69cede1db3a6af67b1c01c3d --key time --report REPLICATION-PartB.md

# Using positional arguments
cd 246302
python3.12 ../tools/get_sivacor_info.py tro-69cede1db3a6af67b1c01c3d.jsonld computing

# Using file path directly
python3.12 tools/get_sivacor_info.py --jsonld 246302/tro-69cede1db3a6af67b1c01c3d.jsonld --key time

Requirements¶

• Python >= 3.12

• Standard library modules: json, argparse, sys, os, glob, re, datetime

Output Format¶

Information is formatted as Markdown bullet points with the SIVACOR Job ID displayed in backticks for proper rendering.

Computing Output Example¶

- SIVACOR Job ID: `69cede1db3a6af67b1c01c3d`
- Processor: AMD EPYC-Milan Processor
- CPUs: 16
- Total Memory: 58.8 GB
- Operating System: Ubuntu 24.04.3 LTS (Version 24.04)
- Kernel Version: 6.17.0-14-generic
- Docker Image: `dynare/dynare:6.1-R2024a`
- Max CPU Usage: 315.32%
- Max Memory Usage: 2.34 GB
- OS Type: linux

Time Output Example¶

- SIVACOR Job ID: `69cede1db3a6af67b1c01c3d`
- Started: 2026-04-02T21:22:40.882500138Z
- Finished: 2026-04-03T05:13:05.690180035Z
- Duration: 7h 50m 24s

Report Integration¶

When using the --report option, the script:

1. For computing keyword:

   • Locates the “Computing Environment of the Replicator” section

   • Inserts SIVACOR information after existing environment items

   • Adds under “SIVACOR” heading

2. For time keyword:

   • Locates the “Findings” section

   • Inserts timing information after the heading and INSTRUCTIONS

   • Adds under “SIVACOR Execution Time” heading

Duplicate Detection¶

If the script detects that a SIVACOR section already exists in the report:

• Displays a warning message with ⚠️ emoji

• Shows the existing information in Markdown format

• Does not update the file (prevents duplicates)

Example warning output:

⚠️  WARNING: SIVACOR computing section already exists in report.

Existing information in Markdown notation:

**SIVACOR**

- SIVACOR Job ID: `69cede1db3a6af67b1c01c3d`
...

Job ID Detection¶

The script can automatically detect the SIVACOR Job ID in multiple ways:

1. Via --jobid option: Directly specified by user

2. From filename: Extracts from pattern tro-{jobid}.jsonld

3. File search: Recursively searches for tro-{jobid}.jsonld when --jobid is used

Workflow¶

1. Parse command line arguments

2. Locate JSONLD file (by path or job ID search)

3. Read and parse JSON data

4. Extract relevant information based on keyword

5. Format information as Markdown bullets

6. If --report specified:

   • Check for existing SIVACOR section

   • If dry-run, display what would be added

   • Otherwise, insert into appropriate report section

7. Output results to stdout or update report file

Error Handling¶

• Validates JSONLD file existence

• Checks for valid JSON format

• Verifies keyword is supported (computing or time)

• Ensures report file exists when --report is used

• Handles missing SIVACOR fields gracefully

• Prevents duplicate section creation

See Also¶

• download_sivacor.py - Download SIVACOR artifacts

• SIVACOR TRO (Transparent Research Object) specification

• Replication report templates (REPLICATION-PartB.md)