Description¶
Compares two SHA256 manifests (sha256sum format, as written by automations/02_create_manifest.sh) and reports what changed in a meaningful way. Usually this is the previous and current ingest of a deposit, for example across revision rounds.
Every file in the two manifests ends up in exactly one category:
| Category | Rule | Meaningful? |
|---|---|---|
| Unchanged | same path, same checksum | no (count only) |
| Content changed | same path, different checksum | yes |
| Renamed | same checksum, different file name (possibly also a new directory) | yes |
| Moved and changed | path gone; a file with the same name appears elsewhere with different content, and the name is unique on both sides | yes |
| Added / Removed | everything else | yes |
| Moved only | same checksum and file name, different directory | no (listed grouped by directory) |
Matching is done on full paths first. Files are paired by checksum only when their path changed, so files that share a name in different folders (for example firms.txt in each yearly folder) are never cross-matched.
Added files whose content is identical to a file in the previous manifest are flagged as copies. Added or removed files that make up a wholly new or wholly removed directory are listed as that directory with a file count, e.g. “input/adoinstalled/ (new directory, 292 files)”.
Usage¶
python3 tools/compare_manifests.py PREVIOUS CURRENT [--summary-file FILE]PREVIOUS,CURRENT- the two manifests, oldest first.--summary-file FILE- also write a short summary: the counts table plus the meaningful changes, capped at 25 entries per category.
The full report (Markdown) goes to standard output. automations/22_compare_manifests.sh writes it to generated/manifest-comparison.md, which is included in the report appendix, and writes the summary to generated/manifest-cmp-summary.md.
Example output¶
| Category | Files |
|---|---:|
| Files in previous manifest | 742 |
| Files in current manifest | 742 |
| Unchanged (same path and content) | 740 |
| **Content changed** (same path) | 2 |
| **Renamed** (same content, new name) | 0 |
| **Moved and changed** (same name, new location, new content) | 0 |
| **Added** | 0 |
| **Removed** | 0 |
| Moved only (same name and content, new location) | 0 |
**Content changed** (2):
- `README.pdf`
- `_step_1_pt_16_descriptives.do`