aea-box-clean-folders

Source

Cleans up Box folders for completed Jira cases:

1. Scans the Box root folder for case folders (aearep-XXXX).

2. Checks whether each case is ready for purging via its Jira status.

3. For ready cases: disables the folder’s upload link, deletes data files (CSV, DTA, ZIP, …), keeps documents (PDF, DOCX, TXT, …), and moves the folder into the 1Completed subfolder.

The upload link is the file request whose ID aea-box-create-folder recorded in the folder description. Folders without a recorded ID are reported; disable their upload link in the Box web interface.

aea-box-clean-folders --test          # dry run
aea-box-clean-folders                 # process all ready cases
aea-box-clean-folders --case 1234     # process one case
aea-box-clean-folders --list          # list cases and their status
aea-box-clean-folders --all --email   # also send restricted-data deletion notices
aea-box-clean-folders 1234 --force-delete  # also delete files the filter keeps

Files not deleted¶

For each processed case, the files to delete and the files kept by the extension filter (documents and unrecognized types) are reported, summarized by extension with file count and total size. The SUMMARY at the end gives the files not deleted across all cases, with the same breakdown by extension.

Deleting everything (--force-delete)¶

When the extension filter keeps files that should also go, --force-delete lists every kept file of the case (path and size) after the data files are deleted, then asks for confirmation (default No) before deleting them. The confirmation is per case and is not skipped by --yes. Under --test, the list is shown and the deletions are previewed without prompting.

Restricted-data deletion notice (--email / -e)¶

For a cleaned-up case whose Jira issue (or, if revised, the last issue in its “is revised by” chain) has a Request confidential data subtask, the script can additionally:

1. Post an internal comment on both that issue and the subtask, confirming the Box deletion, reminding the reader to also delete any offline copies, and giving the aea-box-recover-files --case <N> --days 30 command to undo it. The comment is prefixed with a 🤖 marker and ends with an italicized note identifying it as automated.

2. Set the issue’s Was data deleted? field to Yes.

3. Send an author-facing deletion-confirmation email (based on the restricted-data request template) to dataeditor@aeapubs.org, with the subtask key and manuscript number in the subject line. The email includes an editor’s note asking the recipient to fill in the author’s name and the manuscript title and forward it — the author’s own address isn’t available in Jira, so this can’t be sent directly. If sending fails (e.g. no mailbox password available), the email’s To/Subject/body are printed to the console instead, to copy into a mail client by hand.

Cases with no such subtask are skipped silently — there’s nothing to notify.

This is off by default. Without --email, you’re prompted interactively (default No) once before processing starts. Pass --email/-e to enable it unconditionally, including under --test (which only previews the notice).

Environment¶

Variables not set in the environment are read from ~/.envvars.

• Box: the Box app credentials (see Box setup)

• Jira: JIRA_USERNAME, JIRA_API_KEY

• For --email: DATAEDITOR_EMAIL_PASSWORD (fallback if the 1Password CLI op has no session, e.g. when running remotely; otherwise read from the 1Password item “Email for AEA Dataeditor”), optionally DATAEDITOR_SMTP_HOST (default mail.aeapubs.org) and DATAEDITOR_SMTP_PORT (default 587)

Deleted files can be restored with aea-box-recover-files.

Box setup¶

The scripts authenticate as a Box JWT app. Its credentials are a JSON file named <ENTERPRISE_ID>_<KEY_ID>_config.json, available at 🔒 this private Box link.

1. Download the JSON file and store it in ~/.config/box/, readable only by you:

   mkdir -p ~/.config/box
   mv ~/Downloads/*_config.json ~/.config/box/
   chmod 600 ~/.config/box/*_config.json

To keep the JSON file elsewhere, also add its directory to ~/.envvars:

BOX_CONFIG_PATH=/path/to/directory

Credentials are looked up in this order:

Shared settings¶

The non-confidential settings of the Box scripts are in box_config.py. An environment variable (or ~/.envvars entry) overrides the corresponding setting: