Description¶
Lists the data files in a directory, identified by extension (dta, csv, xlsx, rds, parquet, sas7bdat, archives and others; see the extensions variable in the script). When generated/manifest[.tag].txt and generated/metadata[.tag].txt exist (from 02_create_manifest.sh), it filters them instead of walking the file tree again.
Usage¶
./automations/03_list_data_files.sh <directory> [tag] [zipfile]directory- Deposit directory, or another directory such asrestricted.tag- Optional suffix for the input and output names.zipfile- Accepted for consistency; not used.
Outputs¶
generated/data-list[.tag].txt- Data filesgenerated/data-metadata[.tag].csv-filename,bytes
See also summarize_data.py, which works from these lists.