AEA Data Editor

AEA Data Editor

The AEA Data Editor’s mission is to design and oversee the AEA journals’ strategy for archiving and curating research data and promoting reproducible research.

Discussion with the Research Publication Group at the Federal Reserve Board

Date:

Mode: Online · Language: 🇺🇸 · Participants: 15

AMA at Federal Reserve Board, Washington, DC

Topics

The meeting was interactive. Questions:

  • Let us know about the latest developments in replication practices for the AEA and the profession.
  • Can you outline the basic principles and purpose of replication? Perhaps this would be a summary of AEA requirements and mention of how other publishers are addressing this.
  • How has the profession been adapting to new replication requirements?
  • Do you see a need for additional/stronger replication standards? When might those standards be adopted?
  • How do you coordinate replication standards with other journals? (We have observed more journals hiring data editors—e.g., recent RFS job posting—and adopting the AEA standards.)

  • Are you aware of plans for replication requirements at any working paper series, e.g., NBER Working Papers?

  • Could you describe a little more the process that you use to test the replication files that are submitted? - Is the process entirely manual or is there an automated (AI?) code reader that is helping to identify issues with the replication package? - If (part of) the process is automated, is the software system you’re using commercially available? - If (part of) the process is automated, what’s the role of the human in the process?

  • How could researchers at the Federal Reserve Board make it easier for editors and data editors to approve their replication packages?
    • Apart from the publicly-available AEA guides that we’re familiar with, what other best practices would you recommend?
    • Which other institutions are at the frontier of replication approach(es) when publishing analysis based on confidential/regulatory/licensed datasets?
  • What about something like “Some institutions publish analyses based on confidential/regulatory/licensed datasets. How do they handle replication?”
    • For cases like the above, would the use of synthetic and sanitized version of confidential or restricted data in code replication be useful? Even if it could not directly replicate the paper results, but only demonstrate the functionality of the code base?
  • You had recommended having an in-house team to replicate accepted papers. Do you still recommend this approach? Could you describe how other institutions have implemented this approach, and the level of effort?