The AEA Data Editor's mission is to design and oversee the AEA journals' strategy for archiving and curating research data and promoting reproducible research.
In this post, I’ll pick up questions that have been asked over Twitter or during the 2021-01-19 Fireside chat, in long form (Twitter is not always the best medium). Over time, you might see some of these questions and answers be migrated to FAQ and/or best practices documents.
Read more
I wanted to highlight a recently (2020-04-30) published article in the AEJ:Economic Policy as an example of reproducibility checking when data is restricted-access.
Read more
Since July 16, 2019, the American Economic Association has used the AEA Data and Code Repository as the default archive for its supplements. The AEA also announced that it would migrate the historical supplements, hitherto stored as ZIP files on the AEA website, into the AEA Data and Code Repository. Between Oct 11 and Oct 13, 2019, openICPSR ingested 2,552 historical supplements. Additional supplements were ingested in December 2019.
Authors of articles may have received the following email:
Dear Dr. So-and-so,
Since July 16, 2019, the American Economic Association has used the AEA Data and Code Repository at openICPSR as the default archive for its supplements. The migration increases the findability of your data through a variety of federated search interfaces such as Google Dataset Search, the openICPSR search interface, and the general ICPSR search interface.
[more details]
For articles with more than one author, each co-author is receiving an identical email with an individualized link. Thank you for your effort!
While immediately making these replication packages findable through Google Dataset Search and ICPSR search interfaces, the metadata for these deposits is limited to title, authorship, abstract, and JEL codes. Modern metadata standards allow for a much richer description of the materials, such as geographic context, statistical coverage or sampling techniques, etc.
Request to Authors
Authors have the best knowledge of the data they used. We are thus asking authors to enter additional metadata through an online form. Once the project is completed, all metadata provided will be integrated with the openICPSR database, and will henceforth become available publicly.
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Read more
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Read more
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Read more
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Replicability is at the core of the scientific enterprise. In the past 30 years,
recurring concerns about the extent of replicability (or lack thereof) of the research in various disciplines have surfaced, including in economics.
In this talk, I describe the context in which the current discussion in the
social science is occurring: what are the definitions of replicability and
reproducibility, what is failing, and to what extent. I discuss progess over the past 15 years.
Finally, I discuss the concrete measures that have been implemented under
my guidance at the American Economic Association, and the first preliminary outcomes
from those measures. I conclude with some observations on how to integrate reproducibility
into the scientific workflow in the social and statistical sciences.
The solutions to these problems will change the way research will be taught
and conducted, in economics in particular, and in the social sciences more
broadly. The implications affect undergraduate and graduate teaching, research
infrastructure, and habits.
Description: Reproducibility and replicability are critical elements of credible scientific research. Data provenance is an important, but often neglected piece of replicability.
In particular when data cannot be published, but can be accessed by shared community, properly documenting provenance is essential, but difficult.
I report on the experience gathered from nearly 1,000 reproducibility reports, and on the guidance we give to authors in order to provide good-enough data provenance.
Description: Reproducibility and replicability are critical elements of credible scientific research. Data provenance is an important, but often neglected piece of replicability.
In particular when data cannot be published, but can be accessed by shared community, properly documenting provenance is essential, but difficult.
I report on the experience gathered from nearly 1,000 reproducibility reports, and on the guidance we give to authors in order to provide good-enough data provenance.
Description: Reproducibility and replicability are critical elements of credible scientific research. Data provenance is an important, but often neglected piece of replicability.
In particular when data cannot be published, but can be accessed by shared community, properly documenting provenance is essential, but difficult.
I report on the experience gathered from nearly 1,000 reproducibility reports, and on the guidance we give to authors in order to provide good-enough data provenance.
Notes
Focus on reproducibility in restricted-access data centers
Description: Reproducibility and replicability are critical elements of credible scientific research. Data provenance is an important, but often neglected piece of replicability.
In particular when data cannot be published, but can be accessed by shared community, properly documenting provenance is essential, but difficult.
I report on the experience gathered from nearly 1,000 reproducibility reports, and on the guidance we give to authors in order to provide good-enough data provenance.
The Beyond the Numbers 2020 website has a video recording.
Read more
Description: Reproducibility and replicability are critical elements of credible scientific research. Data provenance is an important, but often neglected piece of replicability.
In particular when data cannot be published, but can be accessed by shared community, properly documenting provenance is essential, but difficult.
I report on the experience gathered from nearly 1,000 reproducibility reports, and on the guidance we give to authors in order to provide good-enough data provenance.
Read more