Ungated logo
Ungated logo
About this siteFAQContact

About this Site

What is the purpose of ungated research?

Recent research shows that high pricing of academic journals present barriers for researchers, especially in low income countries. After all, reading recent published work is essential to learn about new methods and identify potential contributions to the literature.

The website provides two main services:

  1. Assemble and provide access to all publicly available working papers for research in leading economics journals in one place
  2. Provide AI-generated non-technical summaries of research for these papers

Given the limitations of AI-generated content, we envision that these two services are used jointly: summaries are used as starting points for engagement with the full papers. Below, we provide more information about the method and limitations.

This website is an attempt to provide a public good to the research community and students. We will never charge fees, have advertisements, or collect user data (other than website traffic statistics). The website is housed out of Bowdoin College.

There are different ways you can use the website:

  • Browse by journal to get a broad overview of (newly) published work. Simply click on the journal of interest on the home page.
  • Use the filter function (circled in blue) to get a list of articles in a certain sector, location or use a specific method.
  • Use the search bar (circled in red) to look for specific keywords or authors. You can use this in combination with the filter function.
    • search-bar-demo
  • One of the most useful functions is to filter a combination of tags - e.g. searching for field experiments in behavioral economics in certain locations. Articles that meet these criteria automatically show up on the left.
    • search-bar-demo-2

We use the following steps to compile the database and AI generated summary:

  • Identify titles from journals’ publicly available table of contents
  • Search the internet for publicly available working papers of articles. We are able to find working papers for around 95% of journal articles.
  • Copy the introduction of each article into generative AI (Gemini) and use a standardized prompt (see FAQ) to create a summary

Depth of summaries

The summaries provided on this website are generated by AI (specifically by Gemini), based on the introduction of the working paper version of academic articles. These summaries aim to offer quick insights but will not capture all nuances of the original work. It may also not fully reflect the authors' intentions. It's also important to acknowledge that AI may occasionally produce "hallucinations", or inaccuracies not present in the original text. We envision that AI-generated summaries are used as starting points for engagement rather than definitive interpretations. For comprehensive understanding and to engage critically with the subject matter, readers are encouraged to consult the full articles that can be accessed via the “Download Working Paper” button.


Assessing quality of summaries

To assess the quality of summaries, we contacted a (non-random) group of 20 researchers who have published in these journals and shared summaries of one of their articles using our standardized prompt. We asked them to give confidential feedback on the accuracy of the summaries. The graph below shows the accuracy ratings (on a 1-10 point scale) for aspects of the summaries and the tags. The overall accuracy is relatively high (7.63 points) - however, none of the researchers gave the maximum 10 score (and one person said their research would “not get a passing grade”). The two highest scoring categories were research question (8.28) and findings (8.24). Two summary categories (“Theory” and “Limitations”) only received accuracy scores between 5 and 6 and were thus excluded from AI summaries provided on the website. Ratings on tags were also varied: while sector (8.36) and location (9.27) received high accuracy scores, method tags received more varied reviews (6.94).


ratings-chart

To assess the hallucination problem, we also asked researchers if summaries include incorrect information. There were several comments that limitations and theory summaries included errors. We subsequently took them out. One author noted errors in the method description and one author in the sample / context description. Importantly, there was no case of incorrect information among “Findings”. But please keep in mind the small sample size for this quality and do not solely rely on the summaries.



Scope of research

An important caveat is that we are only covering a limited number of journals and time of publication. Ungated Research is therefore not a good tool to conduct systematic literature reviews.

We are currently providing articles from ten journals for (roughly) the last five years. We are aware that we are contributing to the “myth of the Top 5” with our selection and hope to expand the scope, both in terms of journals and extending the period of time covered.



Non peer-reviewed paper versions

For legal reasons, we base our summary on the most recent publicly available working paper, not the peer-reviewed published version. The graph below shows the time lag between the journal publication and publishing date (calculated by subtracting the working paper year from the publication year). The time lag is approximately one year or less for 49% of the working papers. One explanation for this short time lag is that authors often upload updated working paper versions of their research. However, the AI-generated summaries may not capture any changes made during the peer-review process.


time-lag-chart

Repositories to promote the free dissemination of research

  • SSRN (Social Science Research Network) - Primarily for social sciences, SSRN is a repository for researchers to share papers in economics, law, corporate governance, etc..
  • arXiv - Primarily for physics, mathematics, computer science, quantitative biology, quantitative finance, and statistics.
  • RePEc (Research Papers in Economics) - Provides bibliographic data on economic research and distributes working papers and links to published articles.

Research Summaries and literature reviews

  • Vox Dev - Summarizes research and provides literature reviews with a focus on development economics.
  • JPAL - Policy-focused summaries of experimental research with a focus on development economics.
  • 3ie (International Initiative for Impact Evaluation) - Provides an evidence hub of impact evaluation research, and systematic reviews with links to published articles.
  • Pep-net - Summarizes research and provides literature reviews with a focus on development economics.

AI-powered tools to summarize research

There are a range of services including Elicit, Consensus, and ScholarAI . These are paid services and only offer limited free plans.


Email us with suggestions of other helpful websites to list.

    PIs

    Martin Abel (Bowdoin College), Susan Godlonton (Williams College)


    Web design/development and research analysis

    Olivia Wirsching, Zane Bookbinder, Ulemj Munkhtur, Mingi Kang


    Research Assistants

    Brando Izquierdo, Tyler Lenk, Becky Walker, Jonathan Hartanto, Rafaela Delgado, Zia Saylor, Daniel Carrera Mora, Simon Wainana, Aikedumanwen Izogie-Eghe


    Volunteers

    Jonathan Li, Rickey Karunadhara, Anh Nguyen, Shannon McCall, Alex Wiseman, Maddy Kosmoski , Sam Wilson, Liam Hurtt, Chase Lenk, Joey Zheng, Brendan Deppen, Julie Jansen, Raghad Mohamed, Sam Lieman, Siddhu Srivatsan

If you want to volunteer and offer at least 5 hours of your time to search for working papers of published articles, please email us at ungatedresearch@outlook.com.