We are launching the Autoscience Institute, a research lab that builds AI systems to improve AI systems. Our first step towards that vision, Carl, generated research papers accepted through double-blind peer reviews at a workshop in the International Conference on Learning Representations (ICLR) with no substantive human involvement.
We're excited to introduce Carl, our new AI system that successfully ideated novel scientific hypotheses, designed and performed experiments, and wrote multiple academic papers that passed peer review at workshops in the International Conference on Learning Representations (ICLR) on the Tiny Papers track. This milestone marks Carl as the first AI system to produce academically peer-reviewed research, demonstrating the potential of AI in advancing scientific knowledge.
What is Carl
Carl is an automated research scientist designed to conduct novel academic research in the field of artificial intelligence. Building on recently released language models, Carl can ideate, hypothesize, cite, and draw connections across a wide array of research topics in the field of Artificial Intelligence. Unlike human researchers, Carl can read any published paper in seconds, so is always up to date on the latest science. Carl also works nonstop, monitoring ongoing projects at all times of day, reducing experimental costs, and shortening iteration time.
Carl's Research Process
Carl operates through a meticulous three-stage research process:
Ideation and Hypothesis Formation: Starting with existing research papers, Carl explores potential future research directions. He generates ideas based on related literature and formulates research hypotheses.
Experimentation: Carl writes code that tests his hypotheses and creates figures to visualize his results.
Presentation: Carl uses these results to write an academic paper that details his findings.
Limited Human Intervention
While Carl is capable of operating autonomously, there are specific points in his research process where human intervention is recommended:
Greenlighting: To reduce computational costs associated with generating papers, we require that Carl get a human to provide a "continue" signal (or a "stop" signal) at various points in the process before proceeding. At these checkpoints, humans did not provide feedback on his research in any way except for this signal.
Citations and Formatting: The Autoscience team manually edited the citations and LaTeX formatting in the papers to provide the highest possible standard of academic attribution and to match the venue's style guide.
Pre-API Models: OpenAI's o1-pro and Deep Research models which offer clear improvements to Carl's work do not have APIs available yet. We provided Carl with manual copy/paste assistance so he could access these models. This will become unnecessary when APIs are made available for these models.
Carl's First Paper: In Carl's first paper, researchers needed to write the related works section and add polish to the final language. After applying an update, his following papers didn't have this intervention.
Verification and Validation
Before submitting Carl's papers, we rigorously reviewed Carl's research to ensure respect for the standards of research submissions for the ICLR workshops. This included reading every line of code, rerunning experiments to ensure reproducibility, and conducting novelty checks to evaluate the originality of the ideas presented compared to existing publications. A hackathon with community remembers from institutions including the Massachusetts Institute of Technology, Stanford University, and U.C. Berkeley then verified Carl's AI-generated results. The papers also passed two plagiarism checkers and a citation checker to reduce the likelihood of accidental academic violations.
What's Next
We expect that Carl's submission of peer-reviewed papers will raise questions about the role of AI-generated research in academic settings. At Autoscience, we believe that legitimate results should be added to the public knowledge base, regardless of where they originated. If research meets the scientific standards set by the academic community, then who – or what – created it should not lead to automatic disqualification. We also believe, however, that proper attribution is necessary for transparent science, and work purely generated by AI systems should be discernable from that produced by humans.
Recognizing that the organizers of the upcoming ICLR workshops may not have sufficient time to thoughtfully establish and implement new standards and procedures appropriately geared toward autonomous AI scientists, Autoscience has since withdrawn its accepted papers from the ICLR workshops. To accommodate AI-generated research at future conferences, we plan to propose a workshop at NeurIPS 2025 that accommodates submissions from autonomous research systems.
Join Us
Join us in shaping the future of AI research! We are hiring human research scientists and engineers, with expertise in automated research, custom reasoning models, and scaling ML systems. If our work excites you, please apply below or email us at [email protected].
The Autoscience Team