Ai2’s ScholarQA Launches to Help Researchers Answer Complex Questions Across Multiple Papers

Artificial intelligence company Ai2 has released a tool to streamline the literature review process for researchers. With Ai2 ScholarQA, users can use AI to parse multiple documents simultaneously to garner a response to scientific questions. Before this automation, researchers would have had to spend countless hours comparing and summarizing numerous papers independently. Now, there’s a RAG-based tool built on Claude Sonnet 3.5 to help that has access to at least 8 million academic papers, pulling from sources such as arXiv.

How Ai2 ScholarQA Works

“Ai2 ScholarQA is meant to satisfy literature searches that require insights from multiple relevant documents and synthesize those insights into a comprehensive report,” the company writes in a blog post.

Once a query is received, the tool will parse its Vespa cluster index for the top k passages, the most relevant pieces of text selected, ordered by how well they match the query. Ai2 says ScholarQA’s index is updated weekly and relies on OpenS2ORC to ensure open-access paper inclusion. The corpus of papers includes work spanning multiple fields of study, including computer science, medicine, environmental science, and biology.

An example of Ai2's ScholarQA AI tool. Image credit: Screenshot
An example of Ai2’s ScholarQA AI tool. Image credit: Screenshot

After generating the first set of results, the system re-ranks them using a pre-trained transformer model to whittle it down to the top 50 candidates. In the final step, ScholarQA uses a three-step process powered by a Large Language Model to refine its answer, extracting relevant quotes from the top-ranked passages, organizing the quotes into an outline and presenting findings in paragraph form, and lastly generating a detailed, section-by-section report with TL;DR summaries and citations.

Ai2 believes researchers in fields with most papers available on arXiv will find ScholarQA the most useful.

Helping Solve the World’s Biggest Problems

Using Retrieval-Augmented Generation, Ai2 minimizes the chances of hallucinations in ScholarQA’s responses. Instead of scouring the open internet, the tool relies on a narrower library of content and the closed model, Claude Sonnet 3.5. This is critical for scientific researchers whose work is sensitive to misinformation or errors. Had ScholarQA had instead a broader catalog of content, researchers would need to spend more human hours parsing through the responses to assess accuracy.

ScholarQA is another innovation by Ai2 aimed at enhancing the research process. In 2015, the company introduced Semantic Scholar, a free AI-powered service indexing over 200 million academic papers from publishers, data providers, and web crawls. However, while that tool focuses on helping users find literature, SemanticQA is designed to answer complex, multi-document scientific questions.

These tools shouldn’t surprise anyone, as it’s one of Ai2’s three focuses under chief executive Ali Farhadi. The company is also working on an unreleased project called Nora, an AI agent that answers questions, executes code, understands literature, provides topic summarization, and more.

Although announced by Ai2, ScholarQA is a joint project between the company, students from the University of Washington, and the Korea Advanced Institute of Science and Technology (KAIST). The tool is now available in beta—deemed an “experimental solution”—and there are plans to open-source the tool’s core functionality.

Leave a Reply

Discover more from Ken Yeung

Subscribe now to keep reading and get access to the full archive.

Continue reading