Chaoqun Ni and Ian Hutchins are developing data-driven tools to understand and improve the way science moves forward in the age of AI.
By Thomas Jilk
Associate Professor Chaoqun Ni and Assistant Professor Ian Hutchins came to UW–Madison in summer 2020, at a time when COVID-19 put scientific research firmly in the public spotlight. But science had been in Ni and Hutchins’s spotlight for years already, as the two researchers separately began asking similar data-centric questions about “how scientists behave, how they can do better, and how funding can be better allocated,” Ni said.
Today, as co-principal investigators in the Information School’s Metascience Research Lab, Ni and Hutchins, along with their PhD students, are pushing frontiers in a field known as “science of science.” Often using massive data sets and machine learning methods, the group is exploring novel approaches for predicting scientific breakthroughs, identifying AI-generated text in academic research, and bolstering research integrity through AI-powered citation verification.
The lab’s work offers both a data‑driven view into the machinery of modern scholarship and a valuable resource for science policy makers determining how to fund projects, create new standards and regulations, and establish priorities for scientific research. Their work embodies the iSchool’s commitment to harnessing the power of data and technology not as ends in themselves, but as tools for investigating and improving society.
Two paths converge
Ni and Hutchins joined the iSchool after independently beginning to follow threads that would eventually lead them into the same emerging field. For Ni, the shift began in her doctoral training at Indiana University Bloomington, when she realized that using cutting-edge data science, “there is way more that I can do and more I can ask from the bibliometric data than what a lot of traditional bibliometricians can do,” she said. “The wave of big data slowly drove me into large‑scale, data‑based projects,” Ni said.
Meanwhile, Hutchins’s path was shaped by early encounters with the political and policy decisions around research funding. He was in the Neuroscience Training Program at UW–Madison, where he watched federal restrictions on embryonic stem cell research ripple through labs around him. “I became aware of science policy as this powerful force affecting everybody’s lives, and I wanted to help influence that in a positive direction,” he said. That realization eventually led him to the National Instititues of Health (NIH), on a team dedicated to using data to guide scientific decision making.
By the time Ni and Hutchins joined the iSchool, their research had begun to converge on big, thorny questions, such as how to make science more equitable and efficient, how to measure the impact of research, and how to predict the next world-changing scientific discovery.
How to see a breakthrough coming
In a yearslong project in collaboration with NIH, Hutchins and PhD student Salsabil Arabi are pursuing what the team calls “an elusive goal”: identifying the conditions that create new scientific breakthroughs. The effort could “accelerate the pace of discovery and improve the efficiency of research investments,” the researchers wrote in a recent preprint paper building on nearly a decade of work.
Using AI-driven methods to track “how fields grow, split, and converge over time,” they traced how subfields of science emerge and interact, sometimes slowly, sometimes in sudden bursts. Looking retrospectively at areas where breakthroughs occurred, Hutchins explained, “We can see these groups of papers cohering into subfields and splitting off into different topic areas that seem to be working toward different goals,” until something causes them to converge.
The CRISPR gene-editing revolution in the early 2010s offers an example, as long‑standing gene‑editing approaches came together with a new, more effective technology, opening new doors for applications of well-known biological principles. Then, Hutchins said, “You have this explosion in gene‑editing research, and you can see that through the dynamics of these topic clusters when you trace them over time.”
The goal is to help institutions understand where transformative work is happening and how to support it. As the researchers write, “Our findings illustrate how a better understanding of the scientific process may lead to greater scientific returns.”
A new tool to check citation accuracy
While Hutchins’s work maps the evolution of ideas, Ni’s recent projects address the challenge of scientific integrity. Ni and PhD student Xiang Zheng helped create a first-of-its-kind AI‑powered tool, BibAgent, designed to verify whether in-text citations in a draft of an academic paper accurately reflect what the cited paper actually says, a task both critical to research integrity and nearly impossible to do manually at scale.
The problem of researchers citing sources unrelated to the content of their papers turned out to be more widespread than Ni initially thought. “I started from a research‑integrity angle and realized how bad it is,” she said. But actually determining the scale of the problem was complicated by paywalls, inconsistent metadata, and even “citation cartels,” where authors can buy citations to inflate their impact.
BibAgent uses multimodal AI models to compare citation claims against the content of the cited paper, even when only abstracts or metadata are available. It is driving toward citation fidelity at scale, making peer-review processes more effective. Early tests of BibAgent show exceptional accuracy when full text is accessible, and the tool has drawn immediate interest, Ni said. “Every time I talk to people about this tool, they say, ‘I want to use it. I want to pay money.’”
‘Fully embedded’: AI in science
The Metascience Research Lab also has a firsthand view of how AI is transforming science, both through their empirical analysis and their day-to-day work. In a recent study, Ni and co-authors used statistical models to estimate the proportion of AI‑generated sentences in academic papers before and after the release of ChatGPT. They found AI use increased across the board, but especially among scholars from non‑English‑speaking countries and lower‑resourced institutions.
This shows that for many researchers, AI tools offer a way to overcome long‑standing language barriers, Ni said. “Speaking English is one thing, but writing is something else entirely,” she said. “With large language models, this may no longer be a problem for many of those scientists.”
However, both Ni and Hutchins emphasize the importance of responsible use, for researchers and students alike. Whether the goal is forecasting the next scientific breakthrough or ensuring equitable access to research tools, the same principle applies: AI is only as valuable as the understanding behind it.
“AI is going to be fully embedded in every stage of research. It’s inevitable.”
Chaoqun Ni
That makes it more vital than ever for students and researchers to understand and justify the AI-related choices they make, Ni said. Hutchins takes a similar view: “AI is another tool in the toolbox. The important thing is that we understand what the code is doing,” he said.
In other words, AI is just one of the forces shaping modern scholarship. By studying the data underlying the vast scientific enterprise, the Metascience Research Lab is painting a clearer picture of how science actually works, and how to improve it for the future.
Visit the Metascience Research Lab’s website.
Learn more about the range of research areas faculty study at the Information School.