Integrity Manager: from detection to investigation

Imagine you are a Research Integrity Officer at a major publishing house. You have just discovered potential data manipulation in a breakthrough cancer paper that is about to be published. Every hour spent working out next steps is not an inconvenience; it is time in which compromised research can influence medical decisions. A single missed or delayed action can damage a journal's reputation built over decades, waste research funding, misdirect future work and erode public trust in science.

This is not theoretical. In 2020 The Lancet and the New England Journal of Medicine retracted a high-profile COVID-19 study that had already influenced global health policy; the WHO had paused clinical trials on its basis. Faster, clearer investigation is the difference between catching that and publishing it.

Our analysis found three gaps. Pattern blindness: users struggled to find meaning across 23 or more individual check failures presented as isolated items, so integrity officers spent more time parsing the list than investigating. Expert judgment undermined: the platform delivered verdicts ("FAILED 23 checks") rather than surfacing patterns for professional assessment, and in high-stakes work professionals reject AI that claims certainty. Workflow inefficiency: reviewing findings, grouping related issues, building an investigation path and deciding next steps all happened outside the product, so teams invented their own tracking systems.

Despite strong market interest, deals were not closing. Publishers needed AI that helped them recognise what to look for, then let them act on their own conclusions.

With backing from the CTO and the VP of Product, we assembled a small team: Leandro, a senior designer from the platform team, Thomas Fortmann, our staff user researcher, and Mădălina Pop, the Integrity product manager, as decider. We wrote everything down as How-Might-We questions and voted: help users recognise patterns across multiple checks; surface meaningful groupings instead of exhaustive lists; present findings as indicators for expert judgment, not verdicts; let professionals investigate patterns and act on their conclusions.

Discovery meant reading every piece of client feedback we had before anyone proposed a solution. Ideation started from the design backlog, then sketching sessions and dot-voting on ideas that gave users both context and a clear next action. By day four we were prototyping; Leandro and I took turns building and writing the usability scripts, then spent two days testing with six people: four internal experts in research integrity and two customer representatives, a Head of Research Integrity and Peer Review and a Research Integrity Manager from a publisher with more than $8M in annual revenue.

Five solutions came out of it, designed to work as one flow. Smart Insights combine related checks into a story: over-citation, duplicated references and self-citation together indicate possible citation manipulation. Check Acknowledgment lets a reviewer mark a flagged check as looked into. Activity Tracking records who did what on a report. Report Sharing brings colleagues into an investigation. Investigation Conclusion gives it an end point: the paper is marked acceptable or unacceptable, and that decision drives the publication pipeline.

What testing taught us: words matter most for power users, and copy in workspace pages and permissions decides whether they act; small actions like sharing are sensitive in integrity work and need careful access control; our clever grouping of checks with recommendations lost to simply showing all checks at once; and an analytics dashboard is only useful when every indicator can be clicked, collapsed and traced to manuscripts and journals.

After the sprint the work continued with just me and the Integrity team, from January to March 2024. The first priority was turning dashboards from data displays into investigation tools. Clickable bars reveal the related papers; filters work across organisational levels; every statistic has a direct path to the papers that need attention.

Publishers investigate at different altitudes. At journal level they look for patterns across all volumes; at volume level for issues within a publication period; at paper level for the specifics of one submission. A plagiarism indicator at journal level now opens every affected paper across every volume of that journal, so broad patterns and specific cases live on one path rather than in separate reports.

From April to July 2024 we returned to the subtlest sprint finding. Some testers thought "acknowledge" meant marking a false positive, others that they had merely seen the issue, others that they were validating the result. In research integrity that ambiguity is a problem in itself, so we redesigned the action system rather than the label.

When a check fails, a reviewer can Mark as Acceptable after investigating; when a check passes but something looks wrong, they can Mark as Issue. Both can be reverted, every decision requires a reason, and each one is written to the activity log with who, when and why. A small feedback icon on the check shows that a decision has been taken, and opens the log.

The principles we wrote down for the product: indicators, not verdicts; patterns, not lists; investigation support, not just detection; transparency for trust.

Engagement with integrity reports rose once they could be acted on inside the product. The Association of Learned and Professional Society Publishers shortlisted Integrity Manager for its 2024 Award for Innovation in Publishing, and the product later contributed to Morressier's acquisition by Molecular Connections.