PhD Students in the AI Era: Blessing or Curse?

Opening

Dear reader, something pretty shocking happened in academia last week. ICML¹, the world’s largest machine learning conference, desk-rejected 497 papers in one sweep. The reason is unusual: it wasn’t a quality problem with the papers themselves — it was that the authors got caught secretly using AI while peer-reviewing other people’s papers.

They had explicitly agreed not to use AI — and used it anyway.

What makes this case interesting is how they got caught. The ICML organizing committee had embedded an invisible “watermark” into the review PDFs — invisible to human eyes. Hidden inside that watermark were instructions readable only by AI, and if an AI followed those instructions and inserted specific phrases into a review, that review got flagged. The phrases were drawn randomly, two per paper, from a dictionary of 170,000 phrases, making the odds of an accidental match less than 1 in 10 billion. This is actually a method I built myself a while back — being validated this way is gratifying, if also a little bittersweet…

But what struck me about this case was a more fundamental question than the detection technique itself. The fact that even researchers at the frontier of AI research depend on AI enough to break rules they agreed to — is that an individual ethics problem, or is something more structural at work?

Today I want to unpack this question across the whole research ecosystem, from PhD students to senior researchers.

Why ICML Researchers Broke the Rules

Let’s unpack the structure of the ICML case a bit more. ICML 2026 ran two policies on AI use. Policy A (conservative) banned AI use entirely during review, while Policy B (permissive) allowed AI for understanding papers and polishing reviews. Reviewers chose which policy they wanted to follow themselves. In other words, anyone who chose Policy A had personally promised not to use AI.

Here’s what happened. Among reviewers who chose Policy A, 506 were caught using AI, and 795 of their reviews (about 1% of all reviews) were struck. Of these, 51 reviewers had more than half of their submitted reviews written by AI, and they were stripped of their reviewer status entirely. Under ICML’s reciprocal review² policy, 497 papers (about 2% of all submissions) authored by rule-breaking reviewers were rejected.

One thing worth noting here: as the ICML committee itself acknowledged, this detection method only catches the most blatant violations — the kind where someone feeds the PDF of the paper they’re reviewing straight into AI and copy-pastes the output. The watermark’s existence was public knowledge for most of the review period, and anyone paying even a little attention could have avoided it. That 1% still got caught anyway suggests the real rate of AI use could be far higher. As I noted in an earlier blog post, that PDF watermark can be stripped out with a single pass of a Python script. And yet… people still submitted reviews like this to a conference as prestigious and international as ICML? It makes you wonder.

Indeed, in a 2025 survey by academic publisher Frontiers of roughly 1,600 researchers across 111 countries, 53% said they had used AI in peer review. Narrow that to early-career researchers (5 years or less of experience), and the figure jumps to 87%. But policy at many journals and conferences hasn’t caught up with this reality.