The Methyl Isocyanate of Law: Supreme Court Quashes Tribunal Orders Based on AI-Generated Hallucinations
• The Verdict: The Supreme Court of India set aside orders from the NCLT and NCLAT that relied on non-existent, AI-generated case laws.
• The Analogy: Justice P.S. Narasimha compared "hallucinated" legal precedents to the Bhopal Gas Tragedy's methyl isocyanate—invisible and catastrophic.
• Human Oversight: The Court mandated a "zero-tolerance" policy for unverified AI citations, asserting that human reasoning must remain in absolute control.
• The Context: The case involved an Essel-linked insolvency dispute where six fake Supreme Court judgments were cited by the tribunal's own internal research.
• The Fallout: The Bar Council of India (BCI) has been directed to form an expert committee to define disciplinary consequences for "hallucination-led" advocacy.
The Supreme Court just drew a hard red line on generative AI in Indian courtrooms. In a ruling setting aside orders from the NCLT and NCLAT, the apex court didn't just reprimand a careless mistake—it formally recognized AI hallucinations as an existential threat to legal certainty.
The facts of the Essel-linked insolvency case are alarming. The NCLT relied on six Supreme Court judgments that simply do not exist. Because both sets of counsel denied submitting these citations, the inescapable conclusion is that the tribunal's own internal "AI-assisted" research fabricated them.
The Invisible Toxin
Justice P.S. Narasimha’s order, written for a bench including Justice Alok Aradhe, captures the stakes perfectly. He compared unverified AI outputs entering the legal record to methyl isocyanate—the gas responsible for the Bhopal tragedy. It is invisible, insidious, and by the time the system registers the poison, the damage is already done.
When an LLM hallucinates a binding precedent, and a judge accepts it without human verification, the adversarial system collapses. A court cannot arbitrate truth if its baseline reality is synthetic.
The Paradox of Progress
This judicial reckoning arrives just as the executive hits the accelerator. Days ago, MeitY announced plans to empanel 20 technology firms to integrate generative AI into government IT systems.
We are watching a real-time collision between the government's push to clear administrative backlogs via automation and the judiciary’s absolute demand for verified reasoning. The Supreme Court's "zero-tolerance" policy dictates that any decision relying even partially on fabricated AI citations is void ab initio.
Analysis: Beyond the Prompt
For BharatLens, the Court's directive to the Bar Council of India to frame disciplinary rules is the real headline. It lays the groundwork for accountability in digital legal practice.
The Court is strictly enforcing the scientific temper required in law. If an insolvency tribunal can accidentally invent Supreme Court cases, the implications for integrating AI into police surveillance, bail hearings, or criminal sentencing are terrifying. The human-in-the-loop is no longer a best practice—it is a mandatory firewall.
Sources
• The Times of India: SC nixes rulings by NCLT, NCLAT based on fake AI citations
• LiveLaw: Supreme Court sets aside NCLT judgment for using AI-hallucinated citations
• Hindustan Times: SC: AI-generated precedents ‘catastrophic’ for justice system
• Ministry of Electronics and Information Technology: Notice on AI-led modernization of Government systems
Comments ()