Texas AG v. Pieces Technologies: First State AG AI Healthcare Settlement

Summary

In the first state attorney general settlement involving deceptive marketing of AI, the Texas AG alleged Pieces Technologies made false and misleading statements about the accuracy and safety of its generative AI, including misrepresenting the product's hallucination rate. The tool summarizes patient conditions and drafts clinical notes and had been deployed at four or more major Texas hospitals processing real-time patient data.

Details

• Violation: Deceptive AI accuracy claims in healthcare settings
• Penalty: No monetary penalty — accuracy disclosure requirements
• Framework: Texas Deceptive Trade Practices Act
• Status: Settled (December 2024)

Key Takeaways

1. First state AG settlement for deceptive AI marketing in healthcare
2. AI hallucination rate claims must be accurate and substantiated
3. Healthcare AI tools deployed at hospitals face heightened scrutiny
4. State AGs can enforce AI accuracy without AI-specific legislation

Injunctive Relief

The settlement requires Pieces to accurately disclose its products' accuracy and limitations, and to ensure hospital staff understand the extent to which they should or should not rely on the AI's outputs, with heightened requirements on future marketing.