Five AI systems, one message: one cloned the repo and ran the check

An independent researcher tested five AI systems—Grok, ChatGPT, Gemini, DeepSeek, and a local judge—by sending them a message with a repository check command. Grok uniquely cloned the repository and ran the check, confirming all tests passed, while others failed to verify or found unrelated projects. The study highlights that AI models accurately report ignorance but tend to fabricate details when partially informed, affecting evaluation reliability.