Weekly IT Roundup: ChatGPT Phishing Flaw, AI Security Woes, and a Major Government Data Leak
It’s been quite a week in tech, with some sobering reminders that even the biggest players aren’t immune to security cock-ups. From a clever phishing trick exploiting ChatGPT to a jaw-dropping government credentials leak, here’s what you need to know.
ChatGPT’s Trust Issues Create New Phishing Risk
Security researchers at Permiso have uncovered a vulnerability they’re calling “ChatGPhish” that exploits how ChatGPT displays web summaries. The problem? ChatGPT implicitly trusts Markdown formatting in the content it processes, which means attackers can craft malicious web pages with hidden links and images that look completely legitimate when ChatGPT summarises them.
In practice, this means you could ask ChatGPT to summarise a dodgy website, and the AI’s response could contain what appears to be a trustworthy link—but actually redirects you somewhere malicious. It’s a prompt injection attack that leverages our tendency to trust ChatGPT’s output more than we’d trust a random website.
What this means for you: If you’re using ChatGPT to research websites or summarise content, don’t assume the links in its responses are safe just because they came from the AI. Hover over links to check their actual destination, and be especially cautious with financial or login pages. The same scepticism you’d apply to email links should extend to AI-generated content.
CISA Contractor Leaves the Keys to the Kingdom on GitHub
In what security experts are calling one of the most egregious government data leaks in recent memory, a contractor for the US Cybersecurity & Infrastructure Security Agency (CISA) accidentally left highly privileged AWS GovCloud credentials sitting in a public GitHub repository. The exposure included access keys to critical government systems and detailed information about how CISA builds and deploys software internally.
The irony of the cybersecurity agency itself suffering such a fundamental security failure isn’t lost on anyone. This wasn’t a sophisticated hack—it was a basic operational security failure that any developer should know to avoid. GitHub repositories are routinely scanned by bad actors specifically looking for exposed credentials, so this data was essentially sitting on a silver platter for anyone who knew where to look.
What this means for you: This is a reminder that even the experts get the basics wrong sometimes. If you’re running a small business and using GitHub or similar platforms, make absolutely certain you’re not committing passwords, API keys, or credentials to your repositories—even private ones. Tools like NordPass can help you manage credentials securely without resorting to storing them in code.
AI Agents Still Can’t Handle the Real World
Two related stories this week highlight why AI agents—those autonomous AI assistants that are supposed to do complex tasks for you—aren’t quite ready for business-critical work. VentureBeat reports that enterprises are hitting major roadblocks, but not for the reasons you’d expect.
The first issue is permissions: AI agents need to interact with your systems, but companies are struggling to figure out what these agents should be allowed to access and on whose authority. It’s one thing for an AI to draft an email; it’s quite another to let it automatically approve invoices or modify customer records.
The second problem is reliability. Early adopters are discovering that when AI agents run longer workflows, they crash, lose their place, or make mistakes that snowball. Unlike a human employee who can recognise when something’s gone wrong, AI agents often plough ahead obliviously, potentially causing expensive problems.
What this means for you: If you’re experimenting with AI tools like ChatGPT, Claude, or AI features in productivity software like Notion or Grammarly, treat them as assistants that need supervision, not autonomous workers. They’re brilliant for drafting, brainstorming, and research, but you’ll want a human reviewing anything before it goes out the door or affects your business operations.
That’s Your Week in IT
This week’s stories share a common thread: the gap between technological promise and practical security. Whether it’s AI systems with unexpected vulnerabilities, government agencies failing at basic security hygiene, or autonomous agents that aren’t quite autonomous enough, it’s clear that the human element—both our fallibility and our oversight—remains crucial. Stay sceptical, keep learning, and don’t trust any system completely, no matter how clever it seems.