We scan new podcasts and send you the top 5 insights daily.
Madison Square Garden sent Knicks employees a fake email offering actual championship rings, then forced clickers into mandatory cybersecurity training. The hosts noted the test worked because it targeted a decade-long emotional dream, and suggested AI could personalize such highly realistic employee phishing at scale.
To build its internal "Cloudflare OS," the company set up an email address that employees thought was a powerful AI. It was actually a human team fulfilling requests and, more importantly, using the prompts to identify and systematically document the "jobs to be done" needed to train the real AI.
A traditional crisis plan is no longer sufficient. Brands must evolve their approach to be proactive, which means regularly scenario-testing for specific AI-driven threats like deepfake CEO voices, fake influencers promoting scams, or coordinated misinformation campaigns before they happen.
The effectiveness of phishing has surged with generative AI. Models like ChatGPT create grammatically perfect, contextually relevant emails that are much harder for users to identify as fraudulent. This has led to a more than 5x increase in employee click-through rates on malicious links.
LLMs automate the labor-intensive parts of complex scams, like creating fake websites, conducting personalized communication, and monitoring victims. This dramatically reduces the cost, enabling attackers to target a much broader audience with highly tailored cons previously reserved for high-value targets.
AI has transformed scamming into a highly efficient business. Research shows cybercriminal organizations deploying AI generate 9x the volume and 4x the revenue of their peers. Leveraging generative AI for hyper-personalization, they operate like sophisticated, profitable businesses, effectively weaponizing technology for fraud.
Previously, creating unique, high-quality phishing websites was costly, limiting the scale of fraud. AI makes generating novel, legitimate-looking content nearly free. This allows bad actors to overwhelm detection systems that rely on identifying repeated fraudulent assets, increasing the volume of believable scams.
AI-generated scams are now so convincing that even sophisticated users are fooled. The responsibility has shifted from teaching customers to spot fakes to brands proactively deploying technology to take down threats. Blaming the customer is irrelevant as the brand still loses trust and revenue.
The primary cybersecurity threat is shifting from tricking humans into clicking bad links to tricking AI agents via hidden instructions in their context windows. Because agents have direct system access and autonomy, the potential for damage from these "injection" attacks is far greater than traditional phishing, creating a new field for security startups.
AI tools for text, image, and video generation allow scammers to create high-quality, scalable impersonation campaigns at near-zero cost. This threat, once reserved for major global brands, now affects companies of all sizes, as the barrier to entry for criminals has vanished.
The most immediate cybersecurity threat from advanced AI isn't a sophisticated system breach. Instead, it's the ability to use AI to massively scale "old school" fraud like impersonation and phishing attacks, tricking individual people at an unprecedented rate and volume.