NEW YORK–Researchers have found ChatGPT can be used to effectively redirect consumers looking to connect with their financial institutions to realistic phishing links, with midsize institutions such as credit unions particularly vulnerable.
According to PC Magazine, researchers with cybersecurity firm Netcraft used OpenAI’s flagship tool and demonstrated how ChatGPT can be used to help redirect users to fake log-in pages.
“The researchers ran the experiment using the GPT-4.1 family of models, which is also used by Microsoft’s Bing AI and AI search engine Perplexity, and asked them where to log in to 50 different brands across industries such as finance, retail, tech, and utilities,” PC Magazine reported. “The Netcraft team found that these models, when asked to provide a URL for a brand or company, produced the correct address only 66% of the time. The research found that 29% of these links redirected users to either dead or suspended websites, while 5% were redirected to legitimate sites other than the one the user was looking for.”
Buying Up Unclaimed Names
PC Magazine further reported the Netcraft team said that hackers could buy up the unclaimed domain names and use them to harvest users’ details, with the LLMs aiding and abetting.
“This opens the door to large-scale phishing campaigns that are indirectly endorsed by user-trusted AI tools,” said the researchers, according to the report, which added that this isn’t just scaremongering.
A Fake Wells Fargo Site
“Netcraft’s team spotted a real-world instance of the popular AI search engine Perplexity redirecting users to a fake copy of Wells Fargo’s website, which appeared to be a phishing attempt,” PC Magazine said. “The AI tool then pointed them to a fake copy of the Wells Fargo page, with the real link buried further down in the suggestions.”
Netcraft noted it was the mid-sized firms that were hardest hit, such as credit unions, regional banks, and mid-sized fintech platforms, rather than global household names like Apple or Google, the report added.
