- ChatGPT can’t tell if a site was hacked, expired, or repurposed for casino spam
- AI-generated answers may seem reliable, even when they cite completely hijacked and fake sources
- Expired charity domains are reborn as gambling sites and still pass as trustworthy AI sources
ChatGPT is quickly becoming a go-to source for people seeking recommendations, from online services to local businesses, but new evidence suggests its AI-generated suggestions may not always be grounded in trustworthy sources.
In fact, some are being drawn from websites that have either been hacked or whose domains have expired and been repurposed, often to promote online casinos and gambling platforms.
Over the past several months, James Brockbank, managing director and founder at Digitaloft, has been documenting how widespread the problem has become, uncovering examples of ChatGPT citing content from sites that have clearly been manipulated.
Exploiting gaps in AI source validation
In one instance, a functioning legal practice’s website, run by attorney Veronica T. Barton, had pages recommending UK casinos buried within it.
“Their site has been hacked and this page added,” Brockbank noted after reviewing the evidence.
In another case, a site once affiliated with a United Nations youth coalition had been transformed into a platform pushing “casinos not on GamStop.”
Although the listicle it hosted contained only one external link, it led to yet another repurposed domain.
The pattern continued with expired domains, including one that had belonged to a now-defunct arts charity previously linked by the BBC, CNN, and Bloomberg.
That domain, now pushing gambling content, was cited by ChatGPT in response to a query about no-deposit casinos.
These tactics exploit weaknesses in how ChatGPT selects and cites sources, as unlike traditional search engines, the model lacks mechanisms for verifying the legitimacy of a site’s ownership or editorial intent.
As a result, content injected onto compromised websites can surface in its answers without any obvious red flags to the user.
ChatGPT appears to favor recent content and still attributes authority based on legacy domain reputation, even when the domain’s content has no continuity with its past – which opens the door for bad actors to manipulate visibility through means that have little to do with credibility.
The bottom line is that users turning to ChatGPT for recommendations should not assume that every answer is backed by a credible source.
A quick check of the cited site’s authority, its history, ownership, and relevance can go a long way in avoiding misleading or harmful suggestions.
