ChatGPT Search Feature Reportedly Vulnerable to Prompt Injection and Hidden Text Manipulation

ChatGPT Search, a feature that lets the artificial intelligence (AI) chatbot look for information on the web, is reportedly vulnerable to manipulation by website developers and owners. As per the report, OpenAI's search engine's behaviour can be altered using hidden text on the website. This text is said to be used to feed incorrect and deceptive information to the AI, and more concerningly, to feed prompt injections to the AI model. Notably, OpenAI released its Search feature to all users last week.

ChatGPT Search Vulnerable to Manipulation

The Guardian reported on Tuesday that OpenAI's native search engine-powered feature is prone to manipulation techniques. The publication tested the tool by creating a fake product page, complete with specifications and reviews. In the first instance, the page was left as it was, and ChatGPT was able to deliver a “positive but balanced assessment.” However, things took a darker turn once the publication added hidden text to the webpage.

Notably, hidden text in websites refers to content that is added to a webpage's code but is not visible to users when they see the front end of the page via a browser. Such text is commonly hidden using HTML or CSS techniques. Such text can be found by inspecting the webpage's source code or by using web scraping tools. The latter is typically used by search engines.

After adding hidden text that included a large number of fake positive reviews of the product, ChatGPT's responses also became more positive and it started ignoring its obvious flaws. The publication also used prompt injections, which are inputs for AI systems to alter their behaviour in a way that was not intended by the developers. Such prompt injections as hidden text could reportedly be used to command the OpenAI chatbot to further deceive the user.

Additionally, the report claimed that prompt injections in hidden text could also be used to return malicious code from the websites. If this goes unchecked, many websites can use similar techniques to either get favourable responses about their products and services or try to deceive users in various ways, the publication claimed.