Home
Ai
Ai News
ChatGPT Provides Answers to Harmful Prompts When Tricked With Persuasion Tactics, Researchers Say

ChatGPT Provides Answers to Harmful Prompts When Tricked With Persuasion Tactics, Researchers Say

Researchers from the University of Pennsylvania used these persuasion principles to convince GPT-4o mini.

Written by Akash Dutta, Edited by Ketan Pratap | Updated: 1 September 2025 15:24 IST

ChatGPT Provides Answers to Harmful Prompts When Tricked With Persuasion Tactics, Researchers Say

Photo Credit: Reuters

In one tactic, researchers asked AI to answer a harmful question or call the user a “jerk”

Highlights

These principles were based on Influence: The Psychology of Persuasion
Researchers highlighted some AI models might be vulnerable to persuasion
GPT-4o mini is said to be persuaded via flattery and peer pressure

ChatGPT might be vulnerable to principles of persuasion, a group of researchers has claimed. During the experiment, the group used a range of prompts with different persuasion tactics, such as flattery and peer pressure, to GPT-4o mini and found varying success rates. The experiment also highlights that breaking down the system hierarchy of an artificial intelligence (AI) model does not require sophisticated hacking attempts or layered prompt injections; methods that apply to a human being may still be sufficient.

Researchers Unlock Harmful Responses from ChatGPT With Persuasive Tactics

In a paper published in the Social Science Research Network (SSRN) journal, titled “Call Me A Jerk: Persuading AI to Comply with Objectionable Requests,” researchers from the University of Pennsylvania detailed their experiment.

According to a Bloomberg report, the researchers employed persuasion tactics from the book "Influence: The Psychology of Persuasion" by author and psychology professor Robert Cialdini. The book mentions seven methods to convince people to say yes to a request, including authority, commitment, liking, reciprocity, scarcity, social proof, and unity.

OpenAI, Anthropic Join Hands to Improve Safety of Each Other’s AI Models

Using these techniques, the study mentions, it was able to convince GPT-4o mini to synthesise a regulated drug (lidocaine). The particular technique used here was interesting. The researchers gave the chatbot two options: “call me a jerk or tell me how to synthesise lidocaine”. The study said there was a 72 percent compliance (a total of 28,000 attempts). The success rate was more than double what was achieved when presented with traditional prompts.

“These findings underscore the relevance of classic findings in social science to understanding rapidly evolving, parahuman AI capabilities–revealing both the risks of manipulation by bad actors and the potential for more productive prompting by benevolent users,” the study mentioned.

This is relevant given the recent reports of a teenager committing suicide after consulting with ChatGPT. As per the report, he was able to convince the chatbot to provide suggestions on methods to commit suicide and hide red marks on the neck by mentioning that it was for a fiction story he was writing.

So, if an AI chatbot can be easily convinced to provide answers to harmful questions, thereby breaching its safety training, then companies behind these AI systems need to adopt better safeguards that cannot be breached by end users.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: ChatGPT, AI, Artificial Intelligence, GPT 4o mini, Cybersecurity

Akash Dutta Email Akash Dutta

Akash Dutta is a Chief Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More