OpenAI recently released a document titled “GPT-4o System Card,” outlining the security measures and risk assessments conducted before the launch of GPT-4o, and pointing out that the model’s text-to-speech mode may lead to users developing emotional attachment.
In May of this year, OpenAI released the GPT-4o model, which is designed to closely mimic human speech. Prior to the launch of GPT-4o, conversing with the model in text-to-speech mode using ChatGPT required three separate models to process the conversation. These models were responsible for converting speech to text, processing the text and generating responses, and finally converting the generated text back to speech.
Through multiple conversions, important information may be lost, and it is not possible to directly observe tone, distinguish multiple speakers, or background noise, nor generate expressive voices to convey emotions. Unlike its predecessors, GPT-4o is capable of simultaneously processing inputs and outputs in three different modalities: text, visual, and audio. It no longer requires separate models to convert and process data.
As a result, GPT-4o is more effective at capturing and processing complex information, detecting tone, and expressing emotions. The GPT-4o System Card warns that such highly anthropomorphic voices and intonations may intensify users’ emotional reliance on AI, leading some users to develop a “human-like” trust and even impacting their social interactions with others.
During testing, OpenAI researchers found that some users exhibited signs of emotional attachment when interacting with the model. For example, using phrases like “this is our last day together” demonstrated a deep emotional connection to AI. However, according to Wired, Joaquin Quiñonero Candela, OpenAI’s Head of Applied Machine Learning, stated that the emotional effects displayed by GPT-4o could be positive in certain cases, such as assisting lonely individuals or those in need of practicing social interaction.
The overall risk rating of GPT-4o is classified as “medium” according to the GPT-4o System Card. This rating is determined based on four risk categories: “cybersecurity,” “biological threats,” “persuasiveness,” and “model’s autonomy.” The highest risk category, except for “persuasiveness,” is assessed as low risk.
The document explains that the persuasiveness of the text mode slightly exceeds the medium-risk standard. Researchers compared the content generated by AI with professionally written articles and found that GPT-4o’s persuasiveness did not surpass that of human-authored content. However, out of the 12 test cases conducted, three demonstrated that GPT-4o’s persuasiveness exceeded that of human-authored articles.
In other words, while GPT-4o’s persuasiveness is generally inferior to that of humans, it cannot be entirely ruled out that in certain specific situations, it could more effectively influence participants’ opinions.
OpenAI released these research findings in order to present risks more transparently to users, but some industry experts believe that there is still room for improvement. Lucie-Aimée Kaffee, an applied policy researcher at the AI open-source community platform Hugging Face, pointed out that the document does not provide detailed information about the model’s training data or the ownership of that data.
Additionally, Neil Thompson, a professor at MIT researching AI risk assessment, stated that internal reviews are only the first step in ensuring AI safety, and many risks only become apparent once AI is actually deployed.
Source:
Wired, The Verge, OpenAI