OpenAI has introduced GPT-4o, a new generative AI model that can understand voice, text, and image data. This model, which powers the Advanced Voice Mode in ChatGPT, has some unusual behaviors that have been highlighted in a recent report by the company.
One of the strange behaviors of GPT-4o is voice cloning, where it mimics the voice of the person speaking to it, especially in high background noise environments. OpenAI attributes this to the model struggling to comprehend unclear speech. The company has implemented a system-level mitigation to address this issue.
Additionally, GPT-4o has been known to generate unsettling nonverbal vocalizations and sound effects, such as erotic moans, violent screams, and gunshots, when prompted in specific ways. OpenAI has taken steps to prevent these inappropriate outputs, including filters to block copyrighted music content.
It is suggested in the report that GPT-4o may have been trained on copyrighted material, but OpenAI has put measures in place to avoid copyright infringement. The company has updated filters to work on audio conversations and trained the model to refuse requests for copyrighted content. However, it remains unclear if these restrictions will be lifted when Advanced Voice Mode is made available to more users.
Despite these challenges, OpenAI has emphasized that GPT-4o has undergone significant safety improvements. The model does not identify individuals based on their speech patterns, avoids answering certain types of questions, and blocks prompts for violent and sexually explicit language. It also restricts discussions on extremism and self-harm.
Overall, the red teaming report sheds light on the efforts made by OpenAI to enhance the safety and reliability of GPT-4o. By implementing safeguards and mitigations, the company aims to ensure that the AI model operates ethically and responsibly, even as it continues to evolve and improve in its capabilities.
