Summary
- OpenAI announced the release of GPT-4o, their newest model that is as smart as GPT4, but two times faster and has improved multi-modal capabilities with text, voice, and vision.
- Users who don’t want to pay for ChatGPT Pro will have access to many features that were once exclusive to the subscription, including GPT-4 and GPT-4V, generating web responses, being able to analyze uploaded files, and having access to the GPT Store.
- They announced releasing a ChatGPT desktop application for macOS, which will integrate into anything you’re doing on your computer. This includes interacting with the improved voice and audio models.
Takeaways from OpenAI’s Spring Release
May 2024 has been an eventful month for OpenAI.
They recently released their Model Spec, a document that specifies OpenAI’s approach to aligning AI with human preferences and ensuring safe, responsible AI. As AI becomes interwoven in our lives, AI companies like OpenAI and policy makers need to begin establishing this infrastructure.
And today, OpenAI announced several new features with ChatGPT and their other AI products that will be rolled out in the coming weeks. Here are some of the biggest takeaways from the Spring update.
For the Free Users
Most people who use ChatGPT have free access to GPT-3.5 Turbo, which is fast but not as accurate on more complex tasks compared to bigger models like GPT-4. Additionally, there were other perks that weren’t available to free users, such as:
- Vision capabilities: you can take upload photos from the web browser or use the ChatGPT app to take a photo and chat about the photos you take.
- Data analysis: you can upload a dataset, like a spreadsheet or JSON document, and you can analyze the data, create charts, and do things to a dataset that were previously only accessible to an Excel guru or a data analyst.
- Document reasoning: you could upload files for summarizing, adding additional context to a prompt, and other writing tasks.
- Accessing the GPT Store for Custom GPTs: OpenAI created a platform that allows users to access custom GPTs for specific tasks. For instance, OpenAI has DALL-E on the GPT store, which allows you to prompt the model to generate images using the ChatGPT interface. Other individuals and groups have contributed to the GPT store as well.
- Memory: You can ask ChatGPT to remember specific details so that it becomes better for your particular use-cases. For instance, you can specify specific pieces of information that it should consider (e.g. your profession), or how ChatGPT should stylize the output (e.g. bullet points for task lists).
Today, these features have been released to free users! Those who opt-in for the paid subscription will still have early access to new features and likely other future perks.
For the AI Power Users: ChatGPT Desktop
I would consider myself an AI power user. Sometimes, I don’t even Google stuff anymore, and I use ChatGPT or Claude to report out answers to my queries. And now, you will have the ability to integrate GPT-4 in your desktop work as well. To see it for yourself, you can play the video below at 01:56.
At this moment, there’s only so much we know about the app. It will only be available for macOS users and Paid subscribers in the beginning. But from the Spring Update Demos, ChatGPT Desktop will become more integrated with how you interact with your computer, allow you to do things like:
- Have voice conversations with ChatGPT from your desktop
- Take screenshots and ask it to do things with those screenshots
This is now my own personal speculation, but I think this is the beginning of OpenAI developing out AI agents that will be able to automatically perform tasks without us intervening. Agents would definitely be a huge step towards Artificial General Intelligence (AGI), but only time will tell.
OpenAI’s Latest Model: GPT-4o
Finally, the moment we’ve been waiting for: OpenAI released GPT-4o (for “omni”). And omni it is - it is currently one of the best models that can handle text, audio, and image.
Other perks with GPT-4o include:
- 2x faster than GPT-4 Turbo
- 50% cheaper in the API
- 5x higher rate limits that GPT-4 Turbo
- Support for 50 languages
The Spring Update demo speaks to how powerful GPT-4o is with its new audio, vision, and reasoning capabilities. If you’re interested in watching just the demo, it starts at 09:18.
There are several demonstrations for various tasks, including general interactions with the audio model, real-time language translation, interacting with real video streams, and data reasoning.
Conclusions
As of the time of this writing, OpenAI has already released GPT-4o to its ChatGPT paid subscribers, and it will be releasing all these cool features in the coming weeks. We hope you’re exciting about these exciting updates in AI, and we look forward to building cool things with these new models and tools.