What is the Potential of ChatGPT’s New Image Analysis Features?

1
268
open ai chatgpt
Image by frimufilms on Freepik

OpenAI has unveiled a notable update to its ChatGPT application for both Apple and Android platforms, bringing in two exciting features to its users: audio responses for a more interactive chat experience, and image analysis capabilities. The latter echoes the functionalities provided by Google’s Bard chatbot, albeit at no cost.

Upon a thorough examination of ChatGPT, it’s intriguing to witness its evolving capabilities. The previously introduced web browsing beta feature in ChatGPT Plus was impressive, yet it also raised concerns regarding its impact on online professional writers among other implications. Similarly, the new image analysis tool, though reserved for OpenAI subscribers, instills a blend of anticipation and caution.

And you might be wondering what is image analysis. It refers to the process of extracting meaningful information from images through the use of specialized computational and mathematical techniques.

Though we have yet to delve into the audio feature, we have had a chance to test the image analysis capability, which is set to roll out soon. Below, we made a guide on how to make use of the forthcoming image analysis feature in ChatGPT, alongside some tips to ensure a smooth experience.

Accessing ChatGPT’s Image Features

The anticipated update is slated for release before the year-end, however, the exact availability of the image and speech features to the public remains unclear. As typical with OpenAI updates, paying subscribers are expected to enjoy priority access. In ChatGPT’s mobile app, three alternatives exist for uploading images:

  1. Tap the camera icon located to the left of the message bar, capture a new photo with your smartphone, and then use your finger to circle the area you’d like the chatbot to analyze before uploading.
  2. Upload images from your device’s storage.
  3. For desktop users, simply upload stored images from your computer.

While video uploads aren’t supported at the moment, you can submit multiple images in a single request.

Tips for Engaging with the New AI Tools

This isn’t the inaugural instance of ‘computer vision’ being made available to the public, yet the seamless integration with an advanced artificial intelligence lends a unique and potentially transformative touch to the whole experience.

A word of caution: avoid uploading personal or sensitive images when testing the image feature on ChatGPT.

taking pictures with a phone
Image by Freepik

To exert more control over how long OpenAI retains your data and interactions, navigate to Settings, select Data Controls, and toggle off Chat History & Training. With this disabled, your data will be deleted after a month. Ensure you repeat this procedure on every browser you use to access ChatGPT, both on PC and mobile.

Our experimentation yielded the best results when we uploaded clear, well-lit photos. ChatGPT managed to accurately identify numerous items within the room, such as an orchid plant and various international currencies, along with a charging cable.

However, it’s crucial to note that the tool isn’t foolproof. ChatGPT can misidentify medication and certain artist pieces. While it can fail to discern the artist or location of a random mural photo, it can successfully identify several famous landmarks from photos, like the Eiffel tower and the Statue of Liberty.

Though there may be limitations, the visual recognition feature of ChatGPT adds an enriching layer to the user experience, especially for those exploring new cities or locales. The potential for an enhanced interactive exploration through ChatGPT’s visual tool is something worth looking forward to.

A key restriction placed by OpenAI on this new feature is the limitation on the chatbot’s ability to answer queries that pertain to identifying individuals. To uphold user privacy and safety regulations, the identification of actual individuals from images, irrespective of their public profile, is restricted. Although the chatbot cannot outright refuse to respond to inappropriate content, it can shy away from providing detailed descriptions of adult performers, opting instead to mention their tattoos.

identification of individuals
Image by Freepik

It’s noteworthy that during a conversation with an earlier version of ChatGPT’s image recognition functionality, we observed a bypass of some of OpenAI’s imposed safety measures. Initially, the chatbot faltered in recognizing a meme featuring John Cena. Following that, ChatGPT can misinterpret a picture of certain celebrities, however upon being questioned, the chatbot corrected its response.

If the imposed restrictions are ever lifted, whether via a compromised version of ChatGPT or a future open-source model, the privacy implications could be rather concerning. Imagine a scenario where every image of you that you took on your newest iPhone, once uploaded online, could easily be linked to your identity with a few clicks.

Or a situation where someone captures a photo of you in public without your consent and instantly locates your LinkedIn profile? The introduction of these new image recognition features could exacerbate the risks of harassment and stalking, especially towards women and other vulnerable groups, unless sufficient privacy safeguards remain firmly in place.

 

 

1 COMMENT

Comments are closed.