ImageBind

ImageBind is an open-source AI model from Meta AI that is capable of binding information from six modalities into a single embedding without explicit supervision. While previous models have combined text, image/video, and audio data, ImageBind also includes depth (3D), thermal (infrared radiation), and inertial measurement units (IMU) that calculate motion and position. Meta states ImageBind is the first AI model to combine all these types of data. With these six modalities, ImageBind makes it possible to identify objects in a photo with their natural language name or description, determine how they will sound, their 3D shape, how warm or cold they are, and how they will move.

Meta AI introduced ImageBind on May 9, 2023, with a blog describing the model and a research paper titled "ImageBind: One Embedding Space To Bind Them All," going into more technical detail. As an open-source model, its code is available on GitHub.

In a demo of the model accompanying its release, Meta shows how ImageBind can do the following:

• Suggest audio clips for input images or videos
• Output images based on audio clips
• Provide image and audio clips based on a natural language input
• Offer related images for a combined audio/image input