Apple unveils SHARP AI model, turning a 2D photo into a 3D scene in under a second

Apple has unveiled a new open-source model called SHARP that can generate photorealistic 3D scenes from a single 2D photo in under a second. The model’s code and weights have been published on GitHub.

Key points

• Uses 3D Gaussian Splatting technology for fast 3D scene reconstruction
• 25–34% improvement on the LPIPS metric and 21–43% on DISTS compared with previous state-of-the-art methods
• Very high speed: renders scenes at more than 100 frames per second
• High accuracy in reconstructing nearby views, but cannot generate parts of the scene not visible in the original image

Applications

• Augmented reality (AR) and virtual reality (VR)
• Headsets and spatial computing experiences