Imagine a world where you give a program a video or highly rendered images of a location like Vegas or even the Everglades and instantly create an environment for you based off of these images and with your high-priced VR setup, it allows you to interact with it instantly.  You can walk through the environment using your VR headset.  Or even interact with a memory for your friend's birthday and you can relive it in 4k through your VR handset.  This is what the world of Gaussian Splatting allows you to interact with.  This technique in technical terms is a rasterization for reconstruction and reordering images.

    If you’re as curious as me, you're probably wondering where the term “Gaussian” in Gaussian splatting comes from. It comes from the last name of the famous mathematician Carl Friedrich Gauss, who originated the techniques of discrete probability distribution in the 1800s. His work laid the foundation for the way Gaussian splatting represents imaging using blurry clouds, rather than the well-defined triangles and precise pixels used in common 3D rendering techniques. But why do deep learning experts and AI artists herald this method?  Because it Depicts a 3D environment through millions of small particles, each embodying a 3D Gaussian. Every Gaussian is characterized by its position, orientation, scale, opacity, and color that adjusts with the viewpoint.  For streamlined rendering, these particles undergo a conversion process, transforming them into a 2D space called “splats” (this is where the “splatting part of Gaussian splats comes from)  followed by meticulous organization and sorting to optimize rendering performance. The use cases for Gaussian Splatting vary across creative disciplines and businesses.

Imagine being a DTC furniture brand and you take splats of all of your incoming product lines, so you take lidar pictures using your iPhone for all of your couches and chairs like technology from your phone on an app like Luma or Spline and create 3D figures that can be implemented into your website and be interacted with via your customer base and they can even see it in their Apple Vision Pro, if they want to get the true 3D Gaussian Splatting experience. 

I think of Gaussian Splatting in these three principles below

Structure from Motion: It is created using a point cloud which is derived from coordinates of 2D images. By itself, the point cloud looks pretty badass as well.

Transformation: These points are then converted into Gaussian representations and create a structure for which more detailed images will be built. Gaussian Splatiting is sorted by the depth and position of the images than it puts all the pixels together based on those positions.

Visualization: From that structure, a rendering process occurs via Differentiable Gaussian Rasterization which converts the Gaussian representations into the best possible image based of that rasterization which acts as a small neural network that knows how to arrange the pixels the best to create the ideal image.

This has been the most helpful way for me to break down the concepts and creations of Gaussian Splatting.  Before we get out of here, I want to show an example of how effective this technology can be, at showing the visual impact of real-world situations.  Below is a 3D Gaussian Splatting of the collapsed Francis Scott Key Bridge.  We now live in a world where we can not only watch the world's most harrowing problems and achievements 2 dimensionally on a television set but now we can experience these same problems and achievements 3 dimensionally in a VR headset.