“You’re going to start seeing a lot more 360 degree video on your News Feed,” Facebook CTO Mike Schroepfer told Popular Science last week at the Future of AI conference at NYU.
And he was right.
Today, Mark Zuckerberg and Facebook’s engineers released information about how they’re making 360 degree video up to 80 percent smaller than the original file, to make streaming the content easier on computers.
The answer lies in geometry. Facebook’s engineers explain that instead of thinking about 360 degree video as a sphere, it’s better to think of the final product as a cube, a shape with 6 equal sides. That equality means less inherent distortion than a sphere, which is geometrically dissimilar than the sensor the image is being captured on. This is great for viewing on a flat screen, but not in virtual reality.
For VR, videos uploaded to Facebook start as a sphere. But Facebook’s software deals with spherical video by putting it into a pyramid, which as a tetrahedron can represent 3 dimensions. Then the software “unfolds” the pyramid, and crops it, making it into one part of the cube. This is inherently a visual process, so let this GIF make it easier to understand.
Facebook makes 30 viewports, or angles to view, in 5 different resolutions, meaning there’s 150 versions of each video on Facebook’s servers that are dynamically streamed to a viewer in virtual reality. To manage data, Facebook splits video in one second segments that are streamed based on the orientation of the user’s head.
They’re hoping on using machine learning to make this process smarter in the future, but for now are relying on hand-coded software.
Schroepfer did say that 360 degree video will still require…