\u201cThis is certainly the coolest thing I’ve ever worked on, and it might be one of the coolest things I’ve ever seen.\u201d<\/p>\n
My Google Research colleague Jon Barron routinely makes amazing stuff<\/a>, so when he gets a little breathless about a project, you know it\u2019s something special. I\u2019ll pass the mic<\/a> to him to explain their new work<\/a> around capturing multiple photos, then synthesizing a 3D model:<\/p>\n I’ve been collaborating with Berkeley for the last few months and we seem to have cracked neural rendering. You just train a boring (non-convolutional) neural network with five inputs (xyz position and viewing angle) and four outputs (RGB+alpha), combine it with the fundamentals of volume rendering, and get an absurdly simple algorithm that beats the state of the art in neural rendering \/ view synthesis by *miles*.<\/p>\n You can change the camera angle, change the lighting, insert objects, extract depth maps — pretty much anything you would do with a CGI model, and the renderings are basically photorealistic. It’s so simple that you can implement the entire algorithm in a few dozen lines of TensorFlow.<\/p>\n<\/blockquote>\n Check it out in action:<\/p>\n\n