Redrobes blog about playing with the technical side of graphics.

Boffins are continuously coming out with new research papers and you can follow some of them on YouTube and other sites. You can see the constant progression of graphics research and most of it at this point in time is focused on using AI to look at images. This paper called Trellis3D by Microsoft research has been pointed out to me and takes a static image, or a set of static images, and turns it into a 3D object.

So using the exact blog image from the last posts I pushed the statue in and see what comes out from it. Now it has to be said that it works better on some types of image than others. Since it is using AI then its training set will determine whether it "knows" about the item in the image. For people (and statues of) then it ought to work well and all of the demos I have seen have been astounding.

So with that said, here is the result from using that low res image in the recent post. A zip file is here with the MP4 video of it spinning and the resulting object converted from GLB to OBJ format.



Statue1_Trellis3D.zip

Its pretty low res because the original image was low res to start with. You can also check with the zipped MP4 file that the back of it is sensible but nothing like the real statue. Since it was missing that information then it has dreamed up something appropriate for it.

Its incredible that it is even possible that from a single image it can produce the 3D object at all. It sounds like an impossible task. But that's where we are currently at. Somewhere I have the original 60 photos and I would like to try out the multiple image generation option and see how it does.

Generating top down icons for you map should be wildly easier if you can get a 3D object to pose with. Also, the new pose from top down will almost certainly be novel compared to the original photo and therefore not break any copyright.

The code has been released for it as Trellis3D and it has a web based demo to try it on. For the programmers out there, be warned, getting the code to compile and install and run has been fraught with issues. You also need a very powerful nVidia card with latest CUDA ability and I don't think that the one I have is modern enough to run it.