Some DEM processing help, please

[Midgardsormr]

I am working on my visual effects portfolio project, for which I need several fairly high-detail models of the Nazca plateau in Peru. I have a 16-bit tiff DEM of the plateau at a resolution of 821 x 523. So far, my efforts at using this heightfield as a displacement map in Maya have given me poor results. The geometry I am getting is adequate for my high-altitude and satellite shots, but I'm going to need more resolution in order to get the closer shots. What I need, then, is a way to increase the resolution of the DEM and add some plausible-looking noise without disturbing the gross features.

I haven't dealt with DEMs much, so I do not have a workflow established to deal with this problem. I opened up Wilbur to see what it could do for me, but it won't import a Tiff, and I am not sure how to convert my data to a format it can use. Is Wilbur the best tool for this job? Are there other options I should look at? I am not unwilling to spend some money on this project, but probably not more than US$200.

[Redrobes]

Maya is a vector mesh app and you would need a LOT of mesh triangles to do that job. You could use a mesh of 1x1 and then use a displacement map. Now sometimes the displacement map affects just the vertices of the mesh and other times on higher end stuff it does a dynamic mesh subdivision. Given that it dont come much more high end than Maya then I would guess that it would be able to do it. Check if it has the subdivide option. If it cant then you are always going get stuffed by res with a 3D app.

Next up. Tiff is a very standard file format for GIS data but no so much in CGI. I'm not sure what a normal format would be but I have to use IFF which is pretty uncommon. Personally I think a 16bit PNG should be ok for terrain height.

For getting resolution in X-Y you can chop down the terrain height field and use that with Maya so that it diverts more of the image to vertices or use a DEM processing tool. I suggest latter. Now which one ? There are many but if you have never used a paint package would you buy PS ? I think you should try and get into Wilbur for a bit at a min cos its free and very capable. Once you know its limitations then you can decide what might be needed to cover for that. Thats like any bit of software. I think Wilbur is very capable of doing what your asking for.

Wilbur can add noise and resample height fields. Now whether by adding noise and blowing up a section would produce a realistic DEM of that area is another thing. That's where all the programming effort has been going for some time. Su-Liam has some experience of adding noise to DEMs and knows about different noise functions and whatever to get what he needs. Personally I only deal with Perlin noise and whittle DEMs away with fluid flows. I don't think you can get that all that much mileage out of *just* using noise but I repeat that's a very humble personal opinion. If you have started with a real DEM and want a little extra from it then you will be able to go there ok I think.

[Midgardsormr]

Yeah, my first displacement resulted in over 2 million tri's, but I was able to simplify it down to around 400,000 or so, which was reasonably light enough to deal with. Getting resolution out of Maya isn't a problem; mostly what I'm concerned with is increasing the resolution of the DEM so that I don't have to try to model a larger scale by hand, which won't look convincing. I was thinking I might dive into Bryce or Vue for the 3d, but I've only poked around in each of those for a few minutes a long time ago, so I'll have an additional learning challenge ahead of me.

Another problem I anticipate is that erosion isn't going to get me where I need to go in this particular case, since the plateau is shielded from the wind and receives less than an inch of rainfall per year. There's just not much going on, which is why the Nazca built their lines there to begin with. I'm hoping su_liam and Waldronate will have some input there.

As an update, I have a 16-bit PNG into Wilbur now, and I'm starting to play around. I am noticing a couple of areas where Wilbur seems to assume a square section, and I am wondering if I should recrop my DEM?

[waldronate]

If the guy who writes Wilbur would ever get off his lazy butt and finish the GDAL plugin then it would be able to handle file formats like GeoTIFF. Meanwhile, I find Global Mapper a pretty good tool for this srt of thing, but it is a little pricey.

Erosion and noise aren't going to do what you want as far as upping the apparent resolution of the DEM. I really recommend getting a higher-resoluton DEM of the area of interest, if you can, and just use that. An alternative is to pick a DEM with similar high-frequency detail to what you want and add that on top of the original. In an ideal world you'd be able to isolate the higher-frequency detail with a wavelet or fourier transform and then use that for the addition part (you keep the underlying gross features and add the new detail on top that way). I keep meaning to do something like this for Wilbur, but it requires multiple active documents or layers for best effect.

On the square section thing, can you elaborate? Wilbur is really just a funny paint program with a mix of really awesome tools and really crappy tools. My long-term goal is to reduce the number of crappy tools (the paint tools need fixing, for example).

[Redrobes]

What res DEM have you got then ? I reckon the SRTM3 at 3 arc second is about the highest I can get. 3 arc sec is about 100m or so per sample.

Another interesting way to get higher apparent res than what you have is to get a normal map of the DEM and lay that on top of the lower res mesh. Still, if res is not what is holding you up then this is not so good. Mention it if this would help tho.

[Midgardsormr]

I have about 30 meters per pixel from ASTER. It took some jumping through hoops to get it, but it's marvelous. Normal maps… I do have access to Zbrush, and it occurs to me that this is exactly the sort of thing it was designed for. That's the next tool to learn, then.

Regarding Wilbur, the default settings in the Resample dialogue are for a destination size of 512 x 512, and the Map Projection defaults assumed a square section, also. Those, obviously, are easy enough to deal with, but I wanted to be sure that the program isn't assuming a square image in other places that might surprise me. I have encountered tools in the past where the programmer had an unspoken paradigm of that sort, and it comes back with some surprising results if you give it input that doesn't match the expectations. The fact that you didn't ask "Why would you give it a non-square image?" is reassuring!

I really need to get some more Calculus. This would be a lot easier if I were capable of understanding the math. Alright, you've clarified my thinking, and I believe I know what my next steps should be: Zbrush and some math lessons. Thanks!

[Redrobes]

30m is pretty good stuff - no problem there. I don't think you need a lot of calculus for dealing with DEMs. An appreciation of the real basics would help like knowing what a normal is in those terms but you know what a normal is without doing calculus. I cant think of much where calculus would give you a big advantage.

To get a normal map you usually need a high res 3D object to extract the map from and then you apply it to a low res mesh and get an apparently higher res model than is actually being used. The point being that you need the high res height map and its not giving you a normal map higher res than the height map. So not sure that this would help. But Waldronates idea of using a different DEMs high freq part of normal map of a similar style terrain applied to your already high res displaced mesh would give you something similar. That's a good idea and better than adding random noise to a DEM I think as you would be able to keep using Maya to render your animation with it.

I have some tools to generate the normal map from a DEM so if your ZBrush does not fill that requirement then I can do that for you. I don't know whether I am able to do a high pass filter on a normal map and I am going to have to try that and see if it works cos that does sound interesting.

[waldronate]

jwbjerk had some good suggestions for synthesizing higher-resolution data at http://www.cartographersguild.com/sh...-project/page2 if you haven't seen it. Unfortunately I'm not sure how well this will work in your case because you're interested in data at a resolution that varies widely statistically (the river areas, for example, will be different than the plateu areas, which are different from the hilly edges of the plains). I have a suspicion that you're hoping for a couple of meters or less resolution, which is about the resolution of the lines, if I recall. If you can find some patches at the resolution you're looking for then the overlay thing will work, but I have a suspicion that you're probably going to end up doing some hand painting on the parts along your flight path.

As far as Wilbur goes, the light model assumes square pixels and the geospatial parts assume something in the equirectangular projection family, but the software doesn't care about square or rectangle. There were one or two really obscure file formats for custom software that require square sizes (and power of 2), but you're unlikely to want to use them because you don't have the software that supports those formats.