New battlemap-creation software on the way

[Redrobes]

Lack of physical RAM is a problem whatever bit program your using I agree. But what I was saying is that if you had say 16Gb of physical RAM and you want to run a 32bit app which then limits the apps process to being able to access 2 or 3Gb or so (depending on the setup) then you have 13Gb of physical RAM you cant access unless you use a 64 bit app. But you can run a 32 bit app which creates many processes (instead of many threads) whereby each process can run up 3Gb of allocation. So its possible to have a 32 bit app use all 16Gb of physical RAM. Most web browsers open a new tab in a new process because a) it isolates the page from other pages which is for security and b) you can have pages running up large memory footprints and have many of those pages. So what I was saying is that being a 32 bit app is not a hard limit to accessing large amounts of RAM. But a lack of memory sticks in the machine is always going to be a problem for large multi layered maps no matter how you slice it. But if you can only compile / write 32 bit apps (see herucas post #12) then its not a dead end situation for the ability to deliver large mapping capabilies.

The XCF thing I agree with you. It didnt appear from post #14 as though that was what you were saying. But with that clarification - I agree.

[heruca]

I do remember Dundjinni taking what seemed like hours to export the final map. I ran some tests with MapForge, exporting a 10Kx10K pixel image in 24-bit color (uncompressed size: 300000000 bytes) and it took between 8 and 31 seconds to complete the export to PNG format. Exporting to JPG and BMP format took much longer, but was still very fast in comparison to DJ.

[heruca]

To clarify, what I exported was just a floodfill tile texture that was randomized to make each tile different. MapForge doesn't really have layers yet, aside from a grid overlay (which adds about a second to the export time, if you export with a grid). This is being tested on an early prototype. Performance may be slower in the final shipping app (which WILL have layers), or it may be even faster once optimizations are added.

[Mark Oliva]

Lack of physical RAM is a problem whatever bit program your using I agree. But what I was saying is that if you had say 16Gb of physical RAM and you want to run a 32bit app which then limits the apps process to being able to access 2 or 3Gb or so (depending on the setup) then you have 13Gb of physical RAM you cant access unless you use a 64 bit app. <SNIP>

I understood, and I agree.

The XCF thing I agree with you. It didnt appear from post #14 as though that was what you were saying. But with that clarification - I agree.

As I said, I wasn't very clear. I think describing The GIMP as a bitmap program with layers will leave a wrong impression with some people who don't have a lot of technical knowledge, leading them to think that The GIMP produces bitmap graphics that one can open in programs like MS Paint with file extensions like BMP, JPG and PNG. This, of course, isn't what The GIMP does, It produces a raster graphic file that consists of one or more layers, each of which is a bitmap. To make this into a flat bitmap that's the kind of file many understand a bitmap to be, the content of the native XCF file graphic has to be flattened and exported as a BMP, JPG, PNG or whatever. I thought this difference should be mentioned.

I ran some tests with MapForge, exporting a 10Kx10K pixel image in 24-bit color (uncompressed size: 300000000 bytes) and it took between 8 and 31 seconds to complete the export to PNG format. Exporting to JPG and BMP format took much longer, but was still very fast in comparison to DJ.

If I counted the zeroes right, 300000000 bytes = 300 MB.

That's a pretty good performance statistic.

Not a plus-or-minus point, but a curiosity. You mention the PNG conversion being faster than the BMP and JPG conversions. As I understand it, all three were exports from Map Forge. A question then: Assuming (perhaps incorrectly) that the conversion was a direct conversion from memory to new file, did you compare the JPG and BMP export times to the times that would result in converting an exported PNG file from Map Forge into BMP and JPG files with an external program?

Whatever the case, I'll be watching this with interest. Even if the end result is nothing more than what amounts to a new and improved version of 32-bit Dundjinni, it's still more than we have now. I'm on your side. I hope you succeed!

Good luck!

[heruca]

I think you counted the zeroes right, because my BMP export was 300MB. I think most of the time saving to BMP format is spent writing the huge file to disk. The PNG export was just 4.9 MB. JPG was around 3 MB, but it took so much longer it's not worth the savings. Probably faster for a user to open the exported PNG in MS Paint (or whatever) and "save as" a JPG.

[Mouse]

...Also, for Mouse, if you have layers that are greyscale (shadows, masks etc) then do set the Image / Mode to Greyscale instead of the default RGB which should quarter the amount of RAM Gimp will allocate for it. But 10K x 10K x greyscale x 135 = 13.5Gb of RAM uncompressed and if they were RGB then that would be 54Gb of RAM required. Not many image editors are going to handle that and CC3 is not going to either if those are genuinely bitmap layers your referring to.

That's a useful tip about using greyscale in GIMP. Thanks Red

In answer to your last point - CC3+ uses bitmap textures to fill polygons, and most of the symbols/icons are tiny bitmap images, but none of them are stored in the CC3 file. They are all referenced by the software and 'called' (I hope that's the right term) from elsewhere to be displayed on the screen and used to calculate the final output render/export bitmap. Its just easier to refer to them as layers when discussing them with non-CC3 mappers

EDIT: Heruca - When I made my earlier comments about 32-bit being perfectly ok, I was trying to encourage your entrepreneurial spirit, not start an argument with anyone! LOL! Best of luck with your project

[heruca]

Thanks. Not sure that anyone's really arguing, though, just clarifying misunderstandings.

[Redrobes]

Agreed, when designing systems the devil is in the details. Its important that the details are well understood by everybody in the same way. Small differences in understanding always lead to big implementation problems. Its not that in this case were designing it but it is important to understand the design details that Heruca is outlining.

300Mb sounds right to me. 10,000 x 10,000 x 3 (RGB) is 300 million bytes. Its odd that it takes longer to compress to JPG than PNG. Its almost extraordinary that it is longer to save an uncompressed BMP file than a PNG. In the process you have compute and I/O. Since the PNG is only 5Mb then it implies that the system on which it was being tested had great CPU performance to compress it but terrible HDD performance to store it. If you had used a slower CPU machine with an SSD I would have expected it to be very different. Also, it implies that the system did not have 300Mb of cache RAM spare to offload the file before it was saved. Most systems / OS's 'save' the file to a system buffer which then chugs it out to the HDD in the background and gives control back to the app. Its possible tho that the app tried to check the file by reopening it (or doing some kind of stat on it) which would force the OS to stall that check until the save had fully completed. I know on my app the PNG is the slowest, JPG is better and BMP is fastest. I do overlap the render and save at the same time for bitmaps (BMP files) but I cant do that for the other two tho. So maybe that skews the timings a bit. Even so I would think that you may need to check those times on various bits of hardware before really relying on them. Its a very unusual result to say the least. JPG is normally quite quick to compress and quick by being small to save so normally wins. Some CPUs have special instructions that can be leveraged to help save JPGs but a JPG library may need to ask for them at run time. If they didn't exist then it could be slower. Lots of reasons the speed could change between hardware.

I agree that CC3 is doing special stuff regarding its layers. It obviously has to in order to do 135 layers. So its not comparable to measure it against GIMP. The texture fill thing is an obvious trade off between memory usage and versatility. As an bitmap / raster / image editor (you pick) Gimp is clearly going to be able to have anything going on at any point in one image layer since it has allocated enough memory for the whole sheet. I know a lot of mappers here at the guild are artists that make great maps rather than CAD like in their approach which is why we get a lot of photoshop / Gimp users. Artists will make use of the layers at all points and so Gimp runs rings around CC3 in that respect. So you win some you lose some by taking the trade off. Incidentally its a trade off that I took with my app and as such you still needed an image editor to use it properly. In my mind CC3 and Gimp (a vector - hybrid and a raster app) operate in two separate spaces. But I know I would be fighting a lonely battle if I kept up with that argument - but I believe it nonetheless. Some apps like Xara bridge it well but no app does both sides as well as the best dedicated ones. To pick a camp one side or the other is to miss the point so to be entrenched into one side and waving the flag proudly is a poor decision.

So yeah, how to make a mapping app that handles the best of vector and raster at the same time ? If you go for one or the other then there are other apps out there already which do it well I think because its easier to persuade people of the benefits of each individually and so the graft of development has already happened. But I am in the camp that if targeting maps specifically as opposed to maps of pure art then there could be a better option. That option is still not implemented. Its a hard task. Part of the task must be to persuade / show or realign peoples mentality about how to go about mapping to the way your app is going to do it - which is something I found extremely challenging. So my very best of luck to you with your endeavor.

[heruca]

I have an SSD on my computer, but it is nearly full and is probably having a hard time because of that. That might explain why the BMP took so long to write to disk.

What is your app called, Redrobes? Good to see you here again, by the way.

[jfrazierjr]

I have an SSD on my computer, but it is nearly full and is probably having a hard time because of that. That might explain why the BMP took so long to write to disk.

What is your app called, Redrobes? Good to see you here again, by the way.

ViewingDale. There is an older Youtube video if you want to check it out. The way it deals with zoom is really really cool. If I did not have a VTT, I might consider it simply because of that feature.