The Köppen–Geiger climate classification made simpler (I hope so)

[AzureWings]

Indeed - there'd be more information associated with each pixel than could be stored in a color value. I've mainly been approaching the problem from the point of view of the scripts I've been writing, which aren't integrated with any image editors like PS or GIMP, so defining a place to store extra data isn't the tricky part. That said, you probably could full-automate the process in them, but working in an environment integrated with a much more heavyweight system like that has a good chance of incurring significant costs in efficiency (especially if you need to use image components like extra layers in large numbers as your data allocation). Not all of that extra information would ever be output to any one image, but would be used to construct them. By 'pixel-by-pixel vector field' I meant a data structure with a vector for every pixel in the source image - antillies' mention of "pixel objects" is pretty much right on to what ultimately would be needed, although the precise layout of that data might differ (not sure at the moment, but it is possible there would be relevant tradeoffs to storing a set of objects-per-pixel or having instead a set of parallel arrays that each hold one data value). For the moment I've been doing more of the latter since I've been working on single steps, but 'pixel objects' is a natural progression in constructing a more generalized approach to a broader/multistep formulation of the problem.

The reason efficiency is a concern is that these images can be fairly large, so whatever operations you do per pixel have a huge constant factor. I'll admit my perception is a bit colored by the fact that testing the code involves running it quite a bit , whereas in general use you're right that a command-line process in this style that could be launched in the background can reasonably take a bit of time to run. The space efficiency can be somewhat of a concern, since that doesn't make a big difference up to a certain point and then it is enormously important - and with the input sizes of the maps it can become an issue. For say a 4000x2000 image which isn't really all that big every byte of information stored per pixel is ~8MB of memory which, while not all that huge in and of itself, can add up, especially if one starts looking at higher-res images for source data (or say, storing a vector that in Python is probably two doubles, so 16 bytes per pixel -> 128MB for the 4000x2000 example - again not a ton by itself, but starting to look more concerning in the presence of multiple other similar data values). And if you move up to a significantly larger resolution like say 16000x8000 - then it's ~128MB per byte per pixel, and as soon as that adds up enough that the OS starts swapping memory to disk (at best, when you approach the amount of RAM on the system) the performance will become much, much worse.

The shortest-path tree idea comes from how a lot of flood-fill algorithms work - they treat pixels as nodes and do a graph traversal. Here we'd just do a weighted graph traversal and annotate each pixel/node with the distance we found from the 'supernode' (which represents all the oceanic pixels together). Practically we might need to separate closed seas from ocean first in constructing that set (e.g. like the Caspian Sea on Earth) but that can just be union-find again (and that can be implemented in linear time in this context; it just takes one pass over the image since we only care about adjacency and not distance for 'what pixels are part of which separated bodies of water?').

As for separating landmasses with a land-bridge, you could do union-find for major landmasses on only pixels meeting a threshold of distance-from-ocean (not just in pixel count, but distance - this both helps with projection distortions and keeps the algorithm's output agnostic to the image resolution). Then you could make these 'major landmass' pixel collections into supernodes to use as roots for a new graph traversal - labeling each pixel with the supernode they have the closest land distance to and using that to allocate pixels to discrete landmasses. Obviously this couldn't be flawless - it would depend significantly on what the threshold for considering something a narrower land-bridge is (i.e. how close to the ocean a pixel is considered coastal as opposed to central to a major landmass), but with the right tuning I'd expect this approach could reliably identify as separate North America vs South America or Eurasia vs Africa.

Regarding generating winds - I hadn't thought of that in terms of overlaying a base 'natural winds' framework the likes of which I think I recall fairly straightforward diagrams of before with another for the pressure systems - which could have systematically defined behavior based on an input of a human-readable color-by-pressure-system map like what is already in the tutorial.

[Azélor]

To me, the landbridge is the same as the coast from any continent. The area is just the same but smaller. Yet I not expert in python and might be missing something.

AzureWings: you need to store more data per pixel if the map is larger? Or maybe I misunderstood ( 128mb per byte???).

[AzureWings]

Ah, no - sorry, the point I was making was that, from a perspective of storing extra data for every pixel, storing a small number of bytes per pixel can get costly if the map is big enough because it adds up rather quickly. My example was that for a 4000x2000 map, storing a vector of two floating-point values (double-precision, in Python, most likely, so 8 bytes each) per pixel is 16 * 4000 * 2000 bytes for the whole map, which is on the order of 128MB. That's for the whole map, so as I said it isn't too bad, but if you store multiple various chunks of extra data it can add up. There doesn't need to be more data per pixel if the map is larger - it's just that even a small amount of data per pixel can add up to a problematic size if the map is big enough.

The issue with the landbridges is if we wanted a script to be able to distinguish between separate continental landmasses - i.e. be able to tell the difference between one large connected continent and two continents connected by a narrow land bridge that is entirely 'coastal' in terms of its ocean proximity. It's easy to tell the land bridge is coast; the trickier part is being able to tell the continents apart. Although it isn't something explicitly used in the tutorial currently, for a script trying to find automated ways to make nicer results being able to distinguish between separate continental landmasses might be useful, and the landbridges add a complication to programmatically trying to tell which landmass a given pixel is a part of.

[Azélor]

Would'nt it be easier to make the delimitations manually? If you don't mind using a graphic software for that.

[AzureWings]

That is certainly an option - although if the delineations were drawn lines, a union-find would still be needed for the program to be able to separate pixels into their respective landmasses. A map with a distinct color per landmass would be easiest, but that would require an entirely separate input map (while it is conceivable delineating lines could be placed on an input image containing other miscellaneous data). It would simplify things in code and probably save running time, though the cost would be requiring additional sets of input data, and there's already quite a bit of input to get set up for it. It's kind of a trade-off, in other words, between how tricky it is to code the script and how much work needs to be put in by a user of the script; when possible I like to err towards the former and leave the script easier to use when possible. With the right thresholds for what marks a pixel as landlocked or coastal, automated major landmass detection seems like something that could largely operate completely automatically and still err on the correct side in how it divides pixels into distinct continents.

I'm getting a bit ahead of myself in this planning/speculation, though - I'm still setting things up for the temperature script to attempt more nuanced approaches from its basic form.

[Vareck Bostrom]

To me, the landbridge is the same as the coast from any continent. The area is just the same but smaller. Yet I not expert in python and might be missing something.

AzureWings: you need to store more data per pixel if the map is larger? Or maybe I misunderstood ( 128mb per byte???).

I scanned this thread so I might not be grasping the nature of the problem, but you can store any amount of data per pixel that you want. You can even use already defined file formats like planar TIFF or PNG. I presume you're going to want all pixel associated data in planar format anyway if you're going to be processing via CUDA or OpenCL, or am I missing something?

Screen Shot 2018-03-19 at 9.08.39 PM.png

[Vareck Bostrom]

The issue with the landbridges is if we wanted a script to be able to distinguish between separate continental landmasses - i.e. be able to tell the difference between one large connected continent and two continents connected by a narrow land bridge that is entirely 'coastal' in terms of its ocean proximity. It's easy to tell the land bridge is coast; the trickier part is being able to tell the continents apart. Although it isn't something explicitly used in the tutorial currently, for a script trying to find automated ways to make nicer results being able to distinguish between separate continental landmasses might be useful, and the landbridges add a complication to programmatically trying to tell which landmass a given pixel is a part of.

And once again I'm probably jumping in before I should, but what's wrong with doing an erode pass or two to get rid of narrow land bridges? Most software libraries (that I know of, anyway) that offer morphological componentization will offer some kind of tolerances around that.

The problem is that "a pixel" has different meanings depending on your location on the map, and a method like this would have to take care with border or wraparound continents. You could also raise the sea level until you have the appropriate number of morphological components, lower it again, and define anything not identified as an individual component as a "land bridge".

I have to read the tutorial, but why do individual continents need to be identified at all? I would think that a somewhat robust climate module would take input boundary conditions and sort of sort all that out for itself?

At any rate, I make use of arrays that are sometimes interpreted as images and are easily imported and exported to common image formats, and I'd recommend this approach in general as it keeps the data in one simple package to move around.

Here, for example, the world I have been working on:
Screen Shot 2018-03-19 at 10.23.20 PM.png

The first image is the daily heat map from solar insolation balanced by thermal energy bleeding into the lower layer of the atmosphere, which of course has a strong altitude component. The second image is the morphological breakdown of the individual land areas - but that's wrong and not used. I have a single continent and a couple large and medium islands.

[AzureWings]

Nothing's wrong with some number of erode passes (though we haven't had any discussion on ways to model erosion in this thread as far as I recall); that also seems a reasonable enough approach to distinguishing separate major landmasses. The by-hand tutorial as is doesn't make any direct use of distinctly identified landmasses, but it's not inconceivable that that information could be of use somewhere - being able to identify coastal regions as bordering a particular landlocked area might be of use (distinguish, for example, a coastal area not within the ITCZ's low-pressure zone in one season but bordering an inland region that is, from another coastal area also not within the ITCZ, bordering a different inland region that also isn't in the low-pressure zone). The continents aren't to be limited to a set number of distinct components - rather, whether they'd be distinct would ideally be a matter of relative connectedness, for which it is likely that existing solutions are out there (though personally I'm often of a bent to construct algorithms myself).

You're correct that simply using pixel adjacency neglects projection issues (or wraparound, though that's generally not too bad of an edge case to handle); I've been using pixel-based representational terminology a fair bit because the simplest beginning portion of the temperature-generation script (and the preceding temperature+precipitation -> Köppen-Geiger classification scripts) operated on individual pixels in isolation across images - the operations done up until the recent more complex discussion on each pixel have been completely separable (i.e. did not depend on the east/west position of a pixel nor its spatial relationship to surrounding ones) and so the varying meaning of each pixel in terms of geographic area wasn't relevant to those implementations. The more current discussion regarding how to better model various aspects of building the necessary data indeed needs to take projection distortion, adjacency across east/west boundary, and the +/- 90° latitude edge case into account.

Thus far (and we've only entered this discussion on automating steps other than the final, rather straightforward mapping from temperature+precipitation to climate class in the last week or thereabouts, I think) we've been more focused on the functionality of the climate modeling part, rather than optimization, so the significant potential boost via parallelism (on a GPU or otherwise) has not really been on the radar yet, re: CUDA/OpenCL/etc. Input for now has been in the realm of hand-generatable images, in line with the tutorial, which means that some conversion on data read is generally necessary - if partly just because it's nice to have the input be more human-visually-intelligible. I agree with storing the data in arrays overall; and at the higher end of efficiency optimizations parallel arrays of data would probably win out over a single array of multivalued objects containing multiple data components, though again at the current point we haven't been especially concerned with optimization. Storing the data internally in an image format with a lot of extra channels to accomplish that wasn't something that had occurred to me; that's a rather neat idea if not quite suitable for the base input format.

Modelling insolation is rather neat, by the way, although it wasn't directly a part of the method used in the existing tutorial here for coming up with temperatures.

[Vareck Bostrom]

Nothing's wrong with some number of erode passes (though we haven't had any discussion on ways to model erosion in this thread as far as I recall); that also seems a reasonable enough approach to distinguishing separate major landmasses.

I meant binary morphological erosion - the opposite of morphological dilation. https://en.m.wikipedia.org/wiki/Erosion_(morphology), not terrain erosion from weathering processes.

[Azélor]

although if the delineations were drawn lines, a union-find would still be needed for the program to be able to separate pixels into their respective landmasses.

Not just lines but fill the whole landmass. I know it's another step but it should not take long to do. Whether it's worth it or not depend on the complexity of the algorithm, which a have absolutely no idea.

I presume you're going to want all pixel associated data in planar format anyway if you're going to be processing via CUDA or OpenCL, or am I missing something?

To be honest, I don't know what CUDA or OpenCL are.