New ArcGIS Server... Potential for us fantasy cartographers? FOR SURE!

[loogie]

Well.. sorry i've been missing for quite awhile.. got some new projects at work and for such i need to learn FLEX... while learning a new API is not easy... it IS a hell of a lot easier learning FLEX then say... .NET

With that effect.. I was looking into the new FLEX API for ArcGIS Server 10.. and well, I can't help but think how utterly AMAZING it would be to be able to sandbox and online realm in it.

For those of you who are firmiliar with ArcGIS Software, but not ArcGIS 10... Two big changes which are of interest for this specific concept are Time based layers... and also the revamp of editing...

New to ArcGIS 10 is the capability to have a feature that is time based. This means you can create a feature that is linked to a time, and query those result... Better yet this capability is alive and well on their new FLEX web API.. meaning you can see these on the web, for instance a timeline slider.

Another great upgrade is a complete rethink of the editing process... Now you are able to create numberous "Template" symbols, so lines/points/polygons that have a certain look, and even set default data to them... (if most of your rivers are going to be water, and only 2 lava, it makes sense to make the type to default to water)... So editing a layer in ArcGIS is now a matter of click "Start Editing" which brings up a panel with all the templates, that you can then just click and start drawing with... Sweet. Again, this option is alive and well on the FLEX web API.. which means... *GASP* we can now do on the fly editing! yes not only did they allow the use of templates in editing online, but also the ability to edit on the fly!


So picture an online realm website... on that website is a map of our entire world, as created using ArcGIS with the land masses, water, trees, and anything else we can create.... now comes the fun part... If you are a member, you can log in, and "edit" the map, adding new data to it.... so for instance, in your last campaign you needed a new port city in the south that has not yet been created... You have made up a simple paper map, labelled some generic zones (market, etc) and are ready to post it online. Using the "City" point template, you can drop a point on the map so you can see it when zoomed out... you have more detail then that tho.. so you zoom in, and use the "city zones" and "roads" templates to draw the road work, and zoning you've done... Great! your edits area actually automatically posted on line, and within seconds everyone is able to see it...

Someone else loves the idea of your city, and decides to run his campaign in it... He's a bit of a perfectionist tho, and wants more of the city layout to be defined... He designs the different city "parcels" a general layout of each city block (who owns it, its use etc), as well as the building footprints. He can then use your data as a guildline (Even edit it to fit his closer detailed shapes) and add his city blocks, and building footprints using the available templates.

As you go, even more people can add, and even more, till you have a rich, creative world on a website.. with everyone contributing.. sort of an online map wiki...

But we all know as a world evolves people aren't always adventuring at the same time.. some games are slower, some people want to adventure in the past... when every city didn't have an arch wizard.. etc.. in comes timed layers... all of a sudden, your world's history is at your finger tips.. you can show the growth of a city, the shrinking of a lumber hub's resources... the drying up of a river, etc.

Pretty cool idea i think

only think i need now is someone willing to spend the thousands of dollars on an arggis server 10 license to be used for gaming purposes...

any takers?

Heres some examples that you should take a look at if you think my idea was a cool idea (shows some samples of the flex api at work)


Time Aware Eg

Editing

[tilt]

sounds interesting

[Redrobes]

Cool. I thought you meant FLEX as in FLEX / BISON but I looked it up and its some adobe thing unrelated so no idea what thats all about.

On the map front, what your describing is exactly a VTT. Although I don't know of any that do the time history thing but all the rest is exactly what ViewingDale is all about and id run that server if my web box was a windows box but it ain't. So I don't have a 24/7/365 machine to host it but in every other way thats a VTT. Forget thousands of dollars and 10 licenses. <$30 or so and your there. I keep trying to say how cool it would be to have this online editable world but I have not had anybody take up the gauntlet and run one on a 24/7/365 basis yet.

[su_liam]

I think he's talking about the license to use ArcServer. ESRI lies heavy on the pocket.

[Redrobes]

Yes I know, but you can do all he is suggesting, bar the same map with different time slots, using something costing comparatively peanuts and designed to be used for gaming.

which means... *GASP* we can now do on the fly editing!

As in yes you could do this for the last 4-5 years for peanuts. That's all I am saying.

Even the time slots thing could be worked around in a fashion. Not so slick I admit but I don't think so klunky that it would put people off either.

What ArcGIS is doing is cool and really good if you must use a GIS tool but its not new and there are alternative options for the same effect built for gaming RPG worlds.

[loogie]

Well I know what VTT`s are, and how they work, but most (Maptool for instance) is heavy on the resources for that type of thing (its not loading stuff into a database, its loading resources right from the computer.. that combined with the concept of chatting, viewing and such it makes it kinda unrealistic to try... From what I understood Viewingdale would go under the same boat.. its made for use in a person to person game (at which it'd work great) but as for working in big scale, with thousands of entries, it'd bog down to a crawl... thats why a GIS system is usefull, (all be it it doesn't need to be a gis system, its just the only one i could see working at the moment).

So PLEASE correct me if I'm wrong, because I would love to spearhead such a project... its something I've dreamed about for years.

As far as ArcGIS's cost.. yeah I'm venturing around 5k for a license.. but I wouldn't be surprized if it was more.. and thats just for THAT software... db's and such would run you more.

wither way. it's exciting to think that it may be possible (even for me)

the way i see it, even if its a very basic map, i'd be happy... if its something we could combine and make a world, then let everyone have their artistic cartographical way with.. it'd be a great project for a community build on fantasy realm design!

I know there have been projects in the past, but an online, realtime sharing WITH (a key point for me) a map involved is a big deal... GIS is great cause it's scalable.. you want to edit the world as a whole.. sure.. you want to edit a city itself? zoom in to the spot... I'll have to talk to you more about ViewingDale Redrobes.. Its definitely an interesting concept.

[Redrobes]

ArcGIS is a CAD type bit of software which in turn deals mainly in vector graphics of lines and polygons to make maps and these, of course, scale well. Its running from a data base which has a pedigree of being able to cope with many users hitting the database in rapid succession. In terms of making the kind of maps you see in Ordnance Survey / Road maps and architectural plans it would do extremely well. In a client / server set up as you describe you could either request a part of the map and let the server send that section of the map down to the smart client where its rendered out or you can render the bit of the map on the server and send down just enough info to render that screen view only on a more dumb client. If the user moves a little then in the first instance the client says to the server I have moved a little, server says yeah ok nothing changed and now go and rerender that scene. Hardly any bandwidth. In the second case client says I have moved a little, server grabs the map from the database again and sorts out a draw list and sends that back and dumb client just renders it again. So in the first case the initial download is large but from there on its small updates or in second case its uniformly middle sized info from the server down to the client.

Now the vast majority of RPG maps are not vector but bitmapped. Ok so a very select few on here use vector stuff - GP with Xara, and some of the CAD, Space ship designers etc but it must be 90%+ of people here use PS / Gimp or whatever to create a bitmap. Even for those that did use a vector format I dont think that many of them would have maps in ArcGIS format. So you are basically cutting yourself off from the existing maps and creating a new world exclusively in GIS formats.

Now if you can embed bitmapped images into the vector GIS formats then you have those maps. Now heres the thing. If you have the smart client option then its a huge huge download for all the bitmapped images once but then its fast to update - this is the MapTool / ViewingDale / FM8 approach. If you have the dumb client then its going to more like ScreenMonkey where it uses your web browser as the client (i.e. kinda dumb) and so must send the portions of the image down each time to make up the screen. In most cases these VTTs use a fixed backdrop image and send down only the moving bits so that it can cache that backdrop image.

So if your going to do a whole world then you really need the smart client way of doing things or else the bandwidth is going to be massive. Since it seems that this ArcGIS demo was using the browser then its not likely to be or maybe its unable to use RPG bitmapped maps. But lets for one second assume it is smart. At some stage the server is going to have to send down all those images required to render the map. You cant avoid the issue if you want to map with bitmapped images. So this is a necessary evil for anyone who wants to do it this way and I think I have described why for the case we want its the only way.

So because everyone must send them then lets assume that you have took the pain and got a load of images downloaded. Its about 5Gb or so. What you would like to do now is reuse as many of those images in as many situations as possible so that you can have different things made out of the same images. So think of castle. We could have some crenelations and make walls and tower out of the same bit. Once you have laid out 20 or so to make a square tower then you would like to put ten towers down you need to set that tower crenelation set as a group and use them as one tower and put down ten towers which is 200 crenelation images. This hierarchical grouping mechanism is what viewingdale is all about and its capable of rendering about 100,000 of these images per second so it can make up a map out of lots of small bits of things. Its likely that a GIS program would do it this way too but its definitely not possible to get your browser to do it. So the system that you have highlighted must render them on the server and send down that post rendered screen. That does not scale with lots of users. So as more users join up on ArcGIS it would slow down severely or, as I suspect, it wont allow bitmapped images.

Because in ViewingDale you only have to say where the ten towers are and how to make a tower the amount of information that needs to be sent down per user per render is tiny - in the hundreds of bytes not tens of K or megs of image. Thats why when you make a change to the map its updated on all the others so instantly unless the change is to add a new image of which then that image has to be sent to everyone. So in reality the real scalability of ViewingDale is in the server upload bandwidth which for me with my asymmetric bandwidth is criminally low so its never been tested to destruction.

ViewingDale connects to other copies of the app. Every app is capable of being the server or a client. When a user connects it as a client to another copy as a server its running in that smart mode. You can also set up one copy as the server and make it serve web pages too. In that case it renders the part of the map requested and then it sends a PNG image back to the web browser as a client so that its more like Screen Monkey or for when not everyone in your group has a copy. But as said that is slow and does not scale very well but it can be done. You cant edit the map on the server using a web browser as a client tho.

When editing the servers map with another copy as the client in the smart mode. One person at a time gets to edit the map which is not as good as using a database. There is what I have called the edit baton which you grab and keep until you give it up. When your editing the map everyone else sees your username as editing the map.

When I say editing the map I mean you can edit any thing in the universe that the map is displaying. So you can edit a pub whilst someone else subsequently takes the baton and then edits a battle map in another area of the map. So its not like your stuck with a battle map and everyone has to deal with that before you go to another location. So its world sized editing at any zoom level just like you wanted.

The number of concurrently logged in users is limited but I think the limit is quite high. I don't think I have ever stressed it with real internet based people enough that I broke it. Its bandwidth limited more than CPU limited cos really, the server doesn't have to do very much. In smart operation its all the clients doing the work which is why it scales well.

I hope that gives you some insight into how these things work, how mine works and some of the issues involved in it all. I have custom built ViewingDale to tackle this precise problem using every available optimization that I know to make that job as slick as possible. If your ok with a vector world and not wanting to use existing peoples maps then sticking with vectors might mean that my solution is not as optimal as it might possibly be. But I tackled mine from the point of view that if you want a lot of people to add to your world then you have to sell something which does not need $5K or custom format maps etc. I can see why govt agencies would need it ArcGIS way but I don't think with RPG nuts it would be a goer. I'm all ears for opinions and comment tho. Id love to know why people haven't taken it on board more than they have.

I dunno if you have seen this vid. I post it now and again cos it covers a lot of what the app does. If connected with multiple users into a server then you could edit the position and size of any bitmap or icon on this map at any scale together - but one at a time.
http://www.youtube.com/watch?v=qELcbKXdRm8

[loogie]

To a point your correct... The difference is things are becoming a LOT more usable in the web design sense, as far as computing resources.

ESRI's old Web software worked like that, as do most Web GIS softwares, moving things were difficult, and client and server load preformed much more like the "dumb client" you spoke of.

However, now with the advent of ArcGIS Server, thing work much better.

I don't pretend to know the functions of ArcGIS like you explained, but the client/server connection works more like a "viewport" then an actual physical web version of a map... This makes things MUCH faster, and MUCH more usable for many GIS ideas.

The basic premise of this is that the client is not physically manipulating the data, but has access to a "window" to the data, which is actually just a screenshot of the current state. The client can manipulate what it sees by using services (ArcGIS uses REST and WMS I believe, but I'm not going to pretend I'm a web guru either). These services are such things as a Map Service (viewing a map with layers, Image service (ortho photography or raster images), GeoData Service (addressing), GeoProcessing (buffers, intersects, unions, etc), and any number of custom services you can create yourself... These services all work the same way, they are ways to remotely tell the server how to change the look of the data... This means that while every person using Maptool is loading say, 500 objects over the net on a large campaign file... in ArcGIS, the server would be loading the 500 objects, tracking the few changes everyone has made, and the clients are simply loading a jpg, or in the case of FLEX a swf (which is in most cases MUCH faster)

The beauty of this concept is that everything as far as loading and manipulating the data (the hard stuff) is all done serverside, the only loading for the client is a series of images (usually PNG for their transparencies) that are "screen shots" of the data at the specific state. Now a single map view on the client usually consists of several PNG layers, generally a basemap layer, any dynamic layers on top of it, and any number of graphical layers on top of that (graphical layers hold text, selection graphics, and any other temp data)

This means that for a client.. the load is simply "load the image the server gives me"... and not only that, it cache's your results, for each layer as well.. so if you use the basemap layer but change a selection, it doesn't have to refresh the basemap, it just changes the selection layer, and so on... it also saves the result itself.. so it knows if you have panned away from a section, and then back.. that it already has the original location saved and loads it. very speedy, very brilliant in my opinion.

On the server takes the brunt of the work and so it usually needs to be fairly beefy, but for web standards its really not bad at all. The main thing is, the bigger and beefier the server, the more services you can run, and the more cache's you can do... I work for a small county in canada, and our webservice is set to around 10 connections (that means it can preform 10 tasks at once.. and generally is able to have 10 people connected to the server simultaneously without any interruption due to overcrowding.. and in my experience, even with more, the interruption is very minimal)

This has make Web GIS go from a static "i have a layer for lakes, and a layer for roads, and buildings, zoom around my map at will" concept... to a "i can do anything in web GIS that arcmap can do by itself" including all of the geoprocessing, like measuring areas, drive times, buffers, viewsheds, watersheds, networking, everything.... all dynamically... and all taking roughly a small portion longer then it would on a desktop GIS...

Being part of the Web GIS scene for a few years (at least in a sense to know the capabilities of a Web GIS system) ArcGIS Server is a HUGE step forward in the field of web based GIS, and ArcGIS Server 10 will be yet another... But back to it being rediculously expensive, and not at all built for fantasy mapping

Besides the community world building project (which as far as i see is a wiki, with individual mapping) has there been any other attempt at a collective world?

[Redrobes]

Cool, ok loogie good info. It sounds like it can do bitmapped images then but it is indeed using what I called the dumb client method cos the intelligence is held on the server. I can see that by having several layers you cache some of the images which is what programs like screen monkey do when the backdrop image is not changing much. Google maps does much the same and you can see that when panning back and forth. The thing is that you don't generally edit google maps so that works better and for a battle map you don't generally edit the backgorund layer much (well, in normal RPG maps anyway) and just edit the character and monster tokens.

10 connections is not too bad really. If you had a really big world like what WoW has then you would have thousands of people connected - maybe millions but they tend to play in it not edit it. Personally I think with an RPG map you might get a lot of people requesting the map but not that many editing it at once. 10 sounds like a reasonable amount.

All the dev work theyre doing makes total sense from a GIS perspective more than an RPG / gamer perspective.

As for another RPG community world I don't know. Maybe there's some stuff in second life where you might find such a place but I don't know of any other really detailed bit of mapping going on by more than a few people. The one here is a lot of maps done by individuals instead of areas concurrently edited by many - you can see that from the vid link I posted. In general people have taken charge of some places and drilled into those areas. I think the lack of a ubiquitous tool to do concurrent mapping is the reason it could not be any other way.

[loogie]

To a point your correct... The difference is things are becoming a LOT more usable in the web design sense, as far as computing resources.

ESRI's old Web software worked like that, as do most Web GIS softwares, moving things were difficult, and client and server load preformed much more like the "dumb client" you spoke of.

However, now with the advent of ArcGIS Server, thing work much better.

I don't pretend to know the functions of ArcGIS like you explained, but the client/server connection works more like a "viewport" then an actual physical web version of a map... This makes things MUCH faster, and MUCH more usable for many GIS ideas.

The basic premise of this is that the client is not physically manipulating the data, but has access to a "window" to the data, which is actually just a screenshot of the current state. The client can manipulate what it sees by using services (ArcGIS uses REST and WMS I believe, but I'm not going to pretend I'm a web guru either). These services are such things as a Map Service (viewing a map with layers, Image service (ortho photography or raster images), GeoData Service (addressing), GeoProcessing (buffers, intersects, unions, etc), and any number of custom services you can create yourself... These services all work the same way, they are ways to remotely tell the server how to change the look of the data... This means that while every person using Maptool is loading say, 500 objects over the net on a large campaign file... in ArcGIS, the server would be loading the 500 objects, tracking the few changes everyone has made, and the clients are simply loading a jpg, or in the case of FLEX a swf (which is in most cases MUCH faster)

The beauty of this concept is that everything as far as loading and manipulating the data (the hard stuff) is all done serverside, the only loading for the client is a series of images (usually PNG for their transparencies) that are "screen shots" of the data at the specific state. Now a single map view on the client usually consists of several PNG layers, generally a basemap layer, any dynamic layers on top of it, and any number of graphical layers on top of that (graphical layers hold text, selection graphics, and any other temp data)

This means that for a client.. the load is simply "load the image the server gives me"... and not only that, it cache's your results, for each layer as well.. so if you use the basemap layer but change a selection, it doesn't have to refresh the basemap, it just changes the selection layer, and so on... it also saves the result itself.. so it knows if you have panned away from a section, and then back.. that it already has the original location saved and loads it. very speedy, very brilliant in my opinion.

On the server takes the brunt of the work and so it usually needs to be fairly beefy, but for web standards its really not bad at all. The main thing is, the bigger and beefier the server, the more services you can run, and the more cache's you can do... I work for a small county in canada, and our webservice is set to around 10 connections (that means it can preform 10 tasks at once.. and generally is able to have 10 people connected to the server simultaneously without any interruption due to overcrowding.. and in my experience, even with more, the interruption is very minimal)

This has make Web GIS go from a static "i have a layer for lakes, and a layer for roads, and buildings, zoom around my map at will" concept... to a "i can do anything in web GIS that arcmap can do by itself" including all of the geoprocessing, like measuring areas, drive times, buffers, viewsheds, watersheds, networking, everything.... all dynamically... and all taking roughly a small portion longer then it would on a desktop GIS...

Being part of the Web GIS scene for a few years (at least in a sense to know the capabilities of a Web GIS system) ArcGIS Server is a HUGE step forward in the field of web based GIS, and ArcGIS Server 10 will be yet another... But back to it being rediculously expensive, and not at all built for fantasy mapping

Besides the community world building project (which as far as i see is a wiki, with individual mapping) has there been any other attempt at a collective world?