Photoshop Maps and Coastlines

[bruixen]

Hi there, so ive been working on a type of fantasy gameplay map where players can move armies build cities etc and enjoy playing ontop the map in a 2d online setting. Anyway, ive really hit a snag in two ways and its really bothering me as i cant figure it out lol. if you can help that would be extremely helpful.

I am working in Photoshop cs6. attached is an image and the .psd file.
Problem 1:
Mountains - I cant get them to stop looking so plastic/metallic/shiny. The Shatteredworlds mountains are what im aiming for, in terms of color and texture. Is there a way to achieve that?

The way i have it now is i used a inner Blevel set to chisel soft w/ a stone pattern overlay on a mountain overlay layer. The shading is set to default with global light on.

Problem 2:
Coastline - Whenever i try to add a glow to the coastline, it also grabs the edge of the map as it interpreted the edge to be a coastline. Its actually not, the map is going to be a 2x2 replication of its self, so the edge of the map ie: bottom left or right, is a large mass of water. How do i stop it from grabbing the edges of the map when i add the outer glow layer style. Id also like to add jitter to the coast and beach area.

I use a outside stroke to outline the "beach". Id like to know how to apply a jitter to this stroke, so the coast and beginning of the beach is not so linear. I use an Inner glow to make the shore noise.

Any help would be greatly appreciated thank you all very much.