[Award Winner] Explanation of Layer Masks in GIMP(and PS)

[jfrazierjr]

There have been a few people who have stated that they don't really know or understand layer masks and thus never use them. I have to say, layer masks are an incredible technique to add to your mapping arsenal.

I hope with this short tutorial to provide at least one person with a new respect for layer masks. If you get something useful from this tutorial, please remember to rate the thread. Likewise, if anyone has any additional tips or tricks I did not include, please feel free to include them in this thread for others to learn from.

Now, on with the tutorial.

The first question is "what is a layer mask": Layer masks are adjustments which are attached to a layer that affect the visibility or the layer.

Second question is: "why would I use a layer mask": To maintain flexibility!

Third question is "how do layer masks work": A layer mask is a gray scale image in which white indicates 0% opacity and black indicates 100% opacity. Any shade of gray in between black and white is the amount of the image that will show though.

So, let's start with a few exercises. First, we will begin with the typical background image and use a layer mask to show a section of a different color.
10-11-2008 9-31-01 PM.png (yea, not to terribly exciting).

The typical way someone who does not use masks would do this would be to make a selection with the circle section tool and fill with the secondary color. Now, this works, but, if you need to edit either color, you have a lot of extra work to do and unneeded extra work is bad.

So, what about using layers:

10-11-2008 9-36-44 PM.png

This is better as you have made a second layer transparent and then used the circle selection tool and filled the selection with the second color. This gets the job done and maintains more flexibility than the previous technique, so is much better. But, since this is a tutorial about layer masks, we will be going further AND providing ourselves with more flexibility if we need to change the second color in the future.

So, with the base layer, add a new layer. Now create your selection. In order to create our layer mask, we will first save your selection to a channel.
10-11-2008 9-44-48 PM.png

A channel is a special type of layer which is in grayscale that can be used to create a layer mask. Creating channels is a way to apply the exact same layer mask to multiple layers quickly. Once you have created the channel remove the selection, fill the new layer entirely with your secondary color. Now, in order to get our image to look like the original reference, we will add the layer mask by right clicking the top layer and selecting add layer mask from the menu.
10-11-2008 9-58-10 PM.png


From the popup window, select the Channel radio button and select the channel you just created.
10-11-2008 9-59-51 PM.png

So why are we going to all this work? Because now, we can change either color simply by selecting the layer and using the paint bucket tool. Likewise, you could fill with a texture also. With the first technique(direct painting) you have TONS of work to do a simple color change. With the second technique(two layers) you would have to select the painted image on the second (top) layer and then fill with the new color. Not hard by any means, but not as flexible as if using masks, which you will see shortly.

I have reached the posting limit of 5 images, so I will briefly explain why using masks is better, before I explain how in the next post. Imagine using the above two layers but with some type of irregular selection, AND in addition to wanting to change the color, you want to expand or contract the size. Not hard if you want to maintain the exact same shape/diminsions, but what if you want to change the shape. Then you have some annoying tedious work to do. This is where layer masks will save the day, as we will see in the next post.

[jfrazierjr]

The final image result now looks like the first attached image(which looks like the first image in the original post).
10-11-2008 10-29-47 PM.png

Not that sexy eh? Rather boring right? True dat.

Now, let's begin to play with our existing layer mask to see what we can do with it. The good thing about layer masks is that you can edit them by "painting" either white or black to edit how the underlying layer shows up. Remember white shows through and black does not with anything in between being partially shown. So... let's get some variable color to show through and get rid of the hard line transition between the two colors.

10-11-2008 10-36-41 PM.png


In order to do this, Select the layer mask on the layer in question:

10-11-2008 10-39-16 PM.png

Note than when selected the thumbnail, either the layer OR the layer mask will have a slight white outline. In this case, we have the layer mask selected, so any operation we do will affect the mask, NOT the layer. What we now want to do here is apply some Gaussian Blur to get the two colors to blend. I used around 300 px blur to get to the final image.

Now, remember since we are not affecting the layer itself, this is a totally non destructive technique. Had we used the same technique as in the original post's second example, and blurred the color/alpha layer, then that layer would be permanently altered, and thus we would loose a lot of flexibility. However, since we just modified the layer mask and not the original layer, we have the blur but with no changes to the underlying layer.

You could do all sorts of other things such as directly painting white onto the layer mask, which would allow the layer to show though.

ONE THING THAT IS VERY IMPORTANT: When you want to modify actual layer the make SURE you have clicked on the layer thumbnail icon, or your will be modifying the mask. Likewise when you want to modify the layer mask itself.

You could also use a layer mask to create a gradient, again, without using any destructive techniques. In order to do this, I am going to delete the existing layer mask and add a new one. However, this time, I will select the "Black" radio button. Now, select the layer mask and change the foreground/background colors back to black/white. Click on the layer mask in the Layers dialog and select the Gradient tool. Now, apply the gradient to the layer mask. Note how the layer mask

10-11-2008 11-01-40 PM.png

is not a gradient and how the base image follows that gradient to merge the two color layers? This is the effect we are after. Now, again, we can change either color layer directly and the gradient effect would remain exactly as it is now, just with different colors, and that is the power of layer masks.

[Badger]

getting five stars from me !!

[jfrazierjr]

Now, what if you want to get some more control over the actual layer mask? How can I see the actual layer mask? WE CAN DO THAT.
10-11-2008 11-16-15 PM.png

On the layer, right click and select show layer mask. This shows the actual layer mask as seen in image two. Now, any changes you make here will show up in the layers, just like before.
10-11-2008 11-17-29 PM.png

What if you want to see the image without the layer mask? You can temporarily disable a layer mask if you like.
10-11-2008 11-21-40 PM.png

Likewise, if for whatever reason, you no longer need the layer mask, you can delete it also from the right click menu. There is also a menu option called "Apply Layer Mask".
10-11-2008 11-27-36 PM.png

What this does is merges the layer mask directly into the layer.
10-11-2008 11-30-12 PM.png
See how the layer mask is gone, BUT the effect is blended into the underlying image on that layer?

I tend to NOT like this since it is a permanent change to the underlying layer and thus we loose our flexibility which is the reason for using the layer masks in the first place.

[jfrazierjr]

There will be a bit more to come over the next day or so, but I hope this is more than enough to get anyone started with using layer masks. What I have presented the real "meat" of using layer masks and why they are so darn useful.

If anyone wants to ask any questions or if there is anything you don't understand(or want to add) please feel free.

[bryguy]

Whoah dude, this is awesome. Ive always wondered how to use layer masks, and so hopefully Ill be able to now (well, after I actually read it. Right now I just skimmed cause Im tired and off to bed)


5 stars!

[jfrazierjr]

Whoah dude, this is awesome. Ive always wondered how to use layer masks, and so hopefully Ill be able to now (well, after I actually read it. Right now I just skimmed cause Im tired and off to bed)

Thanks, that's exactly why I wrote it, for people who have so far been intimidated or just have not taken the time to learn how layer masks work. I hope this really helps you, and remember, ask questions if you need to. One thing I have found that this technique is really good for is blending various types of photo realistic texture fills. Check out my Some Creek thing for a very simple example. http://www.cartographersguild.com/sh...ighlight=creek

[xv43]

This is good, thank you.

[jfrazierjr]

Hmmm... I am kind of surprised with the lack of response on this. Is this because people already know this information, no one is looking at this(in which case, I can understand the lack of answering!), it's non understandable yet or you don't see any application in your mapping technique, or is everyone just using PS and don't think this applies to you? (it does, but in a slightly different way).

[mmmmmpig]

well, I for one did not respond because all it would have been was a response that said "Huzzah and kudos." That is why I did not respond.

The application of this is sooo amazingly powerful and versitile. One could easily use this with landform textures and blending them. Find a nice pattern fill for rolling hills and a pattern texture for mountains and a pattern fill for mountains and have at it.

ummm.... Huzzah and Kudos, by the way