GTCS Game Engine:
Stencil Tutorial
You can try stencil effects in the UI demos here.
Introduction
In this tutorial, we will look at how to use a stencil with renderables.
Stencil
There are 4 functions needed in order to stencil.
First thing to note is that the stencil uses the alpha channel of the given renderable. Any white area in a sprite's alpha channel will be drawn while any black area will not be drawn.
To start, we will need to clear the stencil buffer using clearStencilBuffer. This can be called before or after beginDrawToStencilBuffer.
In order to stencil, we need to draw a renderable to the stencil buffer by calling calling beginDrawToStencilBuffer, then calling the renderable's draw function, and then endDrawToStencilBuffer. Any renderable whose draw call is between begin and endDrawToStencilBuffer will only be drawn to the stencil buffer and will not appear on screen.
To stencil a renderable, we use beginStencilCulling and endStencilCulling. Any renderable whose draw call is between these two functions will be stenciled based on the renderable(s) drawn into the stencil buffer. In order for a renderable to be stenciled, it must be called between these two functions.
Important Note: be sure to use the functions beginDrawToStencilBuffer and endDrawToStencilBuffer together as well as beginStencilCulling and endStencilCulling.
Conclusion
Stenciling allows for having renderables only show within a given area designated by the stencil renderable's alpha channel. The area that is drawn to remains separate from other objects allowing for it to be independently manipulated. This is used in multiple UI elements such as the Button and Bar.