In small boat sailing there is a type of capsize called the deathroll, and from the name you can probably tell that it is not the most fun thing to do.
Let me start off with a picture of this looks and it will help with the of how this happens.
Here we can see that is quite windy and that the boat is also on its side, not good. This can happen when one sails in a direction called "by the lee", this is when the flow of wind on the sail becomes reversed from the normal direction.
On the right, called downwind, the wind is following across the sail from left to right, or from the mast (the big dot) to the end of the sail (the leech). This is how normal sail flow works, but when we go to "by the lee" the direction of the wind across the sail changes from leech to luff (end of the sail to the mast) as shown by the green arrow. When this type of flow exists on the sail the direction of the force of the sail also changes direction. On the left of the diagram the boat will tend towards leaning to the right, this is the same direction the force of the sail is in. On the right of the diagram the force of the sail is towards the left causing the boat to want to capsize like the first picture.
This can be explained by this diagram.
Now imagine the direction of the arrows reversed.
(My terrible paint rendition)
The big arrow represents the force of the sail and we can see that it is pointed to the side of the boat that the sail is not on, this leads to our dreaded deathroll.