The way I do it is not mathematically based. Usually when you are trying to increase DOF, there is something close and something far you want both equally in focus. I will refer to them as foreground and background.
1. Start with no tilt.
2. Compose the scene how you wish.
3. Using the viewfinder or liveview, adjust the tilt toward the foreground until it and the background are equally in focus (or out of focus). You will have to adjust the angle of the camera body to adjust you composition as well.
4. Adjust the focus to maximize the sharpness of the foreground and background.
5. Tweek the tilt to maximize the sharpness of the foreground and background.
6. Tweek the focus to maximize the sharpness of the foreground and background.
Sometimes it is not possible to get the foreground and background in focus with the composition you want. You can tweek this by moving forward and back and up and down from the foreground. It takes practice and a feel for the lens.
7. Pick an appropriate aperture for the desired DOF.
8. Take the picture.
Tilt isn't a miracle solution for creating DOF; it does have it's limitations, as you can see in the picture I've attached.
In this case it was very windy, and I had no tripod, so I needed a fast shutter. The flowers in the foreground are near the macro limit of the lens (TS-e 24mm II).
