Are you using a display manager or just booting into text mode then typing startx. Assuming you have the xinit package installed, just boot into text and run startx whenever you want to use it.
If you're using a display manager that boots into GUI, though, I have no idea.
