Have you tried the wine-devel port on amd64? According to the latest quarterly report a lot of work has been done recently on getting the 64bit port stable. Also, what do you mean by Windows virtual machine? To run a real VM, you can use something like Virtualbox and you wouldn't actually need wine.
Alternatively you could try building PAE into your kernel so the system can see the extra RAM. I don't think any application is going to see or be able to make use of the RAM above 4GB without PAE. Of course you'll still probably only be able to give a VM ~4GB or RAM, as on a 32bit system, 32bit applications can only address a maximum of 4GB. The PAE kernel just does tricks to allow different applications to see different sections of the overall RAM in the system.