Pure 64 bit

I did a brief search but nothing came up. Is it possible to get rid of 32 bit libraries? I run a bunch of Gentoo boxes, all pure 64 bit, there really is nothing in FOSS that requires 32 bit support.
There really is no need. I think you're looking much to deep into 64bit. Even on 64bit it's not uncommon to use 32bit code. It's a fairytale that 64bit code will be faster and more efficient per definition, at most it can more easily address larger amounts of memory space, but even that is something you wouldn't be using anyway in your day to day applications.
Speedy , the discussion here has nothing to do with the PAE.
It's a fairytale that 64bit code will be faster and more efficient per definition
Exactly! In earlier days of x86_64 I've run many real life tests, e.g. video encoding, it was faster on a 32-bit OS than on a 64-bit. That time both AMD and Intel suggested using their math libraries to take advantage of the 64-bit systems. Of course, perhaps now the things are changed, but I'm still doubting that there is big (if any) difference in performance. Yes, we need to address more memory, but do we really need to operate 64-bit numbers on a daily basis?..
Most (if not all) of the 64 bit instructions execute in the exact same amount of clock cycles as their 32 bit counterparts. Which means it's just as fast. Things are slightly different when it comes to FPU operations though, certain FPU operations are significantly faster.
This misses one important point which I also missed until I was hit with the proverbial clue-by-four - some non-64-bit processors that support PAE also support execute disable, an important security feature. Older processors with PAE lack 64-bit support and PAE is the only way to get execute disable on them.
Tell me now FreeBSD is an exception and can use effectively that RAM over gigabyte. For me building 32 bit support is building cruft. There is nothing wrong with my Gentoo boxes which have no multilib support.
Not sure where "over gigabyte" came from. An x86 / non-PAE kernel can address 3.mumble gigabytes of memory. The exact value depends on the installed hardware, chipset, BIOS, etc.

An x86 / PAE kernel can address all of the memory it is possible to put in the box (for reasonable values of "all") and gets you execute disable support on at least some processors. This is a useful solution for a certain set of users who have non-64-bit x86 processors (this is apparently still common in the embedded space).

An amd64 kernel can address all of the installed memory without the convolutions required by PAE, as well as obtaining a variety of other possible speed improvements. In the amd64 FreeBSD environment it is possible to both build and execute 32-bit applications. I have a source license for some ancient commercial software that I've been dragging around since 4.2BSD on a VAX, and it still compiles / runs fine on FreeBSD 10.3 amd64. There have been a few edits along the way, like supporting Chris Torek's new (in 1990!) stdio. The code is old and crufty enough that it simply will not build as a 64-bit application. It needs -fwritable-strings and other disgusting things.