Large Memory Configuration (PAE) Kernel for FreeBSD 9.2 i386 (x86)

[sebastians]

Hello everyone,

Here's a tutorial on how to build and install a custom PAE kernel for FreeBSD 9.2.

By default, the i386 processor architecture only recognizes 4 GB or less memory. The FreeBSD kernel supports PAE as a build option. After compiling a PAE enabled kernel, the 4 GB limit will increase to 64 GB.

Recompiling the kernel is best to be done right after the OS has been installed. Nonetheless, a backup of the old kernel doesn't hurt. This will be the “good ol'” boot-able kernel, if the PAE enabled kernel doesn't want to boot. # cp -r /boot/kernel /boot/kernel.GOOD

In case you have other things to backup, now is the time to do it.

Now let's start.

• Download the src.txz archive which contains the kernel sources: # cd /tmp; fetch [url=ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/9.2-RELEASE/src.txz]ftp://ftp.freebsd.org/pub/FreeBSD/relea ... SE/src.txz[/url]
• Remove the /usr/src/ directory (if present): # rm -rf /usr/src/
• Unpack the archive to /usr/src: # tar -C / -xvzf src.txz
• Rebuild the kernel by creating a file (the custom kernel file) in /usr/src/sys/i386/conf:
  # cd /usr/src/sys/i386/conf
# touch KERNEL-PAE
# nano -w KERNEL-PAE
  Add the following:
  

  include PAE

  Optionally you can include these parameters if you want to enable traffic shaping and the PF firewall:
  

  # ALTQ
  options         ALTQ 
  options         ALTQ_CBQ        # Class Bases Queuing (CBQ) 
  options         ALTQ_RED        # Random Early Detection (RED)
  options         ALTQ_RIO        # RED In/Out
  options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
  options         ALTQ_PRIQ       # Priority Queuing (PRIQ) 
  options         ALTQ_NOPCC      # Required for SMP build (SMP support for ALTQ)
  # ALTQ 
  device pf                       # enables PF support 
  device pflog                    # enables the optional pflog pseudo network device 
  device pfsync                   # enables the optional pfsync pseudo-network device

  Also you can include other configurations not present into the PAE or GENERIC kernel.
• Before proceeding into building the custom kernel please take a look at /usr/src/sys/i386/conf/NOTES and /usr/src/sys/i386/conf/PAE for more information.
• Compile and build the custom kernel: # cd /usr/src; make buildkernel KERNCONF=KERNEL-PAE
• If case of no errors, you can install the custom kernel: # cd /usr/src; make installkernel KERNCONF=KERNEL-PAE
• Reboot the machine.

Before having the PAE kernel the dmesg command showed 4 GB of RAM:

[CMD=#]dmesg[/CMD]
... output ommited ... 
real memory  = 8589934592 (8192 MB) 
avail memory = 3928604672 (3746 MB) 
... output omitted ...

After installing the custom kernel with the PAE feature, the dmesg command shows all the memory installed on the machine:

[CMD=#]dmesg[/CMD] 
... output omitted ... 
real memory  = 8589934592 (8192 MB) 
avail memory = 8135839744 (7758 MB) 
... output omitted ...

If something goes wrong and the PAE kernel doesn't want to boot, reboot the machine and at the Boot Menu select option 6, "Escape to loader prompt". At the "Ok" prompt, execute boot /boot/kernel.GOOD/kernel. It should now boot into the GENERIC kernel.

 

[kpa]

Be aware that the use of the PAE option does not increase the amount of memory usable by a single process any higher than without the PAE option. I think it's 2 gigabytes on i386. This is unlike the MS Windows PAE implementation where programs can be specially compiled with the PAE option to address 3 gigabytes of virtual memory for their own use instead of the default 2.

Reference: The PAE(4) manual page.

 

[sebastians]

Thank you @kpa for pointing that out. Also the handbook states some limitations of the PAE kernel:

PAE support in FreeBSD has a few limitations:

• A process is not able to access more than 4 gigabytes of virtual memory space.
• Device drivers that do not use the bus_dma(9)(9) interface will cause data corruption in a PAE enabled kernel and are not recommended for use. For this reason, a PAE kernel configuration file is provided in FreeBSD which excludes all drivers not known to work in a PAE enabled kernel.
• Some system tunables determine memory resource usage by the amount of available physical memory. Such tunables can unnecessarily over-allocate due to the large memory nature of a PAE system. One such example is the kern.maxvnodes sysctl, which controls the maximum number of vnodes allowed in the kernel. It is advised to adjust this and other such tunables to a reasonable value.
• It might be necessary to increase the kernel virtual address (KVA) space or to reduce the amount of specific kernel resource that is heavily used in order to avoid KVA exhaustion. The KVA_PAGES kernel option can be used for increasing the KVA space.

For performance and stability concerns, it is advised to consult tuning(7)(7). pae(4)(4) contains up-to-date information on FreeBSD's PAE support.

Reference: http://www.freebsd.org/doc/handbook/kernelconfig-config.html