2 MB Page Size

Hi,

I am exploring the idea of supporting a page size of 2 MB in amd64. If I am not wrong, the maximum page size supported is 64 KB. I do know that FreeBSD supports 2 MB as a superpage, but I am more interested in providing 2 MB pages at the core. Please let me know if anyone has implemented a 2 MB page size in FreeBSD for amd64.

Re: 2MB Page Size

Superpages are supported since version 7.x.

Hi,

I agree that superpages are supported from 7.x, but, as I mentioned, I would like to change the base page size from 4 KB to 2 MB in the amd64 architecture. I need help for the same.

nmath said:

Hi,

I am exploring the idea of supporting a page size of 2 MB in amd64.

Click to expand...

This is already supported.

If I am not wrong, the maximum page size supported is 64 KB.

Click to expand...

Where did you get such impression? Sizes have to be supported by hardware first and in case of amd64 this is 4 KB, 2 MB and 1 GB.

I do know that FreeBSD supports 2 MB as a superpage, but I am more interested in providing 2 MB pages at the core. Please let me know if anyone has implemented a 2 MB page size in FreeBSD for amd64.

Click to expand...

What is the difference between supporting and supporting at the core? Perhaps you mean support in the VM subsystem for the former and support for such size for a given architecture in the former?

As was mentioned, this is supported already.

Hi,

In sys/amd64/include/param.h, to increase the page size if we modify #define PAGE_SHIFT 12 (4 KB base page size) to #define PAGE_SHIFT 21 (2 MB base page size), the code does not build. There are lot of compile time checks inserted in the code that fails for PAGE_SHIFT =21.

Let me explain what I am trying to explore. Let us look at Intel IA-32e 64 bit virtual addressing mechanism for 4 KB:

• Bits 0 - Bits 11 identify offset within a page
• Bits 12 - Bits 20 reserved for holding PTE (page table entry)
• Bits 21 - Bits 29 reserved for holding PDE (page directory entry)
• Bits 30 - Bits 38 reserved for holding PDPE (page directory pointer entry)
• Bits 39 - Bits 47 reserved for holding PML4 (PML4 entry)
• Bits 48 - Bits 63 reserved for sign extended

When my page size is increased to 2 MB, the virtual addressing mechanism will not have a Page Table Entry and it looks like this:

• Bits 0 - Bits 20 identify offset within a page
• Bits 21 - Bits 29 reserved for holding PDE (page directory entry)
• Bits 30 - Bits 38 reserved for holding PDPE (page directory pointer entry)
• Bits 39 - Bits 47 reserved for holding PML4 (PML4 entry)
• Bits 48 - Bits 63 reserved for sign extended

I hope, I have explained what I am attempting at.

Also, can you throw some light on how is 2 MB and 1 GB is implemented with respect to code as I don't see any part of the code which addresses base Page size anything more than 4 KB. In fact after digging further deep in the code, I have come across certain values which are hard coded specifically for 4 KB page size.

Thanks.

You probably don't want to force all pages to 2 MB (which is what you seemed to suggest originally). This will waste a lot of memory for small allocations probably to the point of making your system unusable.

Transparent large pages work via a reservation system. When a page fault occurs (or during prefaulting in mmap(), the VM system can choose to reserve a large page (2 MB) in physical RAM if it needs a new page and the containing 2 MB of virtual address space is suitable (i.e. no other pages backing it yet). If the application subsequently faults in all of the surrounding pages in that large page (and if they all have the same properties, e.g. all are clean, or all are dirty), then the mapping is promoted to a large page by adjusting the PDE to point to the 2 MB page directly instead of a page table. However, this only happens if all the pages are used. If an application doesn't use all of the pages, then later if the kernel is in a low-memory situation it can break a reservation and reuse those unused pages from a 2 MB page without impacting the application. Currently, FreeBSD/amd64 only supports two page sizes for user pages: 4 KB and 2 MB. A special part of the kernel address space called the direct map will use 1 GB pages on CPUs that support it, but 1 GB pages are not otherwise used. Current CPUs tend to have very few 1 GB entries in their TLBs, so if you only used 1 GB pages you would probably see many more TLB entries. This was true even of 2 MB pages in early amd64 CPUs. Even now you generally want a mix of 2 MB and 4KB pages since modern CPUs have separate TLBs for different page sizes.

You can find more details about this in the Design & Implementation of FreeBSD 10 book.

Nice to be reminded that the book is available. I checked Amazon and was a little shocked when looking at the price tag. I need to think a bit about this...

Yeah, it's priced like a college textbook. Kirk gave a good summary of how superpages works in a talk at EuroBSDCon 2009, but I can't find a copy of that online now. I don't know of another good summary outside of the book.

jhb@ said:

Yeah, it's priced like a college textbook.

Click to expand...

Which usually means it's expensive. If you're a college/university student with a budget for books it doesn't usually matter much, but if you're not a student (or scientist) prices of such books can be rather steep. Sad but true I'm afraid.

fonz said:

Which usually means it's expensive. If you're a college/university student with a budget for books it doesn't usually matter much, but if you're not a student (or scientist) prices of such books can be rather steep. Sad but true I'm afraid.

Click to expand...

At more than 50 Euros, it crossed the border of "As a student, I could have lived for almost a month from this". And you are right, the prices for academica books are bordering on criminal (but I wisely do not say which side of the border I mean). Just try to access some ACM literature without a university flat-rate. Some 30 year old article for about $50? Sure. Only, as a student, you have no choice. You may get a (moderate) discount, but you have to have this book.

To be precise, the price is most likely very deserved for the book. There is not so much of a market, there is a lot of work in it and I will get my copy of it. But at first, it is a little shock.