ZFS native encryption support

throAU said:
If you want encryption performance, surely you want a CPU with AES encrypton in hardware. I don't think an atom is ever likely to give you this, it's simply not the right hardware for the job.
Click to expand...

I know it's NOT the perfect hardware for the job, but it's for a home, three accesses at a time tops, and is what I have here now. I'd like it to be low power demanding, so a Soekris is great for me. I don't want to make it push GB/s, just I'm trying to look for the best way to do it on my hardware.

ondra_knezour: I have one of those on the other Soekris, but unfortunately there is no version for the net6501. And the 5501-70 has too little memory (512MB). Thanks for the idea :)

none
 
throAU said:
If you want encryption performance, surely you want a CPU with AES encrypton in hardware. I don't think an atom is ever likely to give you this, it's simply not the right hardware for the job.
Click to expand...

I agree, sounds like you are asking your hardware more than it can do!
 
fgordon: thanks. But my issue is that I already have the box.

A bit of an update. I'm trying the same config using a different machine. This time it's a server:

Code: 
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-RELEASE #0: Wed Mar 21 19:25:12 BRT 2012
    root@macgyver:/usr/obj/usr/src/sys/net6501-amd64 amd64
can't re-use a leaf (geom_label)!
module_register: module g_label already exists!
Module g_label failed to register: 17
CPU: Intel(R) Xeon(R) CPU           E5410  @ 2.33GHz (2327.55-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x10676  Family = 6  Model = 17  Stepping = 6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 34359738368 (32768 MB)
avail memory = 33056464896 (31525 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7

I have plenty CPUs and RAM (32GB). The disks are SAS 146GB. And after all I still have less them 10MB/s. A file copied from NFS got me 9MB/s.

I'm not complaining about a thing, as I said before my goal is to find the best way to use my hardware and FreeBSD and ZFS with crypto. Apart from a CPU that have AES-fu, is there any other thing that could make it faster?

Thanks,

none
 
Skip the crypto, and things will speed up *a lot*. :)

Thus, you need to find a crypto-accelerator. The simplest method being to use a CPU that includes it (Via Padlock, Intel AES-NI, etc).

Also note that GELI is single-threaded, and thus only uses a single CPU core; compared to ZFS which is multi-threaded and will use as many cores as possible for compression, dedupe, interrupts, etc.

Doesn't matter how fast your disks/RAM are, as GELI is the bottleneck.
 
GELI is not single threaded. GELI will start one thread per logical core per GEOM provider by default. GELI will scale quite nicely. Just throw away the crappy Intel Atom based hardware. My Phenom 2 X6 1090T based desktop reaches > 200MiByte/s with a four disk RAID-Z1 on top of four GELI providers on cheap 5900rpm SATA disks. GELI is the difference between 220MiByte/s and 250MiByte/s. The CPU load at full throughput is < 40% spread more or less equally over all cores. ZFS needs enough RAM and bit of CPU time. GELI needs a lot of CPU time. Neither ZFS nor the in kernel AES implementation where designed for under powered Atom boxes. The same zpool saturates the 1.3GHz AMD K10 dual-core CPU on a HP N36L at 45MiByte/s.
 
Thanks both for the info. As I said, I already have the atom box and changing is really painful. But is there a test on how the Via performs?

I need a low power server, so must be Via. The problem is to know how it performs to figure out if selling the atom and buying a Via board (that is not an easy thing in here) will pay off.

Thanks for all,

none
 
Just with regards to your server test: I don't think that Xeon includes AES-NI either, so it will be similarly slow (compared to a hardware accelerated AES device / CPU). It's a much faster CPU yes, but still running AES in software.

The AES instructions in the Core I series (for example) speed up AES by a factor of about 20-30 over doing it on a Core 2 (which itself is more powerful than an atom by quite a margin).

Using AES on your Atom may be fine for securing low-bandwidth WAN connections, but trying to run disk IO through it, it simply won't keep up - you're just asking too much of it.
 
A fellow ZFS/GELI/Atom homeserver user here...

In light of the assertions above indicating that the Atom should be junked for this workload, I've just run some simple tests, timing the how long it takes to write various things to my GELI'd Zpool.

This is by no means a comprehensive HD test, just a quick look to get some indications of the encryption overhead. I've only looked at large transfers where the onus is on encryption and writing rather than seeking.

Copying a 701MB file file from an unencrypted zpool to a GELI'd zpool takes on average between 10-15 seconds. (Both single-drive pools are identical 2.5" 5400rpm 640GB jobbies). I expect the read is at least partly from the ZFS L2ARC, but can confirm that the disc is physically active for the full duration of this test.

Writing 1GB of /dev/zero to the encrypted pool takes on average 15 seconds.

Again, I know these are not rigorous benchmarks! :)

In any case, for home use it works just fine.

I agree ZFS encryption would be very nice, and I look forward to the day that we may see it, but don't write off the alternatives too hastily.

sim
 
sim,

I got the feeling that Soekris boards was the one to blame, as its buses are unable to handle the data flowing. A test on Intel D525MW got better results (will have numbers soon), and the CPU is only 200MHz faster. I moved the file server to this board, and will finish the migration soon. Ordered some sata stuff from Internet and when it arrives I will be able to make it the file server as the Soekris were (I had a PCIe eSATA card, and this card has no PCIe - just mini-PCIe. My old Sil3124 can't handle the port-multiplier well).

I read some things from Soren (Soekris Engineer), and the bus factor got me suspicious. Still a chance for ZFS and GELI :)

Thanks,

none
 
Performance and low-power never go that well together. Depending on your idea of low-powered, new Xeon at 17W is very cool.
 
bbzz said:
Performance and low-power never go that well together. Depending on your idea of low-powered, new Xeon at 17W is very cool.
Click to expand...

bbzz: that is more then I was expecting from that box. It is a good hint for sure, thanks. But my machine draws just 10w when idle now (the soekris) and the atom board draws around 24w all board. Unfortunately I don't know any board that would be as efficient :(

thanks,

none
 
none said:
bbzz: that is more then I was expecting from that box. It is a good hint for sure, thanks. But my machine draws just 10w when idle now (the soekris) and the atom board draws around 24w all board. Unfortunately I don't know any board that would be as efficient :(

thanks,

none
Click to expand...

To be accurate, from my measurements resulted that PCEngines uses 4W of power (without hard drive but if you use a laptop hard drive just add 1W more) and my atom board (an ASrock 1.6 dual core atom, 330D) uses a lot of power: 35W, without hard drive). What CPU uses that soekris? Is it Geode 500mhz like in PCEnines?

The new Xeon uses 17W of power just for the CPU. How much power will use the entire system with?
 
Not sure but it really is low for such powerful CPU. Also 17W is max, meaning you'll be idling at much lower than that. Rest of components shouldn't take nearly as much power.
 
overmind said:
To be accurate, from my measurements resulted that PCEngines uses 4W of power (without hard drive but if you use a laptop hard drive just add 1W more) and my atom board (an ASrock 1.6 dual core atom, 330D) uses a lot of power: 35W, without hard drive). What CPU uses that soekris? Is it Geode 500mhz like in PCEnines?

The new Xeon uses 17W of power just for the CPU. How much power will use the entire system with?
Click to expand...

The Soekris I said uses an atom 1.6GHz. I measured it and with notebook HDD it draws 10w. Also measured the Intel d525mw using one notebook HDD and using picoPSU readings were 24w, normal HuntKey PSU was 36w.

Is there any board for Xeon energy efficient that is targeted for small vorm factors? A friend bought a VIA solution and said performance was great, limited by the HDD itself. He uses linux though. It's hard to find VIA hardware in here, nevermind specific Xeon configs. But it is always good to know there is the option.

Thanks,

none
 
Netbook ACER Aspire - 1,6 GHz Atom.

ZFS:
Small boot with ZFS unencrypted (1G in size, about 180 Mb used)
Rest is 150 Gb ZFS on GELI ada0p4.eli

Write is about 2 Mb/s (megabytes not megabits)
Read is little more, but at maximum 3 Mb/s.

Quite slow, but this a test setup
FreeBSD 10 i386 CURRENT - compiled world 20.07.2012

Planning to use GELI on my home server - amd64 of course
Intel Core i5, disks SATA 3
System would be 10-CURRENT updated every month by build/install world/kernel

What should I expect on MIRROR of 2 x 2TB WD disks, 16 GB 1600 DDR3 RAM
zpool create zdata mirror ada1.eli ada2.eli (with propper 4K alignment)
Root of the system, and all system data (mean all data that can be restored: sources, ports, packages etc) would be on single SSD disk, about 80Gb:
zpool create zroot ada0
?? Any estimations ??
 
I 've got 40MB/s write, 60MB/s read from my ZFS RAIDZ 4 x 1TB HDD, AMD Athlon II X2 250 Dual Core, 3000 MHz, GELI default.
HTH
 
@MorgothV8

Here is my research on GELI/ZFS. The use of aesni.ko raises throughput to native. I suggest loading that, combined with AES-CBC 128 bit for optimal performance.

GELI Benchmarks

/Sebulon
 
OK 40-60 MB/s with Athlon X2 without AES seems already quite fast - and with AES-HW I expect little more....
It is enough for me.
Thanks, when I finally try it, I'll post about results.
 
You must log in or register to reply here.
Back
Top