GELI performance on Alix board with crypto hardware, vpn1411

O

overmind

Hello

I've just installed FreeBSD 7.1 on a PCEngine Alix board with a Soekris minipci vpn1411 (hifn) card to accelerate crypto.

The card is detected properly, still I have no improvments in performance when using crypto hardware.

Any ideea what should I look for / tune?
Code: 
hifn0 mem 0xe0080000-0xe0080fff,0xe00c0000-0xe00c1fff,0xe0100000-0xe0107fff irq 9 at device 12.0 on pci0
hifn0: [ITHREAD]
hifn0: Hifn 7955, rev 0, 32KB dram, pll=0x801<ext clk, 4x mult>

GEOM_ELI: Device da0s1g.eli created.
GEOM_ELI: Encryption: AES-CBC 256
GEOM_ELI:     Crypto: hardware
(I've checked, everything is setup ok, I've tried with compiled kernel or modules loaded at boot)

When I try to copy via ftp from that box, from encrypted partition I get 2 Mbytes/s transfer rate (that encrypted partition is on a SATA hard drive attached to Alix board).

I am looking to achieve speed of 8-10 mbytes/s, if possible.

best regards
 
I don't think it's problem with geli.
I bet it's because of bandwidth limitations or something like that.
Try simply copying files from one place to another (even better from 1 disk to another) and check speeds using gstat


btw i was copying files from one encrypted drive to another (one sata, other ide) with about 14 to 18 M/s.
And i don't have crypto hardware.
 
overmind said:
Hello

I've just installed FreeBSD 7.1 on a PCEngine Alix board with a Soekris minipci vpn1411 (hifn) card to accelerate crypto.

The card is detected properly, still I have no improvments in performance when using crypto hardware.
Click to expand...

I don't think that is the bottle neck. Here I don't see any difference with or without crypto acceleration.

A test with dd(1) will be more accurate than a transfert via ftp.
 
I've try to copy via ftp to an unencrypted partition, on the same alix board and I have 8 Mbytes/s.

That's right I also, on a dual core PC with no hardware for encryption I get from 14 to 18 Mbytes/s. (from 40 Mbytes/s with unencrypted partitions).

I want to achieve with that hardware encryption on Alix board the same speed as it were an unencrypted partition,

Geli seems ok, I do not get any error, I think still I should have more speed with that hardware since network card is 100 mbps, and copy to unencrypted partitions is at 80 mbps.
 
overmind said:
Geli seems ok, I do not get any error, I think still I should have more speed with that hardware since network card is 100 mbps, and copy to unencrypted partitions is at 80 mbps.
Click to expand...

What is your sectorsize in your geli partition (see geli init)? You could increase it to 4096 (the size of a page). A quick test shows that the performance is around x2 between a size of 2048 and 4096 in hifn(4).

You can also use the AES engine of the geode processor but it can do only AES-128.
 
Sector size for geli partition is 512 (default size, i think).

I tried to enable AES engine on geode processor, by loading glxsb driver.
(please note that i have aes 256)

When I tried to copy via ftp from that patition it started at 8 Mbytes/s which is fine then droped to 4 mbytes then stalled. I've rebooted the machine and I am fscking it for some time now.

Somehow glixb driver affects performance but it might be in conflict with hifn driver? (It should not work because my encription is aes256 and glxb is for aes 128.

I am waiting to finish fsck to do some dd tests (as previously suggested)

Can I increase sector size without reformatting the partition (I guess not)?
 
i bet you don't have max speed when transferring files over ftp even if you don't use geli


also try setting these
Code: 
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535
net.inet.udp.recvspace=65535
net.inet.udp.maxdgram=57344
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535
kern.ipc.nmbclusters=65535
in /etc/sysctl.conf

explanation:
Code: 
[B]net.inet.tcp.sendspace:[/B] Maximum outgoing TCP datagram size
[B]net.inet.tcp.recvspace:[/B] Maximum incoming TCP datagram size
[B]net.inet.udp.recvspace:[/B] Maximum space for incoming UDP datagrams
[B]net.inet.udp.maxdgram:[/B] Maximum outgoing UDP datagram size
[B]net.local.stream.recvspace:
net.local.stream.sendspace:[/B]
[B]kern.ipc.nmbclusters:[/B] Maximum number of mbuf clusters allowed
 
Thank you. I will try your settings right after fsck ends and I will be able to mount the encrypted partition (I hope :D)

Is strange that it seems like fsck is not doing anything:
Code: 
last pid:  1162;  load averages:  0.00,  0.00,  0.00  up 0+03:06:34    16:02:33
30 processes:  1 running, 29 sleeping

Mem: 25M Active, 976K Inact, 12M Wired, 1804K Cache, 34M Buf, 199M Free
Swap: 700M Total, 700M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
  965 root        1  -8    0 31812K 12016K physrd   0:01  0.00% fsck_ufs
  642 root        1  44    0  5876K  2296K select   0:00  0.00% sendmail
And fsck is at:
Code: 
fs# fsck -t ufs /dev/da0s1g.eli
** /dev/da0s1g.eli

***** FILE SYSTEM STILL DIRTY *****
** Last Mounted on /usr/home/fileserver
** Phase 1 - Check Blocks and Sizes
 
overmind said:
Sector size for geli partition is 512 (default size, i think).
Click to expand...
I meant the size specified when you init geli (option -s)

Somehow glixb driver affects performance but it might be in conflict with hifn driver? (It should not work because my encription is aes256 and glxb is for aes 128.
Click to expand...

Yes this is a known problem. See the CAVEAT section in the glxsb's man page.
 
I think still I have an issue with hardware driver (FreeBSD 7.1), fsck to encrypted partition was stalling.

I get 99% cpu idle and 0% for fsck_ufs.
Code: 
965 root        1  -8    0 31812K 12016K physrd   0:01  0.00% fsck_ufs
(fsck-ing for 4-5 hours now).

I've removed the hifn encryption hardware and now fsck works ok.
Code: 
702 root        1  -8    0 34884K 28432K physrd   0:34  5.27% fsck_ufs
with 30% cpu idle on Pcengines Alix 500Mhz.

As soon as fsck finish I will be able to mount the partition.

Here is info about sector size (is that sector size, or what command should I give to get info you reqested?):

Code: 
geli list
Geom name: da0s1g.eli
EncryptionAlgorithm: AES-CBC
KeyLength: 256
Crypto: software
UsedKey: 0
Flags: NONE
Providers:
1. Name: da0s1g.eli
   Mediasize: 468232700416 (436G)
   Sectorsize: 512
   Mode: r1w1e0
Consumers:
1. Name: da0s1g
   Mediasize: 468232700928 (436G)
   Sectorsize: 512
   Mode: r1w1e1

Without the crypt hardware fsck was completed in few minutes.
 
I did some tests with hifn on OpenSSL:

Code: 
time dd if=/dev/zero bs=1m count=100 | openssl des3 -pass pass:test -engine cryptodev -out /dev/null

results:
Without hardware encryption
---------------------------
Code: 
engine "cryptodev" set.
100+0 records in
100+0 records out
104857600 bytes transferred in 46.245892 secs (2267393 bytes/sec)

With hardware encryption
------------------------
Code: 
engine "cryptodev" set.
100+0 records in
100+0 records out
104857600 bytes transferred in 21.653051 secs (4842625 bytes/sec)

It works 2x with hardware (as advertised by others on mailing lists), so I think is a problem with geli+hifn.
 
overmind said:
I think still I have an issue with hardware driver (FreeBSD 7.1), fsck to encrypted partition was stalling.

I get 99% cpu idle and 0% for fsck_ufs.
Code: 
965 root        1  -8    0 31812K 12016K physrd   0:01  0.00% fsck_ufs
(fsck-ing for 4-5 hours now).
Click to expand...

hmmm. May be it is related to this PR:
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/92716

It looks like that hifn misses some interrupts. I've made some tests to see and I do not have any problem. But on my soekris box hifn shares the IRQ with the ata disk so may be this is why it works fine here.

Could you tell me the IRQ used by hifn and if the IRQ is shared whith another device?

Could try to patch hifn with the patch included in the latest comment of this PR?

Here is info about sector size (is that sector size, or what command should I give to get info you reqested?):

Providers:
1. Name: da0s1g.eli
Mediasize: 468232700416 (436G)
Sectorsize: 512
Mode: r1w1e0
Click to expand...

You should increase the sector size but you need to rebuild the geli partition (IMHO)

(I'm going on holidays, do not expect any reply until the next week.)
 
Tried that, still the same result

killasmurf86 said:
can you change pci slot?
Click to expand...

I've changed the mini pci slot (alix with 2 lan cards has 2 minipci slots) still I get the same result
 
I've tried the patch and is not working. (Still same performance and still fsck is stalling).

Irq is 12 i think (here a listing of pciconf -lv).

Code: 
hifn0@pci0:0:12:0: class=0x0b4000 card=0x00000000 chip=0x002013a3 rev=0x00 hdr=0x00
    vendor     = 'HI-FN Inc.'
    device     = '7954/7955 Cryptographic Processor'
    class      = processor

I think is a problem with Geli+that board. I've tried minipci version on pcengines alix (on FreeBSD 7.1) and I've also tried soekris PCI card on a dual core PC with FreeBSD 7.0, still the same result, in fact I've notice a decrease in performance when copy from encrypted partitions from 14 Mbytes/s to 7 Mbytes/s. Fsck still is stalling, and the computer must be rebooted.
 
Conclusion, for now

I've tried different things this weekend and conclusion is that hifn driver (using soekris vpn1401 and vpn 1411, chipset Hifn 7955)
is not working on FreeBSD 7.1 and FreeBSD 7-CURRENT.

As a test, when I copy files via ftp on an encrypted partition it locks the ftp session and then the machine must be rebooted and fsck must be used on that encrypted partition (which fsck is not working if hifn module is loaded and geli is used with hardware encryption).

Tests were made for partion encrypted withAES-128 and AES-256.

So as until now, hifn cards cannot be used with geli (even if uses crypto which should be supported/enhanced by any crypto hardware). I've done some tests for openssl with seems to work but I only tested with:

Code: 
time dd if=/dev/zero bs=1m count=100 | openssl des3 -pass pass:test -engine cryptodev -out /dev/null

and I did not tried in production.

As a motherboard a PCENGINE Alix board was used. I've also tried PCI version (not only minipci) on a PC with Intel motherboard, still I got the same results.
 
Hello,

Any news on this issue? still dont working in FreeBSD-8?

I was looking for a vpn1401 card but if it dont works its a waste of money....

Any not expensive alternative for the soekris crypto cards to accelerate GELI crypto that works out of the box?
 
everyone: I'm also interested in knowing of a good PCI encryption acceleration card that works. I'll be using GELI on a system with 2 dual-core Intel Xeon 2.8GHZ processors. I want a crypto accelerator to ensure disk IO is fast.

Any product recommendation for FreeBSD 8.0 is extremely welcomed
 
I've tried a vpn1401 (PCI card) in two different machines (both 8.0-RELEASE-p2, i386 and amd64). I've added hifn_load="YES" to the loader.conf and I saw "GEOM_ELI: Crypto: hardware" during kernel boot. But both systems stopped working when accessing the eli devices (zfs import in my case). The system works fine without the card.

I'd really like to use this card in my main server with 10 eli-disks (raidz2). If I write to this zpool the system is no longer able to "stream" MP3 via NFS, mpg123 pauses for up to 10 seconds sometimes.
 
You must log in or register to reply here.
Back
Top