8.0 and samba performance on ZFS

W

wonslung

after upgrading to 8.0, i've been noticing much improved samba performance.

I was wondering if anyone else has seen this...in 7.2, it worked fine but my HTPC's would have something like a 3-10 second lag playing a video, browsing a share for the first time or returning from a long idle period.


now i see no such lag, and ZFS filesystems are much improved.



my setup is as follows:

FreeBSD 8.0 amd64
intel q9550 8gb ram
3 raidz vdevs (4x1TB drives each)


3 htpc's build on zotac ionitx atom 330 /nvidia ion platform

1 older htpc and 2 vintage xbox's running xbmc


I have no issues running the 3 zotacs at 1080p at the same time.

I didn't have any issues in 7.2 as far as playback was concerned, but i did see these menu lags and initial "first connnect" lag.
 
I've only updated one server to 8.0 so far (1 CF disk is 7.2, the other 8.0, with the ZFS pool still at v6 with all ports still 7.x binaries). Haven't upgraded the pool yet, or re-installed the ports (running with compat7x installed). So far, things are stable. Haven't seen any performance regressions or improvements yet. Going to run it like this for a week or two to make sure things stay stable, before upgrading/reinstall the ports to 8.x binaries and adding the 7.x CF disk back into the gmirror. :)

I'm thinking of upgrading my home server to 8.0 using a similar method (install to a second USB stick, keep the 7.2 USB stick just in case). Debating going to 64-bit as well to make better use of RAM with ZFS.
 
I was using 7-stable build back in june so it had ZFS 13 already. It was using your suggestion of compact flash drives, though i keep /var /usr/local /usr/ports and /usr/src on the zfs volumes


I went ahead and upgraded everything, but the handbook didn't have the important step of make delete-old and make delete-old-libs so, even though i THOUGHT i had finished everything right, i ended up having to do it again.



anyways, i had no problem rebuilding all ports...i just used portmaster

actually, i take that back, one port ports/archivers/star wouldn't build because ports/devel/smake is marked as BROKEN in 8.0 but after deleting it and trying again, everything else went fine.

I mainly use my nas as a DVR. I have various ways of loading it with video files (torrents, QAM feeds, other downloads) and i have 3 1080p capable htpc's

I run xbmc on my htpcs and use samba (i guess NFS would be ideal but samba is just really easy to set up in xbmc and works well enough)

anyways, with 7.2 i'd notice a slight lag when trying to connect to the nas, or if it went idle for awhile, it would lag trying to reconnect.

I never had a problem with playback so i just put up with it. With 8.0, this lag is gone. I've also noticed that unrar of files is much faster on 8.0, copying files from osx to the nas over samba is faster as well.

I'm not sure if this is due to the upgrade in samba from 3.3.7 to 3.3.9 OR the upgrade from freebsd 7.2 to 8.0

either way, it's performing much better...

I'm interested in knowing if anyone has tried samab4 yet...i noticed it's available now.
 
Jumpy

I'm having trouble with what I think is samba write performance related. I have three machines, 1 FreeBSD, 1 Linux and 1 Windows 7. When transfering large files (10 GB) to my Linux machine I get fairly stable and constant speeds of around 50MB/s. The same can not be said for my FreeBSD machine. Speeds are on about avarage 30MB/s, but very inconsistent as shown by these screenshots, http://bayimg.com/HAKFpAaCG and http://bayimg.com/IaKfAAaCg

Code: 
# dmesg | grep CPU:
CPU: Intel(R) Core(TM)2 Quad CPU    Q9550  @ 2.83GHz (2833.02-MHz K8-class CPU)

# dmesg | grep memory
real memory  = 8589934592 (8192 MB)
avail memory = 8264204288 (7881 MB)

# uname -rm
8.0-RELEASE-p2 amd64

# zpool create tank raidz ad{6,8,10,12,14}
# zpool status
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            ad6     ONLINE       0     0     0
            ad8     ONLINE       0     0     0
            ad10    ONLINE       0     0     0
            ad12    ONLINE       0     0     0
            ad14    ONLINE       0     0     0

errors: No known data errors

# cd /tank/

# dd if=/dev/zero of=zerofile bs=1G count=10
10737418240 bytes transferred in 43.563099 secs (246479670 bytes/sec)

# dd if=zerofile of=/dev/null bs=1G
10737418240 bytes transferred in 29.468526 secs (364369029 bytes/sec)

# bonnie -s 8192
              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         8192 112505 65.4 166915 37.9 126788 30.8 192933 91.0 364150 38.9 134.3  0.8

Just grabbed from top to show memory usage:
Mem: 18M Active, 58M Inact, 2307M Wired, 288K Cache, 827M Buf, 5538M Free

# portmaster -l | grep samba
===>>> samba34-3.4.5_1
 
SirDice said:
http://xkcd.com/612/ :e

The sysctls mentioned in this thread seemed to do the trick for me. I also turned on AIO and configured samba to use it. I don't have ZFS but I did have network performance issues.
Click to expand...
I tried those values, but I didn't see any improvement whatsoever. I also enabled AIO and have tweaked various socket options, but all to no avail. It would be really great to get Samba to max out my gigabit ethernet.
 
you may have some other underlying issue then. If you are using ZFS, you should make sure that your issue isn't with ZFS first. It took me some tuning to get my ZFS and networking to play nice, but once i did i was able to max out gigabit.


Another MAJOR thing to avoid (and i've seen this come up on quite a few occasions) is samba logging. disable it completely. I've seen it reduce performance 10 fold for each log level.
 
wonslung said:
It took me some tuning to get my ZFS and networking to play nice, but once i did i was able to max out gigabit.
Click to expand...
Could you please post how you tuned ZFS and your general network? I've been messing with various kinds of options but can't seem to get it any faster.
 
I've sense moved this machine to OpenSolaris. I wanted to use dedup.


I am not 100% sure of all the settings i had in my sysctls

I would guess you may be having other issues.

what type of hardware are you dealing with? are you sure the underlying disks can write fast enough to saturate gigabit?


I know with AOI and the settings i've mentioned either in this thread or in others, i had no issues with samba.

Though it might make a difference that i have decent hardware and a lot of disks.
 
wonslung said:
what type of hardware are you dealing with? are you sure the underlying disks can write fast enough to saturate gigabit?
Click to expand...
If you take a look at my first post in this thread, you will see that the speed of the disks is sufficient. I have a ASUS P5Q Premium motherboard with a Q9550 processor and 8 gigabytes of RAM.
 
dobby said:
If you take a look at my first post in this thread, you will see that the speed of the disks is sufficient. I have a ASUS P5Q Premium motherboard with a Q9550 processor and 8 gigabytes of RAM.
Click to expand...

Well then, i honestly don't know why you are having trouble. I have the same cpu and same ammount of ram and have no problem.
 
I replaced my network card with one from Intel (1000 PT server adapter) and now the performance is much better and stable.

Though, when I have large files transferred to the FreeBSD server via Samba I see that about every 5 seconds the network output on the Windows machine suddenly drops to zero, and stays there for a couple of seconds. This is at exactly the time when the data is being flushed to disk by ZFS. Since the disks are faster than my network and I have lot's of memory, a constant and even flow should be possible.

Is this something with the ZFS write delay and should I tweak the synctime or what?
 
You must log in or register to reply here.
Back
Top