SMB speed issues and how to solve them?

Hey all,

I have put together a home media server with 16 drives, and using zfs as my choice of file systems. I built a test system previously and it was working well but the new system is not as trouble free. I have been copying to it from a Win7 box and I get 20-30MB/sec from that. Using that same Win7 box I get 100MB/sec to a Thecus NAS box which has FEWER drives in it.

For giggles I took some spare hardware and built an ubuntu box specifically so I could run iperf on it.

From the ubuntu (as a client) to the freebsd FreeBSD (as a server) I get this:
Code:
Client connecting to 192.168.3.31, TCP port 5001
TCP window size: 0.02 MByte (default)
------------------------------------------------------------
[  3] local 192.168.3.21 port 46696 connected with 192.168.3.31 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1094 MBytes   109 MBytes/sec
So it appears that's pretty damned solid.

However, even copying data from that box to the FreeBSD system is slow.

I ran a gstat to see what the system was doing during a copy. I did an rsync between the FreeBSD box and the Thecus NAS. The drives would sit at 80%+ busy for a bit and then go to zero for several seconds. Of course this meant that the files (approximately 5000 8MB files) stop copying as well. It resumes easily enough but there is clearly some kind of bottleneck. I have a feeling like there is something in the Samba config that's causing trouble. I am doing all the testing via SMB because that's how I am going to connect the windows box to it.

Any ideas and help are very much appreciated.

Rob
 
Rebuild samba and turn on WITH_AIO_SUPPORT. That should improve things.
 
SirDice said:
Rebuild samba and turn on WITH_AIO_SUPPORT. That should improve things.

Thanks for this. I added
Code:
## add AIO support
aio_load="YES"
to my /boot/loader.conf.

Where do I need to set the setting to turn WITH_AIO_SUPPORT to on?

Sorry for the hand holding needed here. I will learn however :)

Rob
 
Sort of answering my question:
Code:
cd /usr/ports/net/samba36
make install clean
Compiling now...
 
throb said:
where do I need to set the setting to turn WITH_AIO_SUPPORT to on?
Code:
cd /usr/ports/net/samba36
make config
# Turn on AIO_SUPPORT
make install clean

Unfortunately this option can only be turned on using the port.
 
I had to do a make deinstall followed by a make reinstall then I did the make config and make install clean, rebooting now.

Thanks SirDice!
 
Boom! Why is this not enabled by default? What a huge difference it makes!

Many thanks SirDice
 
Yes, it makes a huge difference :e

I think the reason it's not on by default is because it fails to build on FreeBSD 7.4 (which is still a supported version).
 
Thanks jnbek for that. Will know for next time. :) I am building another box soon so this comes in handy.

It's a shame that the FreeBSD > 7.4 distributions don't come with the configuration set to turn it on. Seems logical to me but hey, I don't build distributions :)
 
There is no 'dist'. All versions of FreeBSD use the exact same ports tree.
 
Ah ok.

Oh, I have been watching my rsync and definitely there is a speed increase. I of course also have a gstat running and utilization is better there too. I still get a pause after ~2 seconds of writing to the pool (approximately a 1 second pause) and then writes begin again.

AIO was a good thing for sure. Where else can I look to try to ferret this out? It's frustrating me to not know how to do this :)

Once again, thanks so much for your help!

Rob
 
You can try setting aio read size and aio write size in smb.conf.

I'm not sure if these are good for you but I have these in smb.conf:
Code:
   socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
   aio read size = 1
   aio write size = 1
 
Ok, set that. I will keep an eye on this to see how this affects things.
It's feeling better so hopefully this sorts it.

thanks!
Rob
 
Here is my pool setup :


Code:
        NAME             STATE     READ WRITE CKSUM
        storage          ONLINE       0     0     0
          raidz2-0       ONLINE       0     0     0
            gpt/pool_a1  ONLINE       0     0     0
            gpt/pool_a2  ONLINE       0     0     0
            gpt/pool_a3  ONLINE       0     0     0
            gpt/pool_a4  ONLINE       0     0     0
            gpt/pool_a5  ONLINE       0     0     0
            gpt/pool_a6  ONLINE       0     0     0
            gpt/pool_a7  ONLINE       0     0     0
            gpt/pool_a8  ONLINE       0     0     0
          raidz2-1       ONLINE       0     0     0
            ada8         ONLINE       0     0     0
            ada9         ONLINE       0     0     0
            ada10        ONLINE       0     0     0
            ada11        ONLINE       0     0     0
            ada12        ONLINE       0     0     0
            ada13        ONLINE       0     0     0
            ada14        ONLINE       0     0     0
            ada15        ONLINE       0     0     0

2 vdevs (raidz2) with 8 drives in each.

Rob
 
Back
Top