ZFS Extreme slow copy speeds ZFS

B

BLVCKRIPPER

My SMB copy speed server to client is 10 mb/s and upload to server 200 mb. My connection is 10gbits over fiber and server connected via qsfp+ 40gbit.
zpool get all
Code: 
rz8tb      size                           29T                            -
rz8tb      capacity                       74%                            -
rz8tb      altroot                        /mnt                           local
rz8tb      health                         ONLINE                         -
rz8tb      guid                           6860128263587254921            -
rz8tb      version                        -                              default
rz8tb      bootfs                         -                              default
rz8tb      delegation                     on                             default
rz8tb      autoreplace                    off                            default
rz8tb      cachefile                      /data/zfs/zpool.cache          local
rz8tb      failmode                       continue                       local
rz8tb      listsnapshots                  off                            default
rz8tb      autoexpand                     on                             local
rz8tb      dedupratio                     1.00x                          -
rz8tb      free                           7.38T                          -
rz8tb      allocated                      21.6T                          -
rz8tb      readonly                       off                            -
rz8tb      ashift                         0                              default
rz8tb      comment                        -                              default
rz8tb      expandsize                     -                              -
rz8tb      freeing                        0                              -
rz8tb      fragmentation                  14%                            -
rz8tb      leaked                         0                              -
rz8tb      multihost                      off                            default
rz8tb      checkpoint                     -                              -
rz8tb      load_guid                      14439749273001701884           -
rz8tb      autotrim                       off                            default
rz8tb      feature@async_destroy          enabled                        local
rz8tb      feature@empty_bpobj            active                         local
rz8tb      feature@lz4_compress           active                         local
rz8tb      feature@multi_vdev_crash_dump  enabled                        local
rz8tb      feature@spacemap_histogram     active                         local
rz8tb      feature@enabled_txg            active                         local
rz8tb      feature@hole_birth             active                         local
rz8tb      feature@extensible_dataset     enabled                        local
rz8tb      feature@embedded_data          active                         local
rz8tb      feature@bookmarks              enabled                        local
rz8tb      feature@filesystem_limits      enabled                        local
rz8tb      feature@large_blocks           enabled                        local
rz8tb      feature@large_dnode            disabled                       local
rz8tb      feature@sha512                 enabled                        local
rz8tb      feature@skein                  enabled                        local
rz8tb      feature@userobj_accounting     disabled                       local
rz8tb      feature@encryption             disabled                       local
rz8tb      feature@project_quota          disabled                       local
rz8tb      feature@device_removal         enabled                        local
rz8tb      feature@obsolete_counts        enabled                        local
rz8tb      feature@zpool_checkpoint       enabled                        local
rz8tb      feature@spacemap_v2            disabled                       local
rz8tb      feature@allocation_classes     disabled                       local
rz8tb      feature@resilver_defer         disabled                       local
rz8tb      feature@bookmark_v2            disabled                       local
rz8tb      feature@redaction_bookmarks    disabled                       local
rz8tb      feature@redacted_datasets      disabled                       local
rz8tb      feature@bookmark_written       disabled                       local
rz8tb      feature@log_spacemap           disabled                       local
rz8tb      feature@livelist               disabled                       local
rz8tb      feature@device_rebuild         disabled                       local
rz8tb      feature@zstd_compress          disabled                       local
root@truenas[~]#

zpool iostat

rz8tb       21.6T  7.38T  1.18K     44   545M  1.81M
THANKS ALL


Edit: SFTP also very very slow 10 MB/s



Edit: Saw this
Code: 
  pool: rz8tb
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: scrub in progress since Sun Jan 31 00:00:02 2021
        21.6T scanned at 2.56G/s, 14.1T issued at 547M/s, 21.6T total
        0B repaired, 65.52% done, 03:57:37 to go
 
  • Looks like you're running the new OpenZFS? Which FreeBSD version are you running? Samba version?
  • Maybe OT, but depending on the FreeBSD version(s) you're using on the server, IMHO you can safely enable at least feature@large_dnode & feature@spacemap_v2.
  • I do not think this is ZFS problem, because zpool iostat shows you have reasonable write speeds. There's no reason that the read speed is lower. I suspect a suboptimal setting (or bug) either on the TCP or SMB side. It might even be a misconfiguration or bug on the client side.
  • To track this down, you can create a FreeBSD jail on the server and use it as a test dummy client (with smbclient(1)). If the copy speed server to dummy is ok, it's clear that the problem is not on the ZFS side.
  • A very common error is to forget to change the settings (e.g. sysctl(8) knobs) requested by the ports installed. Excerpt from my standard disclaimer:
    You may want to add message: "query '[%C/%n] %M'", to the ALIAS section of /usr/local/etc/pkg.conf, then read through all pkg message|less and apply the requested settings
 
You must log in or register to reply here.
Back
Top