1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Optimum Block Size when dd from a hard drive to another

Discussion in 'General' started by overmind, Feb 23, 2010.

  1. overmind

    overmind Member

    Messages:
    332
    Thanks Received:
    29
    Hello

    If I want to move FreeBSD to a new hard drive with:

    Code:
    dd if=/dev/ad0 of=/dev/ad1 bs=1M
    


    What is the optimum size for "bs" ?

    If I do not use bs, and leave it default then bs will be 512 bytes and it takes forever. I've used bs=20M but I want to know what is the optimum size for bs when dd-ing from a sata drive to another and also the optimum size when dd-ing from a usb stick to another.

    (I'm not interested in other ways to move data to a new hard drive, I'm only interested in cloning.)

    thanks and best regards,
     
  2. gordon@

    gordon@ New Member Developer

    Messages:
    404
    Thanks Received:
    82
    I think you'll need to experiment to find the ideal size. I suspect that any sufficient blocksize that takes advantage of a HDD buffer will be "big enough."

    My guess would be to use something like 16M or 32M.
     
  3. fallen_fbsd

    fallen_fbsd New Member

    Messages:
    1
    Thanks Received:
    0
    My guess is also 32M on modern disks. More can be degradation.
     
  4. lme@

    lme@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    628
    Thanks Received:
    204
    And what about USB-Sticks?
     
  5. overmind

    overmind Member

    Messages:
    332
    Thanks Received:
    29
    For USB Stick I've used bs=1M, still I don't know which is right value. I guess it depends on if the stick is detected as USB1 or USB2. Anyway, using bs=1M instead of leaving it default (512bytes) improves speed alot.
     
  6. Ruler2112

    Ruler2112 New Member

    Messages:
    421
    Thanks Received:
    24
    Definitely stick with a power of 2 as a block size for optimal transfer rates. I'd guess a setting equal to the internal cache size of the drive, but that's just a gut feeling supported by nothing.

    If you're serious about finding the optimal rate for a given machine, create an image of a few hundred meg from a CD or something on your drive, copy it a few times, reboot (an important step to eliminate drive caching from the equation), and then dd if=/path/to/filecreated.iso of=/second/drive bs=nM and vary n by factors of 2. Record the results and you'll soon know the optimal block size.

    Note that the vfs.read_max kernel parameter can have a significant impact on disk performance as well, which means that you should find the ideal setting for this parameter before testing the block size. Remember to use fresh image files (that haven't been read or written in the present boot) or reboot for each test to ensure consistent results.

    I've often wondered what I should use as a block size when cloning drives for best performance. Unfortunately, it's not a simple answer and varies greatly with the specific hardware involved.
     
  7. Carpetsmoker

    Carpetsmoker Member

    Messages:
    938
    Thanks Received:
    136
    I've had good results with a bs of 16M, I get transfer speeds of about 100MB/s as measured by iostat.
     
  8. mamalos

    mamalos Member

    Messages:
    435
    Thanks Received:
    44
    The fastest results I ever had were always with bs=512k. You should try all of them by letting dd run for a few seconds and then compare the outcome :)
     
  9. overmind

    overmind Member

    Messages:
    332
    Thanks Received:
    29
    and default bs size is 512 bytes and not 512k right?
     
  10. mamalos

    mamalos Member

    Messages:
    435
    Thanks Received:
    44
    so says the man page.
     
  11. sampablokuper

    sampablokuper New Member

    Messages:
    1
    Thanks Received:
    0
    I wrote dd-opt, a small utility with the sole function of helping to find the optimal setting for the bs parameter.

    Since I'm not a FreeBSD user myself, I'd welcome people trying it out on FreeBSD and submitting patches/refinements/suggestions/etc.
     
  12. rfg

    rfg New Member

    Messages:
    12
    Thanks Received:
    0
    Where is the sources tarball for that?

    I looked but could not find it.
     
  13. wblock@

    wblock@ Administrator Staff Member Administrator Moderator Developer

    Messages:
    11,512
    Thanks Received:
    2,213
  14. rfg

    rfg New Member

    Messages:
    12
    Thanks Received:
    0
    I am in the habit of arriving fashionably late to every party.

    Sorry.