Solved How to use dd correctly ?

bgroper · Apr 23, 2024

I'm trying to write a raspbx image to a micro sd card using FreeBSD version 13.3-RELEASE-p1

Code:

# lsblk /dev/da2
DEVICE         MAJ:MIN SIZE TYPE                                    LABEL MOUNT
da2              0:179  59G MBR                                         - -
  <FREE>         -:-    16M -                                           - -
  da2s1          0:182  59G ntfs                                        - -

Confirmed /dev/da2s1 is NOT mounted.
But when using dd comand, I gets an error

Code:

# ls -alh raspbx*
-rw-r--r--  1 root  user   3.7G Oct  6  2020 raspbx-10-10-2020.img

# dd if=./raspbx-10-10-2020.img of=/dev/da2 status=progress bs=4M
dd: /dev/da2: Operation not permitted

Yes, the sha1sum of the .img file checks correctly.
Please, what is the-right-way to do this ?
Thankyous for any tips or clues.

vermaden · Apr 23, 2024

micro sd card

Possibly a 'physical switch' in SD card adapter that prevents writing?

T-Daemon · Apr 23, 2024

bgroper said:
Confirmed /dev/da2s1 is NOT mounted.

Have you then elevated kern_securelevel, maybe "2" and above?

sysctl kern.securelevel

covacat · Apr 23, 2024

try
sysctl kern.geom.debugflags=16
man 4 geom

Andriy · Apr 23, 2024

bgroper said:
bs=4M

Try to use a reasonable block size.
I think that maximum physical I/O size is limited to 1MB on FreeBSD, but I could be outdated on that.
Some devices / controllers may insist on even lower block sizes.

meine · Apr 23, 2024

AFAIK dd is restricted on FreeBSD for security reasons. The command can destroy a lot if applied to the wrong volume. Below is the order I found earlier and use when needed:

Boot in single user mode as root;
cd to the directory with the image;
camcontrol devlist to see the target device;
umount -v <target_device>
dd if=<input_file> of=<target_device> bs=1m conv=sync

Remove the target device before reboot, otherwise the box might boot on that device instead of the regular system.

Eric A. Borisch · Apr 23, 2024

Andriy said:
Try to use a reasonable block size.
I think that maximum physical I/O size is limited to 1MB on FreeBSD, but I could be outdated on that.
Some devices / controllers may insist on even lower block sizes.

This. Some SD cards / adapters are very picky about what sizes they will accept writes. Start low (4k) and work your way up. (Too small will take a long time.)

tingo · Apr 23, 2024

I use

Code:

bs=4M

with dd regularly when writing usb memory stick or SD / microSD cards. For me it is the sweetspot (the dd operation finishes quickest) on many machines.

gentisle · Apr 23, 2024

bgroper said:
I'm trying to write a raspbx image to a micro sd card using FreeBSD version 13.3-RELEASE-p1

Code:

# lsblk /dev/da2 DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT da2 0:179 59G MBR - - <FREE> -:- 16M - - - da2s1 0:182 59G ntfs - -

Confirmed /dev/da2s1 is NOT mounted.
But when using dd comand, I gets an error

Code:

# ls -alh raspbx* -rw-r--r-- 1 root user 3.7G Oct 6 2020 raspbx-10-10-2020.img # dd if=./raspbx-10-10-2020.img of=/dev/da2 status=progress bs=4M dd: /dev/da2: Operation not permitted

Yes, the sha1sum of the .img file checks correctly.
Please, what is the-right-way to do this ?
Thankyous for any tips or clues.

Are you doing the command as root? sudo or doas are required for dad.

vermaden · Apr 23, 2024

Andriy said:
Try to use a reasonable block size.

1 MB is a VERY reasonable block size.

Andriy said:
I think that maximum physical I/O size is limited to 1MB on FreeBSD, but I could be outdated on that.

The limit is 2048M which is 2G.

Code:

% dd < /dev/zero > /dev/null bs=2047m status=progress
44444312059904 bytes (444 GB, 414 GiB) transferred 59.219s, 7503 MB/s
208+0 records in
208+0 records out

dd < /dev/zero > /dev/null bs=2048m status=progress
dd: stdin: Invalid argument

0+0 records in
0+0 records out
0 bytes transferred in 0.000098 secs (0 bytes/sec)

Hiroo Ono · Apr 24, 2024

covacat said:
try
sysctl kern.geom.debugflags=16
man 4 geom

Do not 'shoot your foot'. If you input wrong device name, it will bring a sad result.

The reason dd denies to write in this case is that there are MBR, slices and partition tables on the device it tries to write. Remove them and 'destroy' the geom scheme first, and run dd like below:

Code:

# gpart delete -i 1 da2
# gpart destroy da2
# dd if=foo.img of=/dev/da2 bs=4m

bgroper · Apr 24, 2024

Well, thanks so much for all the replies, and helping me take some small steps up the learning curve

vermaden said:
Possibly a 'physical switch' in SD card adapter that prevents writing?

Thanks for suggestion. I'd forgotten about those tiny switches on SD card adapters. Checked and already set to read-write.

T-Daemon said:
Have you then elevated kern_securelevel, maybe "2" and above?

sysctl kern.securelevel

Another thanks for suggestion. kern.securelevel: -1

covacat said:
try
sysctl kern.geom.debugflags=16
man 4 geom

Thanks for this clue. It removed the blockage causing dd to fail. Problem solved.

Hiroo Ono said:
Do not 'shoot your foot'. If you input wrong device name, it will bring a sad result.

The reason dd denies to write in this case is that there are MBR, slices and partition tables on the device it tries to write. Remove them and 'destroy' the geom scheme first, and run dd like below:

Code:

# gpart delete -i 1 da2 # gpart destroy da2 # dd if=foo.img of=/dev/da2 bs=4m

Thanks too for this explanation to increase understanding.

All other comments regarding blocksize also appreciated.

smithi · Apr 24, 2024

bgroper said:
# dd if=./raspbx-10-10-2020.img of=/dev/da2 status=progress bs=4M
dd: /dev/da2: Operation not permitted

From dd(1) :

Code:

Where sizes or speed are    specified, a decimal,  octal,  or  hexadecimal
       number  of bytes    is expected.  If the number ends with a    "b", "k", "m",
       "g", "t", "p", or "w", the number is  multiplied     by  512,  1024     (1K),
       1048576    (1M),  1073741824  (1G),  1099511627776    (1T), 1125899906842624
       (1P) or the number of bytes in an integer, respectively.     Two  or  more
       numbers may be separated    by an "x" to indicate a    product.

No mention of upper case being permissible, which may have been the only problem? This is different from GNU dd, which often confuses Linux refugees.

Test theory by using dd to copy some file larger than 4 MiB to another file with bs=4m, then with bs=4M. Same result?

tingo · Apr 24, 2024

Upper case is not the problem for FreeBSD dd.

Alain De Vos · Apr 24, 2024

A block size of 512M or 4M is reasonable.

bgroper · Apr 24, 2024

smithi said:
Test theory by using dd to copy some file larger than 4 MiB to another file with bs=4m, then with bs=4M. Same result?

Thanks for the suggestion smithi.
Tested and I can confirm that bs=4M does indeed work as intended.

smithi · Apr 25, 2024

tingo said:
Upper case is not the problem for FreeBSD dd.

bgroper said:
Thanks for the suggestion smithi.
Tested and I can confirm that bs=4M does indeed work as intended.

You're both right, I was incorrect; finally got access to my X200 to run some tests.

OTOH my user being in wheel and operator groups was sufficient to dd an installation .iso file to a raw unmounted /dev/da0 device without sudo or the debugflags, so it's not clear to me what the original error might have been?

smithi · Apr 25, 2024

Alain De Vos said:
A block size of 512M or 4M is reasonable.

Sure. Earlier I found up to 2g also worked, but not 4G, with conv=sync anyway.

OTOH unless using status=progress, I found no perceptible speed advantage with more than 1m blocksize.

balanga · Apr 27, 2024

smithi said:
Sure. Earlier I found up to 2g also worked, but not 4G, with conv=sync anyway.

OTOH unless using status=progress, I found no perceptible speed advantage with more than 1m blocksize.

Just wondered, is it possible to set a default blocksize?

smithi · Apr 27, 2024

balanga said:
Just wondered, is it possible to set a default blocksize?

Sigh. dd(1)

balanga · Sunday at 9:17 AM

smithi said:
Sigh. dd(1)

I don't see any way to set it. Have looked many times...

gotnull · Sunday at 12:24 PM

balanga said:
Just wondered, is it possible to set a default blocksize?

Are you talking about the bs option?

bs=n Set both input and output block size to n bytes, superseding
the ibs and obs operands. If no conversion values other than
noerror, notrunc or sync are specified, then each input block
is copied to the output as a single block without any aggrega-
tion of short blocks.

That's literally what the 7 last messages were about, so I am not sure what you are really asking though, I am not saying that in a negative way.

balanga · Sunday at 12:39 PM

I meant is there a way to set the a default bs, maybe via a variable, without needing to specify it in the command line...

gotnull · Sunday at 1:42 PM

Okay I see you mean something like a configuration file where you can define the default options in advance.
Well I am not aware of that, but I suppose you could write a little script where all those variables would be defined by yourself, but careful with dd because one mistake can be a disaster.

smithi · Sunday at 5:46 PM

smithi said:
Sigh. dd(1)

balanga said:
I don't see any way to set it. Have looked many times...

Try again. Type 'man dd', read first sentence of description:

DESCRIPTION The dd utility copies the standard input to the standard output. Input data is read and written in 512-byte blocks.
[...]

The following operands are available:

bs=n Set both input and output block size to n bytes, superseding the ibs and obs operands.

[...]

ibs=n Set the input block size to n bytes instead of the default 512.

[...]

obs=n Set the output block size to n bytes instead of the default 512.