Understanding dd if=/dev

Status
Not open for further replies.
I

iic2

Guest


What is the meaning of bs=1m or count=2 or count=10? Are there others? Which is the fastest or slowest and why?

Are there others that you know of that can 0 + 1 + X the disk far beyond data recover?

Code:
dd if=/dev/random of=/dev/ad0s1a bs=1m
Code:
dd if=/dev/zero of=/dev/ad1 count=2
Code:
dd if=/dev/zero of=/dev/whatever bs=1M count=10
The reason why is I have four installs of FBSD and plan to avoid going to single usr mode to do things. Mainly for zero_FILLING of each partition indivually and than restoring back-up files from where I am (FBSD-1).

FBSD-1 is my home base for testing and building things for FBSD-2 and FBSD-3 ... FBSD-4 is where my final install will live and will be copied to other machines of particular use. Pretty cool umm. Well, that my goal.

I messed a lot of things up over the pass few days but I pick up a lot of information about FreeBSD (what it will do and not do). but now I'm ready for my main test and I need to make sure I get it right and understand dd to its fullest. I am tired of screwing-up and starting over again and again.

F1 FreeBSD
F2 FreeBSD
F3 FreeBSD
F4 FreeBSD

I boot to F1. I mount the usr partition of F2 -- ad4s2f as _2f. Actually I don't have to mount anything because I got got them listed in fstab as 2 2 (Dump - Pass#).

Which can I do safely and insure that I zero_FILL only ad4s2f and why?

Code:
dd if=/dev/random of=/dev/ad4s2f bs=1m
or
Code:
dd if=/dev/random of=/_2f bs=1m
or
Code:
dd if=/dev/zero of=/dev/ad4s2f bs=1M count=10
or
Code:
dd if=/dev/zero of=/dev/_2f bs=1M count=10
So far I only tried the following code but it never remove everything. It's strange to me that ad4s2 has LIVE files that should be dead when booting ad4s1. All of my installs has their OWN root partition. It was my very first shocking experience. I did not expect this.

Bottom line, in the end when I get it right I expect all out-side partitions to be DEAD until I boot to that particular SLICE.

Code:
rm -rPfv /_2f
Lots of questions and ideas. I'm only half way there. I appreciate what you have as time moves on.

Thanks in advance
 
OP
OP
I

iic2

Guest


I tried them all and I get:
Code:
dd if=/dev/random of=/dev/ad4s2f bs=1m
Code:
dd if=/dev/random of=/_2f bs=1m
Code:
dd if=/dev/zero of=/dev/ad4s2f bs=1M count=10
Code:
dd if=/dev/zero of=/dev/_2f bs=1M count=10
Operation not permitted.
or
/_2f: Is a directory

I guest FBSD expect you to be in Single User Mode. Since I have the source how can I make FBSD do this from where I am.

or is there a working code like below that will delete those remaining LIVE files

Code:
rm -rPfv /_2f
It can't be impossible. I'm root and they shouldn't be ALIVE in the first place I think. What am i'm doing wrong?
 

graudeejs

Son of Beastie

Reaction score: 695
Messages: 4,615

rtfm dd(1)
/dev/random is random number generator
/dev/zero is 0 "generator"
you can send there anything, /dev/null


to erase some disk
dd if=/dev/random of=/dev/da0 bs=8

to erase mbr
dd if=/dev/random of=/dev/da0 bs=512 count=1

Code:
/_2f: Is a directory
explains everything....
if you want to remove it simply use rm -R /_2f
rm(1)
 

Beastie

Daemon

Reaction score: 517
Messages: 2,253

If you just want to make a slice unbootable, I think using fdisk and zeroing the OS ID (i.e. 165/0xA5 for FreeBSD) and (optionally?) the slice's C/H/S and LBA coordinates should be enough.
 
OP
OP
I

iic2

Guest


How do I kill a busy device?

This should give you a better picture of what I am trying to do.

These are the files required by FreeBSD no matter what. Because when I do rm -R (or) rm -d: I get - Device busy
When I do dd if=/dev/random of=/dev/da0 bs=8 or any other under FBSD-1: I get Operation not permitted. I think it only allowed in Single User Mode unless it can be change by the source code which would be a good idea for this case only if dd'ing can kill the device. YES! YES! YES! If you think this is workable please give me a few pointer than I'll do make World for FBSD-1. It be nice to have it there anyway.

Well, this proves that FBSD-1 had initiated LIVE files on FBSD-2 partitions and that's a big disappointment. Now I got to find a way to kill each busy Device. I think CHMOD or parity bits is the key, but how do you change it when dd can't get to it.

So my questions should have been; How to get the device process ID? And how do I kill that busy device?

FBSD-2 Left-overs:

Code:
/use/bin/chfn, chpass, chsh, crontab, login, opieinfo, opiepasswd, passwd, rlogin, rsh, su, ypchfn, ypchpass, ypchsh, yppasswd
Code:
/usr/lib/librt.so.1
Code:
/var/empty
Code:
/bin/rcp
Code:
/lib/libc.so.7, libcript.so.4, libthr.so.3
Code:
/libexec/ld-wlf.so.1
Code:
/sbin/init
[EDIT]] When I realize that these files would not delete when even on another slice I made notes like this /root/sbin = init .. That could have been too misleading ... should have been for example: /sbin/init [EDIT]


They all are busy DEVICES. And if you try to boot to it:
Code:
ip6addrctl: not found = 6x the same line
re0: link state changed to DOWN
dc0: link state changed to DOWN
logger: not found
mktemp: not found
uname: not found
sed: not found
awk: not found
cmp: not found
usage cp [-R [H | -L etc etc etc etc etc] source_file target_file
	    [-R [H | -L etc etc etc etc etc] source_file target_file
grep: not found
Configuring syscons: blanktimevidcontrol: not found
.
Eval: /usr/sbin/sendmail: not found
Eval: /usr/sbin/sendmail: not found
/etc/rc: WARNING: run_rc_command: cannot run /usr/sbin/cron
logger: not found
That's not bad at all to try to piece back together a tiny BSD or something but this is not the issue right now. I want to delete them all and leave those partitions in tack so that when I'm inside FBSD-1 I still has access to FBSD-2 partitions.

If I modify the boot record I will lose access to FBSD-2 partitions and would have to do things the old fashion way which would defeat the purpose. Dev would have no ad4s2 to operate on.

Remember, FBSD-1 is the only crazy slice that will be modifying FreeBSD on all other slices in all kinds of ways for production, etc. You would consider FBSD-1 as a light-weight developer HOME page (slice)
 

graudeejs

Son of Beastie

Reaction score: 695
Messages: 4,615

uh.... man.... (I have brainstorm after reading your last post)

and since you seem to bee (no offense) noob (or i'm idiot, or bough, or none), I'll make it simple

1) figure out what you need to dd (disk, slice, partition, label)?

/dev/ad[RED]X[/RED] - disk X
/dev/ad[RED]X[/RED]s[RED]Y[/RED] - slice Y on disk X
/dev/ad[RED]X[/RED]s[RED]Y[/RED]z - label z on disk X on slice Y
/dev/ad[RED]X[/RED]p[RED]Y[/RED] - gpt partition Y on disk X
/dev/ad[RED]X[/RED]p[RED]Y[/RED]z - label z on gpt partition Y on disk X

2) check if it's mounted
mount
unmount if nessacery

3) check if you have write permissions
ls -l /dev/adX.....

4) use su or sudo to get extra permission if nessacery

5) run dd
dd if=/dev/random of=/dev/adX.... bs=8m

bs=... is block size (8m means 8MB block)
count=... is how many blocks you want to write
if you don't write count, then you will dd device to the end of it (end of disk, label, slice, partition....)


for example to rewrite random data to entire 2nd slice of 4th disk do
dd if=/dev/random of=/dev/ad4s2 bs=8m
If you don't want to write random data, you can write zeros (0) all the way using:
dd if=/dev/zero of=/dev/ad4s2 bs=8m


p.s.
replace X and Y with numbers, and z with valid label (a-g on FreeBSD 7)

p.s.s
hope, this explains..... and we're bough not idiots

EDIT:
oh and devices doesn't have pids...., and you can't kill then
If they are busy you can force to unmount them
here's your homework: figure it out, how
TIP: umount(8)


EDIT:
Fixed typo
 
OP
OP
I

iic2

Guest


Thanks killasmurf86, you could never offen me, I am a noob even after all these months.

I'll get on it right now. But this is something I found and want to post to let you guys know I am on the job other than the parity bit idea that I founded while reading DD(1). I do follow advice until I loss sometime than give up. Thanks again, I'll report success shortly.

http://lantech.geekvenue.net/chucktips/jason/chuck/1091924337/index_html

Code:
#top
#ctl-z to Suspend

#kill -s HUP 121
Just for testing:
PID is now dead but I don't see anything process running that looks like it came from FBSD-2 slice. They must be sleeping threads. So I try:

http://lists.freebsd.org/pipermail/freebsd-hardware/2006-July/003640.html
Code:
ps -lax
to see them but I can't tell which one comes for which slice; FBSD-1 or FBSD-2 or if there are really there. I don't know what to grep ps.

PS
What!!! :)
oh and devices doesn't have pids...., and you can't kill it
See what I mean. By time I would have read all 25 docs and not gave up I still would have never learned this because NONE never said that.

There is nothing like details. You do usually present details once you know the a person don't really no better. We be loss for years with-out people like you (deep forum help).

By the year 2014 I would have learned that "devices doesn't have pids" but I would be an expert in my mind by than. Now who would think you were wrong for trying harder to help. Tell me his name :(

Long Live forums.freebsd.org
 

Beastie

Daemon

Reaction score: 517
Messages: 2,253

iic2 said:
I don't see anything process running that looks like it came from FBSD-2 slice
Man, what are you trying to do?
Currently running processes (memory management constructs) have nothing to do what so ever with slices (disk management constructs)!


iic2 said:
By time I would have read all 25 docs and not gave up I still would have never learned this because NONE never said that.
But YOU should know it by simple deduction. PID is an abbreviation for "process identifier", and a device is not a process. Therefore a device cannot possibly have a PID.
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,264
Messages: 4,099

iic2 said:
I tried them all and I get:

Operation not permitted.
or
/_2f: Is a directory

I guest FBSD expect you to be in Single User Mode. Since I have the source how can I make FBSD do this from where I am.
Operation not permitted usually means you are trying to access the device node for a disk/slice/partition that is currently mounted. GEOM (the disk access framework in FreeBSD) doesn't permit you to modify disks that are in use.

There are two ways to work around this:
  1. unmount all filesystems on the disk/slice/partition you are working on, or
  2. set the kern.geom.debugflags sysctl to 16 (# sysctl -w kern.geom.debugflags=16)

The first option is the safest.

When using dd to access disks/slices/partitions, you are acting at the hardware level. You want to make sure that nothing above the hardware level is trying to access those disks/slices/partitions at the same time.

The most common options used with dd are:
  • if= this tells dd which file to use for input
  • of= this tells dd which file to write the data to
  • bs= this tells dd how large of chunks to read from the input and to write to the output. The default is 512 bytes, which is very slow. Setting it to at least 1M (1 MB) will speed things up dramatically. Some disks work better with larger bs than others. And all optical drives must have bs set to 2048 or 2K (2 KB)
  • count= tells dd how many blocks to read from if. Useful if you only want to access a certain number of blocks. If this is not set, dd reads from the beginning to the end of the file.

Note: "file" above refers to any file in the filesystem, which includes all the device nodes under /dev. Thus, it can act on simple files, or harddrives, or floppies, or USB storage devices, etc.
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,264
Messages: 4,099

killasmurf86 said:
for example to rewrite random data to entire 2nd slice of 4th disk do
dd if=/dev/random of=/dev/ad4s2 bs=8m
If you don't want to write random data, you can write zeros (0) all the way using:
dd if=/dev/null of=/dev/ad4s2 bs=8m
Note: /dev/null above should be /dev/zero. :) You can only write to /dev/null, you can't read from it.
 

graudeejs

Son of Beastie

Reaction score: 695
Messages: 4,615

phoenix said:
Note: /dev/null above should be /dev/zero. :) You can only write to /dev/null, you can't read from it.
yes, i know (my mistake).... thanks for correcting me
 

graudeejs

Son of Beastie

Reaction score: 695
Messages: 4,615

btw, there is one more common argument of DD
seek=

iseek is also usefull for some advanced things
 
OP
OP
I

iic2

Guest


by phoenix
GEOM (the disk access framework in FreeBSD) doesn't permit you to modify disks that are in use.

There are two ways to work around this:
unmount all filesystems on the disk/slice/partition you are working on, or

set the kern.geom.debugflags sysctl to 16 (# sysctl -w kern.geom.debugflags=16)
....
....

I Temporarily turn off GEOM protection for the MBR.
Code:
# sysctl kern.geom.debugflags=0x10
kern.geom.debugflags: 0-> 16

Than I tested it with /G on 2nd slice
Code:
# dd if=/dev/zero of=/dev/ad4s2g bs=1m
dd: /dev/ad4s2g: shortwrite on character device
dd: /dev/ad4s2g: end of device
99+0 records in
98+1 record out
103708160 bytes transferred in 1.579889 secs (65642685 bytes/sec)
How do I turn GEOM protection back ON and/or what are all the NEXT steps?

Code:
 ???
...
...

Also /G was a 95 megabyte empty partitions. What do 99+0 records in mean and why only 99 ?
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,264
Messages: 4,099

Set kern.geom.debugflags back to 0.

After that, what you do next is up to you. You now have a "blank" partition. You can delete the partition via fdisk/bsdlabel, or you can format it with a new filesystem via newfs.

"records" is another word for "blocks". So 99 records, with bs=1m, means 99 MB of data.
 
OP
OP
I

iic2

Guest


I couldn't wait so I reboot the machine and I got the Single User Mode screen. So I re-boot to FBSD-3 and mounted ad4s1a and edit my fstab by commenting out _2g ... Than I re-booted to FBSD-1 and I was back. My first real use of Multi-Installs. Hey hey

But now I can not access the ad4s2g partition. Unbelievable. One problem down to only lead to another.

Anyone know how I get my partition back and why I lost access in the first place. The goal was only to do was a simple zero_FILL.
 

Beastie

Daemon

Reaction score: 517
Messages: 2,253

Hmm, I wonder why I can't access this goddamn partition. Oh, silly me! That's right, I wiped it completely from A to Z, including all the metadata that is actually used to describe it.
Maybe newfs could create some new metadata? Let's try that.
 
OP
OP
I

iic2

Guest


# newfs -U /dev/ad4s1g
# sysctl -w kern.geom.debugflags=0x00
kern.geom.debugflags: 16 -> 0

I did not know that. I thought it was read after dd. This thread answered a 1000 questions for me from past to future. I needed this complete walk through because it's the most important thing I ever wanted to do other than compile kernel. I always read the doc links and many of them links to many other. Not many goes into these type of details and when they do it's in one line out of thousands using names I never seen before. They are the best but forum help is the plus.

Thanks phoenix


Beastie, I have a total history of 4 months with FreeBSD. I knew nothing outside of Windows. I mention in a thread that I went back to school and that I have many prerequisites blocking me before I can even get to a class that teaches some form of UNIX and that was my reason why I join this forum.

I still got 6 months to go and than all they got for me first is STUPID linux. It be 2 years before I even compile a real UNIX kernel.

So yes I am a noob but how long did it take you and killasmurf86 to get to were you are today? Nine out of ten you work in the field.

Just like with at lease 10 of my past threads I have to explain my question, get ridiculed, than the thread get close before solved.

So my questions should have been; How to get the device process ID? And how do I kill that busy device?
Just for the record in post #5 under my miss-worded question I show a complete list of LIVE files pertaining to the question.

They all are running processes and they all in fact have PID's.

Instead of putting 2 and 2 together I get ridiculed because I wrote the word Device. Now that's the new conversation.

If I had as much experience with FreeBSD as most of you guys and a 3 month newbee wrote all of above, I would have picked that up in a flash.

Yes I don't know jack but you all were the same in the past. I learn more from killasmurf86 since I been here but that give no one the right to follow behind him to flame me behind his un-appealing words. It was only through your help during the past few week that got me here or this would have never happened for me. Don't let it get sour.

Thanks killasmurf86 for the great days

Thanks Beastie for the new day
 
OP
OP
I

iic2

Guest


Well, I really don't know if they are processes or devices because I thought all devices were in /dev. Most of those files are in /usr. I would like to know since we never got back to them.
 

DutchDaemon

Administrator
Staff member
Administrator
Moderator
Developer

Reaction score: 2,841
Messages: 11,319

Should I open a Kindergarten Forum or do you think you guys might play nice? I haven't closed ten threads, but don't think I won't if this continues.
 
OP
OP
I

iic2

Guest


My problem from day1 was not knowing how to umount and using dev properly and I didn't know about editing the fstab until days ago. Sysctl -w made things much more easier. No wonder everyone was going nuts. dd is a wonderful tool. dd and this thread taught me how to start paying more special attention to details. I'll be alright now.

Thanks again :) and sorry about that guys :(
 

graudeejs

Son of Beastie

Reaction score: 695
Messages: 4,615

DutchDaemon said:
Should I open a Kindergarten Forum or do you think you guys might play nice? I haven't closed ten threads, but don't think I won't if this continues.
+1

to iic2
Give a man a fish; you have fed him for today. Teach a man to fish; and you have fed him for a lifetime
We're givving you fish
You can only learn how to fich by reading lots of handbooks, manuals, doing dstupid mistakes (like we all did) and repairing them
 

Beastie

Daemon

Reaction score: 517
Messages: 2,253

iic2 said:
Just for the record in post #5 under my miss-worded question I show a complete list of LIVE files pertaining to the question.

They all are running processes and they all in fact have PID's.
Where exactly in post #5?
For the second time, only live processes have PIDs. If you execute a binary from your file system, the ELF loader copies its sections - text, data, reloc, etc. - to the memory. A process is created and gets assigned a unique PID. That's what it's all about: a running process, not the binary file from which the process was created. You kill a process by its PID. You delete a file by its name. Period. It's the ABC of operating systems.


iic2 said:
Instead of putting 2 and 2 together I get ridiculed because I wrote the word Device.
Ah, come on! You can't stand a little humor? Well, if I offended you, then I'm sorry.


iic2 said:
I really don't know if they are processes or devices because I thought all devices were in /dev. Most of those files are in /usr.
That's exactly right. Processes can be seen using ps(1). Current devices can be seen using a simple % ls /dev. And I repeat: both are fundamentally different things. Also, check hier(7).


iic2 said:
My problem from day1 was not knowing how to umount and using dev properly
Which is why man pages exist. For *whatever*'s sake, they are stored in your own computer. USE THEM! By the way, man pages have a "SEE ALSO" section at the end. For example, when you check mount(8)'s man page, you'll also see umount(8) and many other related commands, and sometimes files, in this example fstab(5).
 

DutchDaemon

Administrator
Staff member
Administrator
Moderator
Developer

Reaction score: 2,841
Messages: 11,319

And if you can't find it by name, use apropos(1), [cmd=]apropos mount[/cmd].
 

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,264
Messages: 4,099

Or use -k for a keyword search of man pages: $ man -k mount
 
OP
OP
I

iic2

Guest


Here's the complete story

Man Man is the world.

Beastie, I know about processes and devices from the Window World. I really do write assembler. I just did not want to argue because it was not the point. Now you just told me all I ever needed to know about this entire issue and if you CAREFULLY read on you'll understand why I came up with the word DEVICE.

ic2 asked: I really don't know if they are processes or devices because I thought all devices were in /dev ....

Beastie said That's exactly right.
Anyway, you guys know how I was... always trying to delete everything under the sun just to see how small FreeBSD could be.

killasmurf86, if man never tasted the fish he would never known what he was missing. It was so tasty I now go fishing with MAN MAN everyday since Multi-Install. I have done a thousand installs for months because I keep screwing up, mostly deleting or opening up the wrong file or typing bad code i saw somewhere on the command line.
handbooks, manuals, doing dstupid mistakes (like we all did) and repairing them
I done 3 months worth. Re-read this thread very carefully like the bible and you'll understand the important of what I stumbled upon.

Now today I got four installs with out the help of MsDos and I am not afraid to screw up anymore and this thread has near finalized that. My next job is to fix my How-to aragon and Beastie style :)

Back to Business:
This might get lengthy but I need you to understand; Example: do we agree that /INIT is a process. See Post #5, than they all are because none of these files are in /dev unless FreeBSD slipped a few in on us :)

But for now who cares what they are called ... The point is why do they still exist after attempting to delete them with [ rm -rPfv /_2f ] which is a very powerful code. These files are on another slice and they are NOT suppose to be running PERIOD. That what I meant by LIVE files, SYSTEM Files running in the dark, obviously, on all four of my slices, each independently installed on PRIMARY.

One could blame it on functionally so I been searching for a work-around as I plan to fight with this at the same time. By right you should not have to bring in GRUB or anything else to stop this when maybe it can be masked than [ rm -rPfv /_2f ] would work or even those processes would die forever and not show up when booted to any particular slice. At lease there should be a temporary shut-off switch.

I don't know what to mask. I don't code for FreeBSD yet. Yes I read this doc in the past, The word Convert gave me hope to preform some kind of TRICK to leave .SNAP along to not destroy the partitions so I don't have to newfs. Than (Understanding dd if=/dev) was born. I did not come up with this overnight. I just did not know how to ask. After so many post by me here, no one knows what I'm trying to say, AGAIN :( :( :(

DD(1) FreeBSD General Commands Manual DD(1)

NAME
dd -- convert and copy a file
................................................
New tricks are invented everyday but I can't figure it out. If I knew how to write C and debug the kernel I would have solved this in under a week.
................................................

This is an PERFECT way for you to understand what I'm talking about if I have not yet made myself clear:

Everybody who got a old machine laying around, please fire it up.

1) Do a full install of any FreeBSD. You only need one install

2) rm -rPfv /usr
3) see what is left over in that partition.
4) Try to delete again ANY WAY YOU CAN like through /dev or any other tricks you can think of with-out destroy the partition, and pay attention to what kind of warning you get.
Now imagine if you had 4 separate installs of FreeBSD. You would never think those files that you can't delete are actually ALSO running on ALL 4 slices. BUT THEY ARE.

Again, imagine your 4 separate installs and you tried this on a OUTSIDE slice. You would never think to get the same results. BUT YOU WILL.

This is bad. You can't do jack. So you are back to the old long drawn out way of doing things.

It not only need answers ... it need a FIX. Who in the heck want something running on his machine that shouldn't be. I DON'T.

To me this is a leak, but than again this could be the SUPER security feature of FreeBSD itself which makes it so powerful as a production server but as for my development machine it's only way, blocking possibility ... I have MANY reasons to see them be-gone, at lease until production time.

Dang, now I want to know which is which, is it a Leak or is it THE PLUG ... if so, I need to flip the switch ... but HOW?
 
Status
Not open for further replies.
Top