constant MASSIVE data/files losses on HDD!

Well, in my case hardware is very resistant and I tested it, so no problems there.

Loss of files ONLY happens IF hang during boot, happens.
Once successfully booted system, will not suffer file loss, even if I forcefully turn off power.

I will soon try verbose mode, but I need to catch that hang.

Then I'll install RELEASE.
 
richardpl said:
boot into verbose mode, and/or use serial console, so you can enter kdb early, during boot. All of this is explained in the handbook.

But you didn't show anything which could claim that problem is in kernel and not in rc.d scripts.

Ok, I Booted FreeBSD with verbose logging
Looks that hang appears at the start of init.

Any last recommendations, before I start installation of RELEASE 8

PS: As further files got deleted, now I can't startx :p
 
The problem may be neither hardware nor freebsd-related.
Once I have experienced this sort of trouble after I messed with partitions using third-party tools in windows(tm). Some tool (maybe even windows(tm) stock "disk administration service") wrote wrong data to the partition table and the FreeBSD partition became "smaller". It did boot, however, and filesystems got mounted, and there were some files left, but fsck failed and even newfs later said that "partition is larger than available space" or sort of.
To my understanding, it's impossible for a kernel to overwrite data between actual loading of the kernel and actual mounting of filesystems. And if the kernel hangs before FS gets mounted, then it's highly unlikely to be the cause of FS damage. To what degree a system must be crap, to allow files to be deleted :)
Imho, it must have been either a result of usage of some low-level HDD tools, including dd, or a failure of disk controller, which is subject to failure, regardless what brand and price your PC is.
 
Meh...Im just getting back here, and dont really have anything important to add, but have to say myself, that its a little disappointing at how easy it is to lose data with UFS. Ive had numerous power outages on my Exchange cluster, and those babies pop right back online, if I was using FreeBSD, it would all be shot by now.

That being said, Ive been a little bored lately, and have been booting over to Vista this past week to play some games that wont run on Wine, and what a pile it is, after using FreeBSD, I absolutely dislike going back to Windows.

All these years of development, and its still this easy to hose an install, I honestly cant believe something hasnt been done by now about how easy it is to lose data, on a sever OS. Sure in a server environment, backups, and redundant power should be in place, but what about the small biz owner that might be not have all the cash to invest in the hardware initially, to provide the redundancy, or the specs to run ZFS decently. Have been yanking cords on Win boxes for over 8 years now, and they always still work.

I love FreeBSD, and wouldnt want to run anything else, but it just seems a bit nuts to me.
 
Ok guys, I did it!
I've downgraded from 8.0-STABLE to 8.0-RELEASE-p2

No more hangs, at boot time and file erasures.
So, just to be safe, I did same to my server.

But really, when you loose files on UFS then you REALLY LOOSE FILES! Worse then with NTFS.

Now, why is there no warning in handbook for UFS??
They just state: "fs might appear just a little older"!

And if I'am asked file integrity is exactly the most important for servers -> definitively more than for Win / ntfs / desktops
 
NTFS has the USN (Update Sequence Number) journal, which is the Windows version of what killasmurf86 posted right above your post.

If you're so concerned about your filesystem's consistency and your files' safety, enable journaling, stick to a RELEASE and make multiple backups in different physical locations.

UFS in itself has no problem whatsoever. I've never lost a single block in years even if I had many freezes caused by human errors, buggy software and hardware problems.
On one machine, I recently had freezes every few days. I kept testing it for a month or so till I discovered it was bad acceleration support in Xorg. Needless to say the machine had been freezing dozens of times during that period.
 
Beastie said:
NTFS has the USN (Update Sequence Number) journal, which is the Windows version of what killasmurf86 posted right above your post.
...
And USN is enabled by default?
So that is why I never have any file looses on WinXP?

And to achieve "same" on FreeBSD I just need to utilize gjournal?
 
To be honest, I haven't read the entire thread, but please test your hard drive properly.
I did see you used dd, but using dd does not consist of a good hard drive test.

Use MHDD to test your drive, also be sure to look at the SMART data (Using MHDD or smartmontools in ports). Look for reallocated sectors and/or CRC/seek/read errors.

Here is a thread with useful information about hard drive testing, Read this thread entirely since much of the more useful info is towards the end.
http://www.daemonforums.org/showthread.php?t=4042&highlight=mhdd

Hard drives are bloody unreliable machines :(
 
>Sadly, sysinstall is such a spaghetti mess of code that nobody really wants to touch it with a 10 feet pole and unfortunately it seems that installers in general are something that people with the know-how care the least about working on.

That's not true, Randy Harper - aka Freebsdgirl - tries to tame the beast.
 
Hm, ok...
For now shift from STABLE to RELEASE fixed an issue for me.

I'll do recommended HDD tests, later.
 
It might be interesting to know if anyone else is running FreeBSD successfully on the same laptop.

(I had an ASUS motherboard whose SATA controller caused file system inconsistencies almost on demand with normal usage. I replaced it with a Gigabyte with the same chipset and same SATA controller and no problems at all. The ASUS in now in my wife's Windows machine and has had no disk issues since being installed a year or so ago. Go figure.)
 
Jago said:
That being said, I hope that FreeBSD will be moving towards using GPT and ZFS by default as soon as possible (leaving the older options available as an alternative). Sadly, sysinstall is such a spaghetti mess of code that nobody really wants to touch it with a 10 feet pole and unfortunately it seems that installers in general are something that people with the know-how care the least about working on.

I am working on this, but really, don't expect it anytime soon. I *hope* to have it in 9. The default will still be using the old style editor, but the expert mode will most likely have the option of using a new editor that allows for GPT using libgeom.

There have been updates to sysinstall recently, such as adding support to install from USB mass storage devices. Also, sysinstall code isn't *that* ugly or even difficult to understand. It's just outdated, as it doesn't account for improvements and advancements that have been made such as geom or devfs. Please examine the pr database or even commit log and/or related code before spreading such comments. Thanks!
 
I ran into something similar yesterday where the system seemed to hang right around that point and my harddrive light was solid on. I was able to hit CTRL+C at that point and it stopped the fsck it was running and kicked me to single user mode.

I do agree though that you really should not be losing files. I have been debugging NVIDIA 64bit for the last month with lots of crashes and the only thing I've lost are temp files that were open at the time.

I didn't see where you had posted what your harddrive layout is. Are you using one huge / partition like linux does or with a smaller / partition and other partitions such as /usr and /var?

Rusty Nejdl
http://networking.ringofsaturn.com
 
HDD Layout:
Code:
******* Working on device /dev/ad4 *******
parameters extracted from in-core disklabel are:
cylinders=310101 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=310101 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 7 (0x07),(NTFS, OS/2 HPFS, QNX-2 (16 bit) or Advanced UNIX)
    start 63, size 41945652 (20481 Meg), flag 0
	beg: cyl 0/ head 1/ sector 1;
	end: cyl 1023/ head 254/ sector 63
The data for partition 2 is:
sysid 15 (0x0f),(Extended DOS (LBA))
    start 41945715, size 192490830 (93989 Meg), flag 0
	beg: cyl 1023/ head 254/ sector 63;
	end: cyl 1023/ head 254/ sector 63
The data for partition 3 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 234436545, size 78140160 (38154 Meg), flag 80 (active)
	beg: cyl 1023/ head 254/ sector 63;
	end: cyl 1023/ head 254/ sector 63
The data for partition 4 is:
<UNUSED>

******* Working on device /dev/ad4s3 *******
parameters extracted from in-core disklabel are:
cylinders=77520 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=77520 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
<UNUSED>
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 0, size 50000 (24 Meg), flag 80 (active)
	beg: cyl 0/ head 0/ sector 1;
	end: cyl 1023/ head 254/ sector 63


# /dev/ad4s3:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  1048576        0    4.2BSD     2048 16384     8 
  b:  8388608  1048576      swap                    
  c: 78140160        0    unused        0     0         # "raw" part, don't edit
  d:  9398272  9437184    4.2BSD     2048 16384 28552 
  e:  1048576 18835456    4.2BSD     2048 16384     8 
  f: 58256128 19884032    4.2BSD     2048 16384 28552
 
That wasn't quite what I was looking for. Can you provide the output from FreeBSD of a df -h ? Also, it was touched upon here but I see you are dual booting and not sure if you are seeing any conflicts there. A few people I believe mentioned where the boot block can become slightly corrupted and cause the effect you are seeing.

Sincerely,
Rusty Nejdl
 
Code:
# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad4s3a    496M    370M     86M    81%    /
devfs          1.0K    1.0K      0B   100%    /dev
/dev/ad4s3e    496M    3.1M    453M     1%    /tmp
/dev/ad4s3f     27G    4.5G     20G    18%    /usr
/dev/ad4s3d    4.3G    2.9G    1.1G    72%    /var
/dev/fuse0      20G     15G    4.8G    76%    /mnt/win_c
/dev/fuse1      92G     87G    5.1G    94%    /mnt/win_d
 
does sysinstall even allow to use zfs in its current state?

the one part of freebsd that I doesnt seem to ever change for years and years is sysinstall and it still has some nasty glitches. I really would suggest rewriting the default one from scratch and not only updating the expert mode because the sysinstall is what scares many people of to linux.
 
chrcol said:
does sysinstall even allow to use zfs in its current state?

the one part of freebsd that I doesnt seem to ever change for years and years is sysinstall and it still has some nasty glitches. I really would suggest rewriting the default one from scratch and not only updating the expert mode because the sysinstall is what scares many people of to linux.

Would you like to provide some patches?

If people are scared away from FreeBSD by sysinstall - an installer which many people actually *like* because it has looked the same for years - then quite honestly, we don't need them. Show them OpenBSD's installer. Watch them cry.

The reason for only updating the expert mode for now is simple. Switching from libdisk to libgeom is no small change. There will be many bugs, and the last thing we need is to leave people without the option to use the old method should the new method be busted. Although most things are easily tested by all the people running -CURRENT, sysinstall is not one of them. Do you know how many people use sysinstall to install -CURRENT? Probably like 4. I'm not relying on 4 people to catch all the bugs in a completely new piece of code.

But you could solve all of this by providing patches that are spectacular and bug free, something that could redeem FreeBSD to those users that go "eek, I can't use ZFS with sysinstall, time to go to Linux!" (then later realize that Linux doesn't even have ZFS). I look forward to receiving them.
 
randi@ said:
If people are scared away from FreeBSD by sysinstall - an installer which many people actually *like* because it has looked the same for years - then quite honestly, we don't need them. Show them OpenBSD's installer. Watch them cry.
Personally I really like sysinstall. For me it has always worked. The only think I like better is OpenBSDs installer, that is really a piece of functional software. It takes me less time to install OpenBSD to a functioning base than it takes me to get the windows installer to greet me. So I would like to take the opportunity to thank everyone who is working on sysinstall.

And on an additional note, so what if users are scared away to linux. Is that REALLY a problem?
 
I really like OpenBSD's installer as well, but the point remains - if someone is all "FreeBSD's installer is scary", do you really think they are going to find anything other than an X based install that much better? It's friggin' dialog boxes. It's not rocket surgery.
 
randi I personally am fine with sysinstall for installing freebsd, however it does seem glitchy on a already running system.

I know people who rent out servers and they dont support freebsd for the simple reason they cant use sysinstall, they used to point and click install they get in centos. Yes it does seem silly but I feel at this point sysinstall the first thing new users are faced with is enough to scare people off.

I am only making a suggestion in what I feel will increase freebsd uptake in the server environment I never said I think its easy to do.
 
The server environment rarely involves point and click - at least, not for serious operations. The focus in a production environment is to be able to roll out new installs quickly and easily - which sysinstall does have support for. It's possible to script or do an install over serial. Good luck having point and click over serial. :)

If you want point and click, go to PC-BSD. It's not happening here.
 
you misunderstood me, its not what I want, its what datacentre staff want.

It is sad the tone of your reply, I guess freebsd development is not after a higher takeup. If that is the case then fair enough.

The 3 main feedbacks I get given for reasons for a dc to not support freebsd are.

1 - lack of point and click on installer, apperently redhat based os's have this hence them been so popular.
2 - lack of working on network unattended installs, if freebsd does support this then I can pass this info on.
3 - various hardware hanging on bootloader, using external drives for media is very common, although I think this improved with freebsd 7 and I guess freebsd8.
 
Back
Top