Solved Hot chipset or failing SSD?

Yesterday my little Atom-based server's chipset fan was making a horrible noise. It's a cheap fan and tends to do that sometimes but I also noticed the drive light was continuously lit, which isn't normal, so I logged in to see what was going all. After a few minutes the SSH session froze and the machine stopped delivering web pages. I tried to log in again but got connection refused. it was time to hit the power button and instead of the expected ACPI shutdown the machine stopped instantly. I connected a keyboard and monitor.

On reboot the system complained the file /var/tm-main did not exist and went into single-user. I tried to edit /etc/fstab (which did indeed have a mount point of tm-main instead of tmp-main listed) but was told it was on a read-only file system. mount hung with no output and the same was true for most other commands I entered. Eventually I forced a reboot into the BIOS settings, where I found the SATA-0 port was claimed to be empty., even though that is where the / partition (ada0s1a) lives.

I switched off and set up my backup VM I use when the server isn't available and updated its contents and certificates before switching the router over to send requests there. Then it was time to open the case and check connections. Nothing was loose, so I switched on again, and this time the machine booted properly into single user and mounted / rw, so I could edit fstab and correct the error. After that I got a normal boot and after satisfying myself all now seemed well boxed the case up again and switched my router back to it.

So what was going on? Is the SSD holding the main / partition failing, or was the noisy fan failing to cool the chipset properly, perhaps causing the chip to overheat enough to make the SATA port malfunction? How do I guess between these?

I suppose the safest option is to replace fan and drive, but copying the system onto the new drive would be a pain.
 
Ah yes, I wondered how to run smart on FreeBSD. I've added the package to my Poudriere package list and it should be built tomorrow morning. I've ordered a new fan because seizing or not the noise is annoying, but don't want to replace the SSD unless I need to.
 
Given that you mention Atom how old is this hardware? Most Atom hardware is very old and some series such as C2000-series degrade over time (known issue).
 
Noisy fan, probably choked with dust, or the fan itself has had it. So cpu overheats, machine crashes hard without clean shutdown, and next time you boot its a mess. The fix: take the thing apart and use your trusty portable vacuum cleaner to suck the dust out of the fan and heatsink. Vacuum dust out of ventilation grilles as well. Make it spotless. Make sure the fan is running smoothly and quietly with no obstructions. It might be worth checking the heatsink compound too, it sounds like a pretty old machine, but go for the vacuum treatment first. If you haven't got a vacuum, use a small paintbrush to remove all the dust and muck you can see that is clogging it up and blow it out. If the fan still makes a noise after this treatment, you can try giving the bearing a squirt of wd40 or similar, but the fan is probably on the way out anyway, and its time to get a new one. Ebay is your friend. If it's still unstable after cleaning or changing the fan, then renew the heatsink compound.
 
I bought a pair of Noctua 40x10 fans for my hot running Intel 550 10gb NICs.

At $15 each they are Noctua quality at an affordable price. Tiny fans, just the ticket for hot running chipsets.
 
Thanks, everybody. I bought the MB and case in 2009 and used it as my desktop machine until 2015, when I bought a more powerful machine to build FreeBSD packages. Soon after that the Atom box was repurposed as a server to run a few websites and a mail server and is still in use. The fans have been replaced a few times but always become noisy after a few months, so their bearings are probably not up to the job. I've ordered a Noctua replacement this time. The PSU dates from 2003, though it had a few years in a cupboard before I adapted the case to take it after the original PSU failed. As you can see, I try to find a use for old kit if I can - waste not want not. The latest upgrade was to replace the PATA HDD with two SATA SSDs.

This is the output from Smart, but I'm not sure how to interpret it:
Code:
# smartctl -HA /dev/ada0
smartctl 7.5 2025-04-30 r5714 [FreeBSD 15.1-RELEASE-p5 amd64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   050    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       60575
 12 Power_Cycle_Count       0x0012   100   100   000    Old_age   Always       -       77
168 SATA_Phy_Error_Count    0x0012   100   100   000    Old_age   Always       -       8
170 Bad_Blk_Ct_Lat/Erl      0x0003   100   100   010    Pre-fail  Always       -       0/53
173 MaxAvgErase_Ct          0x0012   100   100   000    Old_age   Always       -       67 (Average 32)
192 Unsafe_Shutdown_Count   0x0012   100   100   000    Old_age   Always       -       40
194 Temperature_Celsius     0x0023   067   067   000    Pre-fail  Always       -       33 (Min/Max 33/33)
218 CRC_Error_Count         0x000b   100   100   050    Pre-fail  Always       -       16
231 SSD_Life_Left           0x0013   100   100   000    Pre-fail  Always       -       98
241 Lifetime_Writes_GiB     0x0012   100   100   000    Old_age   Always       -       607

I think this probably means the poorly-performing fan is the most likely culprit and I spotted it in time to save the chipset from destruction, but I'll wait for a more expert opinion.
 
Temperature of the SSD looks fine (mine tend to be warmer). It's had almost 7 years worth of power on hours, that's quite old. But the other values look ok, not that worried about its apparent age. It has some physical SATA errors, but put against the 7 year lifespan those numbers are quite low.

The SATA controller might be the one crapping out though, not the SSD. I've had a cheap 4 port SATA controller a few years ago, started acting up, the controller chip on the board was extremely hot, as in burning your fingers hot. It's an old technician approach to finding faults, put your finger on it, chips that are broken tend to be extremely hot. Warm is fine, hot is not. Be careful though, I once burned the outline of a DIP chip on the tip of my finger, it took several weeks to heal.
 
SATA_Phy_Error_Count and CRC_Error_Count are a bit concerning, these should not normally occur. These are most commonly related to hardware issues, be it cabling, motherboard / drive itself or power management.
While I understand this is might be "poor timing" a simple SBC (Radax's RK3399 based ones are still available at "normal" pricing and supported by FreeBSD) would run circles (in more ways than one) around your current setup without breaking the bank.
 
Fans usually last more than a few months before they get noisy. Well, maybe it's just the ones you've been using, of course. The noctua one should be an upgrade anyway. After you put the new fan in, check what your cpu temps are at idle and under load. If its getting up to around 70-80C, it's time to replace the heatsink compound, especially for an atom that is supposed to be a low-dissipation chip. Since it's an old box the original compound has probably long-since dried out. Any decent quality gunk should work, I use the arctic silver mx just because it's easy to get hold of but there are lots of other types, just avoid the super-cheap 1 dollar aliexpress stuff that is junk. Aim for normal running temps of around 40-50C if you can. Strange that it's chewing its way through fans, they are usually pretty reliable, even the cheap ones. Is it a high dust or grit environnment?
 
I don't know about the sata phy error cound and crc error count, I guess diizy knows more about those. But those smart attributes don't look too bad. The 'cooked' values are all still on 100. If the temperature is 67C, I guess that's about ball-park, maybe it's a bit warm, but it's not like it's 80C.

The bad block count is showing 0 for 'later' and 53 for 'early'; early bad blocks are blocks that are detected as bad at the factory. This is a normal part of the production process. Later bad blocks are blocks that have gone bad since the drive left the factory. You have none, which is good. So it looks like the controller thinks your media is fine.

Personally I wouldn't be overly worried about that drive, I guess you can keep an eye on the phy error and crc error counts and see if they are rising over time. Of course you can reseat the cable (at both ends) a few times, or try a sata different cable.
 
From what I'm reading here and on the Internet generally, the SATA errors point to the MB hardware rather than the drive, and we know there was an error there because when the problems first appeared the BIOS couldn't even detect the drive! As for SSD temperature, 33 is the raw value and since the threshold is given as 0 rather than 100 I suspect 67 is 100-Temp to comply with the general "higher is better" principle of SMART.

If at 7 years old it really is only 2% through its expected lifespan it has another 300+ years to go. That strikes me as a little optimistic.

I think the concensus is the drive is probably OK and the chipset probably overheated because the cheap Gelid fan is failing.

As for replacing heatsink paste, I don't think it's possible to get the heatsinks off without specialist equipment. They're clipped onto soldered-in loops on the board so if dried-out paste is the problem the board is EOL.

The CPU has a TDP of 10W and doesn't need a fan! The fan is on the chipset (North and South bridges).

I'm going to conclude I have my answer, but it's been an interesting discussion. Thanks everyone!

Postscript: Getting the heatsink off proved quite easy - just depress the clip and twist the body sideways until it passes out of the loop. The paste was solid. This probably wouldn't matter that much if things never got moved - solid paste still fills the voids and even if its thermal conductivity has fallen a little it's still basically there, but when screwing a fan onto the heatsink it will twist and the fan also vibrates in operation, shaking the joint. Once it cracks there are new voids and so it stops working completely. Pasty paste can move around a bit, just like Mrs Hancock's gravy.
 
Last edited:
Back
Top