I think it's poor cooling but I can't find the cause.
The symptoms you describe are definitely poor cooling.
With no dust and three fans the cooling should be sufficient enough at low and medium loads the fans don't run at max rpm. This can have several causes, as
T-Aoki already pointed out:
Sufficient coolant? You see, the coolant in "water cooling" is not actually water - or better not be. Pure water (worst case tap water, or even worse swamp water
) causes problems like rust, and over time algae grow which reduce the flow till a total block. That's why coolant consists of demineralized water with supplements like alcohol to supress algae. That's why the correct technical term is liquid cooling.
Plus the system needs to be filled up completely. If you hear some gurgle noises then this come from air bubbles inside. A correctly full filled up liquid cooling system delivers only the noise of the fans and pump. Over time coolant gets lost. Not much really. There must be no moisty spots. If so there is a leak: shutdown the machine immediately and replace the junk. But nothing is totally 100% completely closed. Those amounts are very tiny and are getting lost directly into the air, but over time it sums up.
Depending on the brand, the age and the coolant this shall take years. But it also depends on if the system you got was delivered already filled up correctly, like it should be the case with a ready to use already filled up delivered system, or an empty delivered system which needs to be filled first while assembling. When in the latter case the system was not filled up correctly to its full (you need to hold it right and turn it slowly to get the air out; see according handbook), then you get air in the system already in the first place.
A bit air doesn't do much besides gurgle noises, and of course it lowers cooling efficiency. But too much air will make the pump cannot pump enough water, so the cooler stops working. That's why you get coolant delivered also with already filled up ready to use systems, so to refill when there is a gurgle noise.
As
T-Aoki also already pointed out, the cooler needs to be attached the CPU correctly. If this is not assembled correctly (not the correct cooler fitting the CPU, loose mounting, too much thermal paste used, or air gaps left) then the heat cannot flow sufficiently into the coolant and being transported to the radiator.
What also can be a cause is if the pump works sufficiently enough - or at all. But since you did not report any thermal shutdown the latter one cannot be the case.
As a mechanical module the pump suffers wearing. Depending on the age and if you do not bought the cheapest junk the pump normally last many years.
But there is also the settings. In most today's motherboards BIOS settings you can do several adjustments for the fans and one for the pump of a liquid cooling system. It's for to find your personal compromise between silence and cooling power at different loads.
If the pump runs too slow or its rpm is not adjusted to higher values when the temperature rises, the fans can work their asses off but not helping the fact that there does not come enough heatflow from the cooler they can blow off the radiator, 'cause the pump delivers too few, it simply pumps not enough coolant.
If all this is checked as being OK, there could be some dirt somehow gotten into the cooling system which blocks the flow. Then this can be a warranty case but anyway the need to replace the cooler.
