How does a UPS work?

mehdi

mehdi

Is my understanding correct that a UPS can be configured to shut down a connected server in a safe manner in the event of a power outage?
Is it such a functionality that the Operating System should implement it?

Which UPS do you use with FreeBSD, and how does everything work in your environment in the event of a power outage?

Thank you
 
A simplified explaination:

Think of a UPS as a battery (because they really are). You have utility power coming into the UPS that does two things:
Keep the battery charged
Provide power to systems

The UPS has electronics inside that monitor the utility input and if that disappears will switch over to providing power from the battery so the attached systems stay up.

Some UPS (more expensive ones) will have the ability to provide signalling like:
Lost Utility Power
Recovered Utility Power
Ready to turn off

Software on a computer will "talk" to the UPS and handle these signals. When the computer receives a signal saying "I have 5 minutes of power left", the computer can trap that and start a graceful shutdown.

The "nut" that bakul referenced is software running on the computer, monitoring signals from the UPS and performing configured operations.

There is also apcupsd that ties a bit closer to some APC UPS models.

As for your last question, I use a couple of different APC models, they have a USB connection to the computers and if the utility power is out long enough (say 30 mins to 1 hour) the systems shut down gracefully.
 
mehdi said:
Is my understanding correct that a UPS can be configured to shut down a connected server in a safe manner in the event of a power outage?
Click to expand...
It's not the UPS that does this. All the UPS does is keep the power on in case the mains fails. Lots of UPS can be monitored from a computer though, and it can therefor signal it's been switched to battery power and also how much capacity it still has. The monitoring software on the computer can use this information to gracefully shutdown the machine when the battery power is below a certain percentage, i.e. it's running out of juice.
 
SirDice said:
It's not the UPS that does this. All the UPS does is keep the power on in case the mains fails. Lots of UPS can be monitored from a computer though, and it can therefor signal it's been switched to battery power and also how much capacity it still has. The monitoring software on the computer can use this information to gracefully shutdown the machine when the battery power is below a certain percentage, i.e. it's running out of juice.
Click to expand...
Yeah, I'm no expert but the UPS and computer pairing is only part of the story - if the power goes and you immediately lose your cooling solution in a data centre, a UPS keeping the hardware alive is actually going to be detrimental! Old place did this, the aircon and liquid pumps went off but the UPS's kept the kit running for a graceful shutdown. Warped and melted stuff in minutes!

I think they changed it so that the aircon and liquid pumps also run off a UPS, and a system monitors the health of the grid power and UPS. When the grid power goes, it fires up a diesel generator. They used to have dry runs to ensure that the diesel generator would fire up faster than the UPS power supplies would deplete. The running services weren't the critical factor here, the cooling system was.
 
taiwan740 said:
if the power goes and you immediately lose your cooling solution in a data centre, a UPS keeping the hardware alive is actually going to be detrimental!
Click to expand...
It still allows you to gracefully shutdown the machine instead of just killing it instantly. That's never good for your filesystems, caches and whatnot.

taiwan740 said:
They used to have dry runs to ensure that the diesel generator would fire up faster than the UPS power supplies would deplete.
Click to expand...
At one of my former jobs we had to do a dry run. While we were testing we got a call from reception. A bunch of firemen had come storming in, fully geared up. Turns out the people across the street noticed big black billowing smoke coming from the roof and called the fire department as they thought our building was on fire. It was the diesel generator on the roof, it hadn't been used in quite a while and started blowing a lot of thick black smoke when we turned it on 🤣
 
SirDice said:
At one of my former jobs we had to do a dry run. While we were testing we got a call from reception. A bunch of firemen had come storming in, fully geared up. Turns out the people across the street noticed big black billowing smoke coming from the roof and called the fire department as they thought our building was on fire. It was the diesel generator on the roof, it hadn't been used in quite a while and started blowing a lot of thick black smoke when we turned it on 🤣
Click to expand...
You were lucky. At the last place I worked where we had Diesel generators, every "test" caused an outage. Never once did enough of the diesels come online fast enough to handle the full load. One of the worst things that happened was the diesel dedicated to cooling failed to start up, but the others didn't. We wound up with a bunch of angry melty Vaxen when that happened. Kinda like what taiwan740 said.
 
The tricky part is deciding when to turn the power to the computer back on.

You don't want to keep the computer down indefinitely, obviously.

But you also don't want the computer to boot up when you just got city power back and you have 0.001% battery. You want it charged to a decent amount before booting the computer. But batteries age and if you put a too high limit today then it might not reach that limit at all in the future.

You might want to wait for a minimum time before powering the computer back up, to buffer fiddling on part of the power company which might lead to repeated downturn events in a short time.
 
As others have suggested, the UPS provides clean power and signals about its (battery) state to the host (usually via USB these days).

A good UPS can be programmed in many ways. The most expensive ones have a selection of power outlets which can be managed differently (e.g. to allow you to sequence the power supplied to network devices, SAN switches, SAN, then computers).

There are two software packages commonly used on Unix-like systems to manage a UPS, and the signals it produces. These are sysutils/nut which is a generic network UPS management tool, and sysutils/apcupsd which is specifically for managing the APC brand of UPSs.

APC gets a lot of "good press" for their UPS products, and sysutils/apcupsd does a very good job if you have an APC UPS.

NUT is your "go-to" package for other brands of UPS (including APC, if you want). But beware of "other brands". Some are perfectly good. Others are not. There's a lot of commodity UPS brands aimed at "the Windows desktop PC". Do your research, and take advice before buying.
 
  • Like
Reactions: mer
Funny you should ask right now. My personal answer to your question "how does a UPS work" is "extremely badly".

Now serious answers: Agree with most what was said above. I happen to use an APC UPS. Not because they are particularly great, but because about 10 or 15 years ago when my previous UPS died, I needed one urgent, and the local office supply store had a small APC UPS on sale for about $80, which is why I have that model. To answer your question of how to shut the computer down: I happen to use the apcupsd software mentioned above, which works really well for me, after considerably fighting to configure it correctly. I have tried installing nut, and after some back and forth and it being less than perfect, I went back to using apcupsd. Agree with gpw928 there. And finally, for normal households neither cooling nor big diesel generators are a real issue.

And as cracauer said: Configure the program that monitors the UPS to look at both remaining runtime AND battery charge. Don't let the battery discharge so far that after the power is restored, the battery is nearly empty. I have my UPS configured (theoretically, see below) to shut down at 50% battery charge OR when the remaining run time is less than a few minutes (it takes less than 30 seconds to do a clean shutdown of my server). At 50% battery charge, I have more than a few minutes, so I'm safe.

Here is our situation at home: My server (and the networking gear for it: WiFi hub, wired ethernet switch, and phone company DSL modem) are all on that cheap $80 UPS that is being monitored. In addition, we have an automatic propane-fueled generator, which automatically comes within 15-20 seconds (not minutes!) after a power outage. We shut the generator off at night, because when we're sleeping, we don't need lights or computers, the refrigerator makes it through the night just fine, and we don't like the noise. Obviously, when we shut the generator off, we first turn the server (and its UPS) off. In addition, the pump/well control system (which runs on 24V DC internally) has an industrial UPS. Because it uses so little energy (about 8-10W) and that UPS has over 24 hours of run time, it is not even actively monitored (just has a low battery shutoff), since we'd always run the generator within a day. If we have long power outages (and in rainy/stormy winters, we sometimes have a week without electricity), we roll out a portable generator, and let the main generator stop twice a day, to check the oil and do regular maintenance on it (air filter cleaning, check spark plugs and valve lash, and so on).

So much for the theory. About two years ago, I replaced all three 12V batteries (one for the UPS in the house in the server, two for the 24V system for the pump control) with new ones. And Monday night we had moderate rain, and two 12-hour outages (with 4 hours of electricity in between). Problem 1: The Raspberry Pi in the pump control system didn't survive, lost its SD card. The weird part is that it shouldn't even have shut down or crashed, since it theoretically has 24 hours of battery backup. So much of last night and today was spent re-installing that (fortunately I had a spare card at home, and yes, I will look into industrial cards). Problem 2: After we came back up in the morning, I'm getting an e-mail every hour from apcupsd that my battery has died. I tested it this morning, and indeed, the battery in the UPS is so weak, I only have about 10 seconds (not minutes!) of UPS runtime. Problem 3: This made me suspect that something is wrong with my batteries, so I tested the 24V UPS for the water control. The news there is slightly better: About 2 hours of UPS run time, but it should be at least a whole day. So right now, I'm not happy with UPSes in general, and SD cards in particular. My wife brought home one UPS-size battery from the trash pile in her office (which is likely better than what I have at home), and tomorrow there will be a lot of time spent on battery testing and charging. I suspect that I'll find that the new-ish (2 year old?) batteries are trash, and I by mistake bought crap. And then I'll probably get three now Panasonic- or Yuasa-brand batteries for Christmas.

At least the reinstall of the Raspberry Pi is mostly done.
 
Bulk LifePo4 battery pack prices are down to under $140/kWh & electronics are cheaper too but UPS vendors don't seem interested in a) designing the right kind of UPS (for end users) and b) passing on these savings. They don't even tell you the energy capacity of their UPSes. For example this APC SMTL1000RM2UC costs $1560 at Amazon but their ads don't tell you Amp-hours capacity. Users report say about 100Wh but who knows. You can now buy 12Vx100Ah batteries for your RV for about $240 but a UPS needs a bunch more. Another thing is companies that make Li battery "power packs" such Bluetti don't know how to build UPS (I got one that was advertised as having a UPS function but that didn't work).

I really hope some hardware hackers will put together a kit for building a proper UPS at home.
 
Lithium batteries are used in large server- and rack-grade UPSes. Obviously, those are very expensive, and the big ones are not even available on the open market.

But Lithium batteries have their own interesting problems. While normal (lead-acid) batteries are dangerous enough (ever had to mop up sulfuric acid spills?), lithium batteries like to burn or worse. Here is an example that made it to the press: A data center used by some of the large cloud companies experienced a water leak and fire:
www.theregister.com

Google Cloud suffers outage in Europe amid water leak, fire

Paris-based europe-west9-a zone still MIA
www.theregister.com www.theregister.com

Google Cloud Service Health

status.cloud.google.com status.cloud.google.com
If you read carefully, you can see that the fire in the UPS battery room actually burned for 18 hours! People who were actually on site afterwards describe it more as an "explosion" than a fire: walls were blown out, and the concrete/steel floor of one story was damaged badly enough that it was sagging and structurally unsound. Rumor has it a small water leak (perhaps caused by rain) caused a small fire, which turned on the automatic fire suppression system (sprinklers that dowse everything in water), and some of that water than wrongly flowed into the UPS battery room, causing the lithium batteries to explode. That in turn caused things to go pear-shaped :)

I don't think I would want that to happen in the basement of my house. I don't even store gasoline indoors.
 
ralphbsz said:
Lithium batteries are used in large server- and rack-grade UPSes.
Click to expand...
I have zero interest in these. I am only concerned about the home computers scene (more so as it is clear one can not rely on storing one's stuff in the "cloud" only).
ralphbsz said:
If you read carefully, you can see that the fire in the UPS battery room actually burned for 18 hours! People who were actually on site afterwards describe it more as an "explosion" than a fire
Click to expand...
These were likely Li-ion batteries. Teslas have a similar problem! LifePo4 batteries do not release oxygen when heated up so they are more resistant to thermal runaway and combustion. [Edit] given the worsening quality of PG&E power grid in California I am seriously thinking of putting in a solar + LiFePo4 battery + generator setup even if energy credit (for feeding power back into grid) rules are not so good any more. Of course, if I do that, I can retire local UPSes!
 
You can have your solar panels
and your turbines on the hills;
You can use the warmth of sunshine
to reduce your heating bills.

You can dream you’re self-sufficient
as you weed your vegie bed;
As long as you make sure to keep
A diesel in the shed.
 
ralphbsz said:
...ever had to mop up sulfuric acid spills?...
Click to expand...
Yes. This is why I will not buy APC UPSes any more. I've had 3. All three overcharged the battery causing it overheat, crack, and leak Sulfuric acid all over my house. I believe the problem is cheap and bad voltage regulators. Maybe yours is old enough to have been built before the Harvard MBAs at APC decided to cut costs.

Previous thread on UPSes: https://forums.freebsd.org/threads/which-ups.84810/
 
Jose said:
Yes. This is why I will not buy APC UPSes any more. I've had 3. All three overcharged the battery causing it overheat, crack, and leak Sulfuric acid all over my house. I believe the problem is cheap and bad voltage regulators. Maybe yours is old enough to have been built before the Harvard MBAs at APC decided to cut costs.

Previous thread on UPSes: https://forums.freebsd.org/threads/which-ups.84810/
Click to expand...
Thank you for the link to that discussion!
 
We had a couple of Eaton UPS in my last job, using nut under Debian. Worked well, and quite often in our subtropical rural high thunderstorm area. Apart from usual (gel) battery depletion over time, we had no problems with them.

With a range of blackout times from less than a minute, some minutes through to hours, with mains power recovery from fully back on, on for a few seconds then off again - perhaps repeatedly - or on for a minute or two then off again, it can be tricky choosing best timings, allowable discharge, and recharge before rebooting.

And of course which gear to power from UPSes (for up to half an hour) and which to let die (mostly network clients).

As long as the station's FM transmitter was running, we were obliged to make digital audio logs continuously; another constraint. A job I enjoyed ...
 
I have good experiences with Schneider Electric (formerly APC), big setups at work and smaller, used APC SmartUPS at home. For home use, the older APC models appear to be more reliable than current models. A battery change every three to five years, that is all the maintenance I ever needed. My oldest APC SmartUPS 1400NET died this year, at age 29 (!) and was replaced with a much newer model (14 years old). My UPS at home are still connected to RS232 ;-)

All UPSes are configured to only return power to the connected computers once they have reached a minimum charge (e.g. 25%), Shutdown is reliably completed if started at the five minute remaining mark.
 
You must log in or register to reply here.
Back
Top