"If it isn't broke don't fix it"

[getopt]

This is a sentence most of us will agree upon, at least those who have some experience. But wait! What about the context and situation?

We know that even a car needs some maintenance. But what are we talking about when we use such sentences in IT context? When is a system considered not to be broken? When it is not failing when used by users? When it runs really fast? When the boss, who knows as much about IT as my dog has nothing to complain about? When the system runs just fine for years now without any updates? When the system is mis-configured and nobody knows/cares?

Did you ever get the experience after some configuration work that you now have the perfect system? How long did this feeling last when you thought you’re "done"?

 

[wblock@]

There are not a lot of static systems any more. Pretty much anything connected to the Internet is going to need continuous maintenance, like security updates. These systems are never really "done", and any time they are not up to date on patches, they can be considered at least partially broken.

 

[SirDice]

If there are security patches it means it's broken and the patch needs to be applied. I've met way too many people thinking: "It works I'm not going to touch it." And sooner or later the system gets hacked and they ask me how it happened. "But it's been working for years!" The question is never: "Will it get hacked?", the question is: "When will it get hacked?"

Also keep in mind IT is a fluent business; things change, nothing is ever 'static', nothing is ever finished.

 

[arp242]

A classic article on the subject: http://www.joelonsoftware.com/articles/fog0000000069.html.

An example from FreeBSD: the new package system (PKGNG) introduces a lot of bugs, problems, missing (essential) features, etc. while the old package system, while it had its problems, worked quite well.

A better strategy would have been to start improving the old pkg_* tools, even if you change (for example) the database format, you're not changing >90% of the code, which isn't broken and therefore doesn't need "fixing".

So, "If it ain't broke, don't fix it" also applies to parts of the system, rather than the system as a whole.

If there are security patches it means it's broken and the patch needs to be applied. I've met way too many people thinking "It works I'm not going to touch it". And sooner or later the system gets hacked and they ask me how it happened. "But it's been working for years!" The question is never "will it get hacked", the question is "when will it get hacked".

Also keep in mind IT is a fluent business, things change, nothing is ever 'static', nothing is ever finished.

I think the problem here is that it's often difficult to apply just the security patches. What you get instead, is the security patches plus a whole bunch of other stuff, which is potentially backward incompatible. A very notorious example is PHP, where each PHP version introduces small changes to the language and/or defaults; it's very difficult to check if your programs will run with the new PHP version beforehand, since PHP does not statically analyse well.

This is why things like CentOS exist. You probably shouldn't run FreeBSD on a server unless you have time every week to update it.

 

[beatgammit]

This is why things like CentOS exist. You probably shouldn't run FreeBSD on a server unless you have time every week to update it.

This seems like a lot of FUD. CentOS isn't meant for lazy admins who can't be bothered to maintain their system, it's meant for admins who want the reliability of RHEL without the support cost (meaning they'll provide the support themselves). Also updating is only really necessary if there's a security issue, which is far less frequent than once per week (once a month is usually more than sufficient), and most advisories (i.e. OpenSSL, PHP) also affect nearly all operating systems that use it (even CentOS/RHEL/Debian).

If you don't have the time to manage your system yourself, opt for managed hosting. Even something as stable as CentOS can break (that's why RHEL provides support). If you have some time to run updates, but don't have time/resources to actually fix problems when they happen, get a support contract (i.e. RHEL). If you don't want to pay for support, but also don't have a lot of time to fix breakage, opt for a reliable system like FreeBSD (or CentOS, Debian, etc.) and allocate a few hours "just in case" each time you run updates.

However, when relying on software not maintained by the security team (like FreeBSD ports/packages), it's your responsibility to manage updates. You don't have to upgrade your ports/packages each time updates are released, but you should update if there's a security issue in ports/packages (watch /usr/ports/UPDATING) or the base system (subscribe to security announcements). You should always test after every update, even if it's just a security patch.

Even if CentOS were more stable than FreeBSD (which it may), FreeBSD has other security features (e.g. jails) that mitigate security vulnerabilities that shouldn't be overlooked when assessing fitness to use as an operating system. Because of this, depending on the type of vulnerability, it may not be necessary to update something like PHP if it's properly quarantined in a jail.

There may be occasional breakage on any system that provide updates; if this is unacceptable and you don't have time to fix it, then paying someone else to manage it for you is the only logical course of action.

 

[SirDice]

However, when relying on software not maintained by the security team (like FreeBSD ports/packages), it's your responsibility to manage updates. You don't have to upgrade your ports/packages each time updates are released, but you should update if there's a security issue in ports/packages (watch /usr/ports/UPDATING) or the base system (subscribe to security announcements). You should always test after every update, even if it's just a security patch.

There are no security warnings in /usr/ports/UPDATING, it's not its purpose. You're better off keeping track with pkg-audit(8) or http://www.vuxml.org/freebsd/.

 

[arp242]

There are no security warnings in /usr/ports/UPDATING, it's not its purpose. You're better off keeping track with pkg-audit(8) or http://www.vuxml.org/freebsd/.

But VuXML isn't necessarily complete, at least, I never assume it is, it's up to the port maintainers to add entries, and if they don't know about security issues, then it won't be there. For example, the first vulnerability that jumps into my mind is the recent one in cryptmethod=blowfish in Vim, which was discovered to be insecure (you need to use cryptmethod=blowfish2 now), but this is not recorded in VuXML.

This seems like a lot of FUD.
[snip]
There may be occasional breakage on any system that provide updates; if this is unacceptable and you don't have time to fix it, then paying someone else to manage it for you is the only logical course of action.

This is not "FUD"; on FreeBSD, I have to have the latest version of a port to have all the security and stability fixes (as said before, VuXML is incomplete). Not so with CentOS, where those fixes are backported.

Of course CentOS is not perfect, impervious to breaks, or suited for all goals, but it's a VERY large advantage to be able to update your packages and not be afraid that some feature has been deprecated. Have you read the PHP example in my original post? If I want to run a secure version of PHP 5.2 on FreeBSD, then that will be very difficult; on CentOS it's just a yum install php.

This has nothing to do with being lazy (as if that would be a bad thing BTW).

Jails stop a very limited set of problems; they won't prevent you from leaking all your credit cards, for example. It's really a second line of defence, and you should always try to have the first line as good as possible (by having no known security problems).

 

[SirDice]

But vuxml isn't necessarily complete, at least, I never assume it is, it's up to the port maintainers to add entries, and if they don't know about security issues, then it won't be there.

True, but if you're only looking at /usr/ports/UPDATING you will never see any security advisories

 

[kpa]

True, but if you're only looking at /usr/ports/UPDATING you will never see any security advisories

To add to that, /usr/ports/UPDATING doesn't have a strictly enforced format and that leaves it open to misinterpretations and errors when it's parsed by automated tools. The VuXML has a strictly enforced format being XML after all.

 

[beatgammit]

There are no security warnings in /usr/ports/UPDATING, it's not its purpose. You're better off keeping track with pkg-audit(8) or http://www.vuxml.org/freebsd/.

I guess I was mistaken because I typically update ports, then run pkg-audit(8), then read /usr/ports/UPDATING if there's anything to fix. I don't actually read the whole file, just the bits that are relevant to what I want to update.

This is not "FUD"; on FreeBSD, I have to have the latest version of a port to have all the security and stability fixes (as said before, VuXML is incomplete). Not so with CentOS, where those fixes are backported.

I guess I don't have much experience with PHP on FreeBSD and I was mostly recalling Samba, which has a port for minor releases (net/samba35, net/samba4, net/samba41). I see your point, but I stand by my statement that FreeBSD makes a fantastic server even if you don't update every week.

 

[wblock@]

I’m not sure what you would define as a static system. Regarding modern operating systems there are none, IMHO. I’m not sure if a definite state machine, which are more likely to be found on microcontrollers would meet the criterion static. Can you please elaborate what you meant by “static system”?

Sure. For an example, think of a computer driving some industrial machinery. It's not hooked up to the net. The jobs it does are the same, and any obvious bugs were worked out long ago. There are lots of these systems.

In contradistinction to that suppose you have an air gapped LAN or isolated workstation. Once installed you have confidence in your installation but after some times there are signs that suggest to no more trusting the installation. The system was maintained properly as every security advices were patched. Would you consider this system now broken or is it not?

If there is ever a doubt, it needs investigation.

The central question is still when a system is to be considered as broken. Without a case related risk analysis there is likely no general answer to that.

True. I don't think of "broken" as a binary condition. Time is involved, so immediately after setup, a system might be 100%. But like any sort of thing people build, time wears it down. Parts fail, assumptions change, the whole thing accumulates entropy in the form of rust, dust, cat hair, corrosion (both physical and digital), and weakening of immunity to security attacks.

 

[sulman]

I will say the old aphorism 'better is the enemy of good' has been true for me many times. I've done more damage trying to improve things, than just being happy with regular maintenance.

FreeBSD was the first thing I used that taught me to really read, if that makes sense. It's highly reliable, but you have to watch what you're doing, because the ports in particular can change a lot. A recent example I can think of is sysutils/bacula-server which changed configuration directories. The client too also underwent a major version change. I came from RHEL/CentOS and got used to these things being pretty much maintenance free, especially if you're using RHEL's Satellite where things essentially look after themselves.

 

[hitest]

I try my best to keep all of my PCs patched and to read security bulletins. A proactive approach to security is best.

 

[ondra_knezour]

Sure. For an example, think of a computer driving some industrial machinery. It's not hooked up to the net. The jobs it does are the same, and any obvious bugs were worked out long ago. There are lots of these systems.

...and then Stuxnet visits your nuclear installation

Those SCADA systems have been addressed many times as potentially vulnerable, because of lack of upgrades (you may have to recertify each patchset, you don't have any other nation-wide power grid on hand for tests before in-production deployment, some embedded components are not longer supported etc. - each potential upgrade is painful, risky or not available at all) and controlling potential high-value targets, from local (factories) to nation-wide (power grids and other utilities). See http://en.wikipedia.org/wiki/SCADA#Security_issues as starter for various links.

 

[wblock@]

Those systems were hooked up to networks, either directly or indirectly through a computer that was used to control them. It's an example that shows every possible mode of access should be considered. Some businesses disconnect USB ports for the same reason.

 

[jdonaghy]

I generally take issue with that statement. Even things that aren't broken can usually be improved. Why settle?

It reminds me of my brother, though. Back when he didn't have his own PC, he was always coming in my room and using mine. At the time, I was constantly changing distributions and desktop environments and window managers, etc. He said that to me one day: "If it's not broke, don't fix it." I tried to make him understand it had nothing to do with "fixing", but I don't think he ever did. But since it was my computer, I just kept on with my tweaking.

 

[shepper]

My sense is that a good part of this discussion depends on the projects development model. Using the three major BSD's as an example:

FreeBSD: At the time I write this 10.1, 9.3 and 8.4 are available for a number of different architectures. The 8.* release series is over five years old. I would be very surprised if someone put the effort into UEFI boot of 8.*. Binary package managers, available file systems, device allocation and X.Org all vary. The user is left to decide which option provides, to use the OP's term, the "perfect" system.

OpenBSD: 5.5, 5.6. Binary package managers, file systems and device allocation are unchanged. The project will only support the system as "perfect" for no more than twelve months and preferably six months. For my desktop needs, this is the best compromise.

NetBSD: No matter what version you pick, it is never perfect. The project exists largely for developers striving for perfection.

CentOS/RHEL: Development occurs in Fedora. By the time it makes its way into CentOS/RHEL, the basic code has been used/debugged/optimized by the Fedora community for several years.

 

[SarK0Y]

I have a testbed and have a workhorse as well. Thereby ye have begotten a painless (as much as 'tis possible) scheme to find new solutions without risks of ruining the work of yours with endless downtime. Yes, the testbed is restricted by available resources.