FreeBSD has serious problems with focus, longevity, and lifecycle

vermaden

vermaden

Read the whole discussion which starts here:
http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/thread.html#37294

Friends,

I was disappointed to see that 8.3-RELEASE is now slated to come out in
March of 2012. This will be ~13 months since 8.2-RELEASE and is typical
of a trend towards longer gaps between minor releases.

I also see that undercutting the current release before wide deployment
and maturity is continuing. 7.0 came (barely) after 6.3, which was bad
enough, but not as bad as 8.0 arriving with 7.2, and now 9.0 with 8.2.

Finally, the culture of "that's fixed in CURRENT" or "we built those
changes into (insert next major release)" continues to get worse. It's
difficult to escape the notion that FreeBSD is becoming an operating
system by, and for, FreeBSD developers.


The Problems:


Between JohnCompanies and rsync.net, we have nearly one thousand full
blown FreeBSD systems running on three continents. We've been deploying
these systems since 2001 and since "the rift"[1] have been increasingly
subject to the following major issues, listed from most general to most
specific:

1) A widening gap of understanding between the developers and the end
users. Not everyone has a fantastic tool chain and build environment that
allows them to jump around from one snapshot to the next, cost free.
We've got a thousand of these things, and not only are we going to run
RELEASE software ONLY, but we're going to do everything we can to match
that environment up across as large of an installed base as possible.
The daily chatter on the lists about getting stable and getting current,
or moving to the next major release reflects a complete disconnect between
developers and end users.

2) Having two simultaneous production releases draws focus away from both
of them, and keeps any release from ever truly maturing. In January of
2012, we should be on 6.12 (or so) and just breaking ground on 7.0.
Instead, each release gets perhaps two years of focused development before
every new fix is applied only to the upcoming release, and any kind of
support or enthusiasm from the community just disappears.

This means that any serious or wide deployment gets stuck with a bad
choice every two years - keep running something that's already essentially
abandoned, or spend all of that time and money on QA, testing,
documentation, shipping, etc., all over again, just like they did two
years ago.

Of course the retort will be: "we added ZFS and ULE, etc., and those
warranted a full release" - and maybe that's true, but since ZFS is only
now, circa 8.3 (god willing) ready for any kind of prime time
deployment[2], it would have been just fine to "release" it today, in 7.0.

3) Having two simultaneous production releases draws investment away from
FreeBSD, because the relevance and longevity of those fixes are unknown.
I am sure we are not the only organization that either needs new features,
or needs fixes, that just aren't on others' priority lists. In the past,
we've donated time and money[3][4] to try to push some of these things
forward, but it makes less and less sense when the lifespan of any
particular fixes are limited to the shorter and shorter lifespans of the
releases. Why pay to get this fixed today when it's either "already fixed
in CURRENT" or is already irrelevant ? Meanwhile, end users that don't
have the option to hire contract coders just lose out.

4) New code and fixes that people NEED TODAY just sits on the shelf for 8
or 10 or (nowadays) 13 months while end users wait for new minor releases.
Not only does this hurt end users, but it also hurts downstream projects,
like pfsense and FreeNAS.

Here's a good example: somewhere around 2007, a great many PCMCIA network
cards (of interest to pfsense users, like me) just quit working.[5] I
found that this was still the case in 2010. I asked Warner about it, it
got MFC'd, and I donated some hardware towards keeping these devices
maintained. So far so good. But this was in June of 2011 which means
that 8 months later this still isn't released and certainly isn't in
pfsense. Ok, fine - if I'm fiddling with PCMCIA cards in 2011, then
perhaps "get CURRENT" is a reasonable response ... but be honest, do you
have a build environment that allows you to take FreeBSD CURRENT and
generate a new pfsense build from that ? Do any pfsense end users have
that ? I don't. More frequent minor releases would be a boon here.

5) New code and fixes that people NEED TODAY often get pushed into the
next major release, since that's what people are working on anyway.

A few years ago we were dying for new em(4) and twa(4) drivers in FreeBSD
6, but they were applied only to 7 and beyond, since that was the "new
production" release (as opposed to the "old production" release). It's
the same bad choice again: make major investments in testing and people
and processes every two years, or just limp along with old, buggy drivers
and no support.


Suggestions:


Here are the specific actions that I think would dramatically improve the
focus of the project, the experience of real end users, and the ability
for third parties to truly invest in FreeBSD:

- Stop the trend of FreeBSD being an operating system by and for FreeBSD
developers. Think about the processes and costs that large and wide
installed bases incur. Think about the logistics of major upgrades and
the difficulty of running snapshot releases, etc. Remember - if it's not
fixed in the production release, it's NOT FIXED. Serious (large) end
users have very little use for CURRENT.[6]

- Focus on one production release at a time. Preferably for a solid five
years. In addition, provide actual legacy support for that release for
another five years after that. Five years of production and another five
years of legacy would provide a very stable platform for development and
investment, and would signal to large, complex organizations that FreeBSD
takes their needs seriously.

- Release a minor revision every 4 months. This sounds aggressive, but
it's a lot easier if the project isn't simultaneously working on a second
"production" release at the same time.



Thank you for reading this. I look forward to comments and discussion.

John Kozubik
Click to expand...

SOURCE: http://lists.freebsd.org/pipermail/freebsd-hackers/2012-January/037294.html
 
Comments (I've been using FreeBSD in production since 2001):

It would perhaps pay to target new OS life-cycle timeframes to be slightly longer than the typical hardware refresh cycle.

For big business, I suspect that this will be between 3 and 5 years. 3 years is your typical hardware warranty, and this can often be pushed out to extended support for 5 years.

I guess for this to happen, driver backporting (in addition to security fixes) needs to be a priority for at least 3-5 years?
 
Thanks for pointing out the interesting discussion. It was great to see that there were only few "if FreeBSD doesn't change everyone will move to Linux" arguments. As an optimist I see that open discussions like this will bear something good to FreeBSD.

I have noticed that FreeBSD has gotten attention because of ZFS and PC-BSD, and I believe that it is time to consider how end users, not only the developers, see FreeBSD.

About the topic: I would also like to have more frequent minor releases.
 
I also responded to that mailing lists thread, but message awaits approval and its not guaranteed to make it up there since I am not a member of freebsd-hackers group, so here is my 'rant' that I sent today.

This well known 'open secret' FreeBSD problem also hit me many times,
but as experience tells me, all of us would chick-chat here a lot what
can be done to make things better and in the end ... nothing will
happen.

Many of You say 'step-up' and help/do something, I am actually or
tried to do that in the past.

I submit PRs and try to help test them as some developer/commiter will
pick up the PR, submit a patch to test, but it was MANY times that the
response from developer/commiter was way too long that I even DID
NOT HAVE THE HARDWARE anymore ...


Long time ago I had a lot of fun with emulation/virtualization using
QEMU on FreeBSD, so I decided to write a FreeBSD Handbook section
that would cover what I already known and it will help A LOT of people
that would also want to use it the same or similar way.

Here is one of the messages that I sent by then to the mailing lists:
http://lists.freebsd.org/pipermail/freebsd-doc/2007-May/012507.html

... and NOTHING HAPPENED, no one told me what to do next, should
I sent a SGML version or anything ... or just GTFO.

Maybe that is the reason why my HOWTO 'QEMU on FreeBSD' was so
popular before VirtualBox 'happened' on FreeBSD.

And these Handbook pages are still available on my site to this very day:
http://strony.toya.net.pl/~vermaden/FreeBSD-Handbook-Virtualization.htm


I have an issue with my Lenovo X300 laptop that the 'output jack'
for headphone did not worked as advertised, I submitted a PR, some
the developer/commiter helped me to make it work by adding some
'hacks' to /boot/loader.conf (and it worked), but IT WAS NOE FIXED,
these changes was never added to any branch, not even CURRENT, whats
the point of fixing something if it is not even commited to at least
CURRENT so it will not hit anyone anymore?


Lets talk about Ports maintainers for a while, ftp/vsftpd maintainer
for example, one of the options of this port is to provide a RC script
so one will be able to start this FTP server with /usr/local/etc/rc.d
script, but ... ITS NOT ENABLED BY DEFAULT, wtf people?

Now every person that wants to use vsftpd NORMALLY will have to
compile it instead of adding a package, so whats the need for building
all the packages if You provide such useless defaults?


Another example, net/samba*, its WELL KNOWN that samba sucks on
FreeBSD with current selected defaults for this port, the current
defaults are to DISABLE AIO support, but You only get good
performance with samba on FreeBSD when the AIO is enabled, so
another bunch of useless prebuilt packages, You need to compile
anyway. Its options for FreeBSD, not every other OS on the world,
so why not enable it? FreeBSD Ports are not PKGSRC where many
systems are supported, if some option is 'good for default' why
keep it disabled?


What I have done about these 'Ports issues'? I contacted these
ports maintainers and said that both RC script and AIO support for
samba should be enabled by default by linking to several threads at
FreeBSD Forums that the problem is known and exists ... and I did
not get ANY RESPONSE till this very day, not even a GTFO (which
would probably be better then nothing).

I got these maintainers email addresses from http://freshports.org
page, are they up-to-date there? Maybe that is the problem and
that my mails jest went to /dev/null ... Just checking for sure.


Its not that people does not try to help, a lot tried (and I am still
trying), but its VERY unpleasant to have awareness, that You dedicated
your time, tried to help as much as possible, made some steps to
achieve that ... and no one even cares about that.

At least some should say 'You are doing it wrong' try this and
that, understand that, that is the way it works etc.


Just my $0.02 on that well known FreeBSD problem and 'crisis'.

With kind regards,
vermaden
 
You make some very good points but try not to put too much emotion in your arguments. It really does read like a rant and it diverts the attention away from your valid points. I know it's difficult I regularly struggle with that myself. It's usually best to write what you wanted to say (emotions and all) and leave the email for a day. Read it again the next day, adjust where necessary and then send it.
 
toddnni said:
It was great to see that there were only few "if FreeBSD doesn't change everyone will move to Linux" arguments.
Click to expand...

When reading the thread, I got the impression that FreeBSD is already heading there, so what would you win by going to Linux? Bazar coding, developers detached from what some think of as the real world (who could say?), conflicting destinations, the "get the latest version" cure for all, balkanization on usefullness (some code runs only on version X, which does not like HW Y), ...

But I must admit that I first used FreeBSD with 7.x, so I can not comment on the release strategy as well based as others here. What I see, in the thread and also in the post Vermaden shared here with us, a lot of history repeating.

All the things that are written there are logical, more or less rational, but in the end they do not add up with each other. What to make of it, I am not sure. There will be more time required to think about it, but then what?
 
Unless I'm misunderstanding, isn't this the same thing going on with browsers? Most browsers were putting off major version changes while smaller changes languished waiting for major projects to be completed. So they now have quicker releases that will get those small changes in and not be held up by the major projects. Hence Mozilla's 6-8 week cycle along with Chrome.
 
toddnni said:
About the topic: I would also like to have more frequent minor releases.
Click to expand...
If I'm not mistaken, OpenBSD has a release once every six months. I think such a cycle allows for quicker introduction of (finished) components/features/drivers from -CURRENT into -RELEASE. I therefore support the idea of more frequent (minor) releases.

Fonz
 
I suspect part of the problem may be attempting to include a 100% properly functional PORTS tree in a release. The ports tree is huge, and growing.

Limit the release to the core OS, and releases will be much easier to manage.

Ports could perhaps have a "tested with v X.XX" tag so that fairly soon after (most likely even before) release the most common ports would be confirmed as working.

Including most of the open-source world in an OS release just seems to be biting off more than required, in my opinion.

Alternatively, maybe a "core ports" or "tier 1 ports" tree subset could be checked off for release with the rest being best effort (and stuff like attempting to provide tier 1 support for "Desktop environment flavour of the month" could be left to PC-BSD)?
 
throAU said:
Alternatively, maybe a "core ports" or "tier 1 ports" tree subset could be checked off for release
Click to expand...
That does of course come at a significant risk of sparking countless (and often silly) debates about which ports are tier 1 and which are not... ;)

Fonz
 
Personally I would like to see more focus on speed & stability(Bug Fixes) and less focus on new features.

More focus on current generation hardware than obscure hardware. Linux has this same problem. I see all the time in the kernel mailing list some one posting a new driver for hardware that hasn't been in active circulation for 5 years.

Xorg maintains support for chipsets, That haven't been made for the last 20 years.

ALSA has the same problem.

I think a lot of the core system tools can be simplified, "How many features does the Cat command really need?" Once again not specific to FreeBSD.

Simplify Filesystem Hierarchy, not only on the System but in the core src tree.

I have LOTS of ideas and I am working on most of them.
 
zester said:
More focus on current generation hardware than obscure hardware. Linux has this same problem.
Click to expand...
Current hardware support largely depends on either manufacturers being willing to publish specs or developers being willing to reverse-engineer the d*** thing. The latter can be a very frustrating process. Plus the hardware must be made available to developers.

zester said:
I see all the time in the kernel mailing list some one posting a new driver for hardware that hasn't been in active circulation for 5 years.
Click to expand...
People are free to write such drivers and to submit them. That doesn't mean it ties up the entire development team and keeps them from doing their thing.

zester said:
Xorg maintains support for chipset's, That haven't been made for the last 20 years.
Click to expand...
FreeBSD is not X.org. If they wish to support such chipsets, that's their business.

zester said:
I think alot of the core system tools can be simplified, "How many features does the Cat command really need?"
Click to expand...
Let's start with what POSIX specifies. Other features can be phased out (not dropped right away) if there's little use for them and their removal doesn't break a butt-load of other stuff. Most things are there for a reason though.

Perhaps you should create a fork called Diet BSD or something ;)

Fonz
 
fonz said:
Current hardware support largely depends on either manufacturers being willing to publish specs or developers being willing to reverse-engineer the d*** thing. The latter can be a very frustrating process. Plus the hardware must be made available to developers.


People are free to write such drivers and to submit them. That doesn't mean it ties up the entire development team and keeps them from doing their thing.


FreeBSD is not X.org. If they wish to support such chipsets, that's their business.


Let's start with what POSIX specifies. Other features can be phased out (not dropped right away) if there's little use for them and their removal doesn't break a butt-load of other stuff. Most things are there for a reason though.

Perhaps you should create a fork called Diet BSD or something ;)

Fonz
Click to expand...


1. That's not really what I mean. I was saying that current generation hardware that is already supported in the kernel. Should just work and work well out of the box.

2. In regards to kernel dev's and Xorg. I was just using those two as an example on how we give far to much attention to legacy hardware when I personally think that current generation supported hardware deserves a little bit more polishing. Legacy hardware could always be a side project.

3. If a code base depends on any specific core system "application". Then your already totally screwed.

Just offering some suggestion and trying to get dialog going. ;)
 
fonz said:
That does of course come at a significant risk of sparking countless (and often silly) debates about which ports are tier 1 and which are not... ;)

Fonz
Click to expand...

Presumably similar discussion occurs on what to contain within the base system.

I maintain that trying to distribute a significant portion of all the worlds open source unix software as a supported ports tree is perhaps too much to ask.

I guess it comes back to the "lack of focus" being discussed in that thread.

PC-BSD focuses on the desktop. Is it worth the FreeBSD dev's time to ensure that desktop oriented ports work 100% for a FreeBSD release (just as an example) - rather than leaving that for PC-BSD? Is that the focus of the FreeBSD project?

But you're right, perhaps some sort of metrics need to be collected to determine the most popular usage patterns for FreeBSD so focus can be turned to those. Or maybe some sort of priority placed on ports that are used in mission-critical infrastructure.

I suspect, as implicated in the original mailing list thread that many of the high profile commerical users of FreeBSD are very concerned about support time-frame and have a fairly common core set of ports in use between them.

The fact that say, Gnome or KDE (as a hypothetical example) is broken would not matter to them at all. On the other hand, something like SSH, BIND, SENDMAIL, APACHE (or other network infrastructure type services) breaking is going to be a showstopper for those attempting to maintain five 9s reliability.

Perhaps a "server" spin of FreeBSD is required? But to me, thats what FreeBSD is for. If you want to run a shiny desktop unix, you'd be better off with Mac OS, Linux or at least PC-BSD, as that is their focus.
 
I do not expect my next message to 'show up' on the freebsd-hackers ML, so I also put it here ...

A simple sollution (at least for a start), for backporting
various bugfixes from STABLE to RELEASE.

Currently we have /var/db/pkg 'db' for installed ports,
where an installed port is like /var/db/pkg/portname-1.0
lets provide another one, /var/db/patch, a separated
'repository' that would list installed patches/backports
from STABLE to RELEASE, for example:

# pkg_info
aspell-0.60.6.1 Spelling checker with better suggestion logic than ispell
automake-1.11.1 GNU Standards-compliant Makefile generator (1.11)
binutils-2.22 GNU binary tools
bison-2.4.3,1 A parser generator from FSF, (mostly) compatible with Yacc
blogbench-1.1 Performance Test of Filesystem I/O
(...)

# patch_into
network-intel_drivers-em-1.0 Some fancy descritpion here
usb-hubfix-3.0 Some other fancy descritpion

Similar for patch_add(1)/patch_delete(1) etc.

LOGIC/THEORY:

Adding a PATCH network-intel_drivers-em-1.0 would first move all
files that will be overwritten to /var/db/patch/files, for example
/boot/kernel/if_em.ko to
/var/db/patch/files/network-intel_drivers-em-1.0/boot/kernel/if_em.ko
and then install the new backported/updatet files into the base system
or should I say RELEASE.

The only other thing needed to do is to make freebsd-update AWARE of
the files that are under /var/db/patch/files/*/ to omit them when
checking the checksums for updating process.

I will not write that as I only know basics of C programming
and creating this in SH and then porting it to C seems useless
efort to me.

Regards,
vermaden
Click to expand...
 
@vermaden: Nice idea, would like to see this also. And, while I may have the skills to write something like that I seriously lack in the time department to try this. :(
 
Personally my main headache is the interface between users and developers: GNATS. Not everyone has enough time to dedicate sufficient work that warrants commit rights, and the step down from that seems like a giant leap into frustration for contributors. I haven't seen the developer backend of GNATS, but if it's as archaic as the user facing bits then all the ignored PRs are understandable. There's got to be a way of improving commiter<->contributor relations...

Vermaden, I hear you about retarded port defaults. Maybe more discussion about this can lead to a Ports Standards policy that can be added to the already excellent Porter's Handbook?
 
The question that really needs to be asked is; What type of operating system is FreeBSD trying to be? If it's trying to be a Production server OS, then the release dates at the moment are too short... Something like solaris or RHEL/CentOS would be a perfect model to follow. At the moment the FreeBSD release lifecycle seems very bursty and erradic, at the end of the day new version numbers really don't matter to anyone[3]. Actually the more i hear of software like Chrome and Firefox moving from version 6->7->8 the less excited i get, the version number is not deserved.

I commend the work of the FreeBSD development team and try to also help out where i can, but it seems like the project is in a huge rush to get no where fast. I'm not a developer but a SysAdmin and i can hold my hand across my heart and say i much prefer longer support for OS versions and more stable code than the barrage of new features and releases. I think that most people would agree that they would be happy to sit on the same version, as long as bugfixes and stability/security improved.

I think the feeling i get from the OS is it's trying to spread itself too thin over a large piece of work. Any OS needs to understand it's strengths and understand that it cannot be the jack of all trades. FreeBSD should not forget it's niche that it filled, which made it once a widley deployed OS... Speed, Stability & Lightness. Sometimes there is no option but to impliment a hetrogenous server farm, freebsd should not try and capture the entire market.

Personally i find it hard to recommend FreeBSD for the environment i work in due to the rapid releases, in favor of something like CentOS which is released far less often.

As a member of the community, Stability and performance as my key concerns. I get more excited hearing that FreeBSD was used to serve some huge capacity website and had no issues/downtime etc than hearing that new features have been implimented.

A poll maybe?

[1] http://upload.wikimedia.org/wikipedia/en/timeline/90aec7d902c1d610e90720ebc42fabd4.png
[2] http://dag.wieers.com/blog/sites/dag.wieers.com.blog/files/centos-intro-1.3-en.png
[3] http://upload.wikimedia.org/wikipedia/en/timeline/7cff5e8e03a4c6593fcf9351c6110ab4.png
 
shitson said:
The question that really needs to be asked is; What type of operating system is FreeBSD trying to be?
Click to expand...

Exactly. This is, I guess what I was trying to articulate; releases should keep in mind the core userbase and focus on the platform's strengths.

FreeBSD currently is not a real contender for a typical end user desktop - the level of commercial application support simply isn't there yet. Sure, you can use it for that, but it's not really what it is best at. You'd probably be better off with Linux, OS X or even Windows (barf). Let the PC-BSD guys focus on that, and focus on refining and polishing the FreeBSD core.

Trying (in vain) to make it all things to all people at the expense of not satisfying (or even alienating) the core userbase (high performance/stable servers) risks satisfying no one.

Given that FreeBSD is not (currently) a major desktop player, bleeding server users is only going to lessen the reason for end users to consider playing with it on their desktop - if it's no longer used on servers and is has less desktop software support, why would you bother? On the contrary, if FreeBSD continues to be a rock solid server foundation and thus gets more commercial support, I suspect that more desktop users will be inclined to try it out.

2c...
 
shadow discussion?

Dear friends,

I'm wondering what this discussion might be good for if it {has already been|still is} discussed at our fine mailing lists? Leading the same discussion in different threads at different places does not buy anything as far as I can see. It's nothing more than just a waste of (human) resources.
 
vwe@ said:
Dear friends,

I'm wondering what this discussion might be good for if it {has already been|still is} discussed at our fine mailing lists? Leading the same discussion in different threads at different places does not buy anything as far as I can see. It's nothing more than just a waste of (human) resources.
Click to expand...

Well, for most of those discussing things here, it buys convinience. I am on this forum, but not on that mailing list. I would need to first set up an account, join, ...

PS: How close to Berlin are you? /me is at the lion, near the wolf :)
 
Crivens said:
Well, for most of those discussing things here, it buys convinience. I am on this forum, but not on that mailing list. I would need to first set up an account, join, ...
Click to expand...

Sorry, I don't want to appear harsh but do we have to mirror CNN news here because you don't watch CNN often?

Look, from a developer perspective you (and your attention) can't be everywhere at the same time. It already takes a lot time to read mailing lists. Monitoring every other possible source for valid points is even more complex and time consuming. So mirroring the same discussion to the forum, that has already started in the mailing lists, means repeating the same arguments, producing much more noise and valid points may get less attention (signal:noise ratio gets worse).

Oh and, you don't need an 'account' to read, enjoy and be part of the mailing lists. Just subscribe to freebsd-hackers and you're good to go. It just takes an email address to participate.

PS: How close to Berlin are you? /me is at the lion, near the wolf :)
Click to expand...

I'm trying to figure out where in Berlin a wolf might be (every other castle has got a lion, like in Charlottenburg).

/me is 5k close to the city border, one rail station outside to the west.
 
[offtopic]
It's also possible to send emails to the mailing list even without subscribing to it, and use the mailing list's web interface to get responses.
[/offtopic]
 
You must log in or register to reply here.
Back
Top