Why do we need d-bus?

[kpedersen]

Which means that a web browser on the "Isolated PC" can download and execute malware.

Yep, like I mentioned (only half-joking); I really wish a web browser was useful offline . Firefox is probably the most trustworthy of the scummy bunch but they are still very clearly eroding privacy with all their "services".

However my main concern when running Windows is keeping the operating system as a whole offline. I.e to interrupt telemetry, consumer updates and any other issues. The start menu or lock screen has absolutely no business going online to update the visual adverts it displays.

I would always recommend running browsers in a VM (or Jail if Windows was perhaps more useful), however then you might as well simply not choose to run Windows in the first place!

 

[Cath O'Deray]

I've never run into a cli program that needed dbus. You got an example?

Not a CLI, but an unusual example of usage might be jack.

<{link removed}>

<{link removed}> – one of two ways to start.

 

[A. D. Sharpe Sr.]

There's two different complaints here: libdbus is kinda junky and why do apps use DBus because the version we have to use on FreeBSD is junky.

Apparently the Linux people have redone DBus twice: Once for SystemD in userspace in 2013 (which apparently works quite well) and once for dbus-broker in kernelspace in 2017 (Fedora uses this as of 2019 and also seems to work well). Linux has more users than FreeBSD, so if they solved their junky DBus problem with a Linux-only solution, then they have mostly extracted themselves from the problem and have taken the manpower to solve issues in libdbus that would affect FreeBSD with them. This explains why libdbus remains junk: The problem was solved in a place FreeBSD can not go.

There's a lot of apps that use DBus, so the most practical plan would be to fix the most pressing issues plaguing libdbus for people using apps that need DBus. DBus isn't going away, so if it makes it into the Linux kernel, I would think the need for FreeBSD to have a decent implementation will grow. Would seem like a Foundation project.

Most of the other ideas in this thread appear to only to solve the problem of "I don't like trying to install some app and then discovering it pulls in DBus" such as:

* Use some other IPC: Untenable and unreasonable. "Some other IPC" requires reprogramming all apps that use DBus.
* Remove all the DBus apps from ports: Also unreasonable for people who use those apps.
* Turn DBus off in make.conf: Reasonable, just don't plan on using packages. There may be ports that will not operate without DBus and can not respect this, however.

This is probably the best comment I've seen on this thread. It has me wondering "what exactly would it take in order to actually FIX the FreeBSD DBus problem?" Because we're using a port, right? Maybe its time for us to actually have a native solution. I mean, of course ours it broken -if it's a port that apparently hasn't been developed as a real solution for FreeBSD.

 

[zirias@]

Disclaimer: I still didn't look into dbus implementation, but it's installed on some of my systems as a dependency and so far, it "just worked". So sure, there might be quality issues and I was just lucky so far not to run into them.

This is probably the best comment I've seen on this thread. It has me wondering "what exactly would it take in order to actually FIX the FreeBSD DBus problem?" Because we're using a port, right? Maybe its time for us to actually have a native solution. I mean, of course ours it broken -if it's a port that apparently hasn't been developed as a real solution for FreeBSD.

I really really think this is the wrong way to go. I've seen this reflex (let's put stuff in the kernel -- yep, addressing the post you quoted here) far too often. A main driver seems to be "performance". A long time ago, Linux had a web-server in-kernel.

This is just plain nuts. There are IPC primitives offered by the system, some of them are standardized by POSIX. These should perform as well as possible. (I remember L4, the microkernel designed at University of Karlsruhe, took this to an extreme, optimizing IPC with custom x86 assembler code, even with special-case paths using CPU registers where possible, etc..). Higher level protocols should never make it into the kernel. Why? Security and stability. Code should never be executed with privileges it doesn't really require. Ok, sure, there's room for compromise. Your classic "monolithic" kernel will probably implement things like IP, TCP, filesystems, etc, and the reason will be performance. But you have to draw a line somewhere. To me, this line must be drawn at anything that's clearly "application layer". And I'd say a message bus system designed for desktop needs clearly fits there.

 

[hardworkingnewbie]

Apparently the Linux people have redone DBus twice: Once for SystemD in userspace in 2013 (which apparently works quite well) and once for dbus-broker in kernelspace in 2017 (Fedora uses this as of 2019 and also seems to work well). Linux has more users than FreeBSD, so if they solved their junky DBus problem with a Linux-only solution, then they have mostly extracted themselves from the problem and have taken the manpower to solve issues in libdbus that would affect FreeBSD with them. This explains why libdbus remains junk: The problem was solved in a place FreeBSD can not go.

For god's sake, please check your so called "facts" before making such false statements.

SystemD is not providing Dbus services, it uses them. Dbus-broker runs in userland, not kernelspace!

There was once made an effort by the systemd guys called kdbus to integrate dbus into kernel space. When it was handed over for integration, Linus Torvalds ran a profiler on userland dbus and came to the conclusion that it was just a piece of badly written, slow garbage - to quote him "pure shit user space code". And instead of creating a kernel component in the hope to make dbus quicker, they should instead improve userland dbus because that's not what a kernel is there for. Just performing quicker if it could be also done in user spaceis not reason enough to accept something into the kernel. To quote again: "We don't merge kernel code just because user space was written by a retarded monkey on crack."

So dbus-broker is the result of kdbus not being merged into the Linux kernel: a new userland implementation of dbus, which is being optimized for speed and should overcome the problems of the old implementation. Furthermore dbus-broker does use extensively kernel features which are only available under Linux right now. Since its authors are coming from the systemd crew, they don't care much about portability.

 

[zirias@]

When it was handed over for integration, Linus Torvalds ran a profiler on userland dbus and came to the conclusion that it was just a piece of badly written, slow garbage - to quote him "pure shit user space code". And instead of creating a kernel component in the hope to make dbus quicker, they should instead improve userland dbus because that's not what a kernel is there for.

Oh thanks. It renders my previous response kind of pointless. But it's good to hear there's some sanity left in Linux world. Takeaway: mistrusting Linux shouldn't go with trusting any posting on FreeBSD forums ?

 

[A. D. Sharpe Sr.]

Disclaimer: I still didn't look into dbus implementation, but it's installed on some of my systems as a dependency and so far, it "just worked". So sure, there might be quality issues and I was just lucky so far not to run into them.


I really really think this is the wrong way to go. I've seen this reflex (let's put stuff in the kernel -- yep, addressing the post you quoted here) far too often. A main driver seems to be "performance". A long time ago, Linux had a web-server in-kernel.

This is just plain nuts. There are IPC primitives offered by the system, some of them are standardized by POSIX. These should perform as well as possible. (I remember L4, the microkernel designed at University of Karlsruhe, took this to an extreme, optimizing IPC with custom x86 assembler code, even with special-case paths using CPU registers where possible, etc..). Higher level protocols should never make it into the kernel. Why? Security and stability. Code should never be executed with privileges it doesn't really require. Ok, sure, there's room for compromise. Your classic "monolithic" kernel will probably implement things like IP, TCP, filesystems, etc, and the reason will be performance. But you have to draw a line somewhere. To me, this line must be drawn at anything that's clearly "application layer". And I'd say a message bus system designed for desktop needs clearly fits there.

I think you have misunderstood my meaning. My point is that our port should be scrapped & a native DBUS implementation should be written -NOT that DBUS should be in the kernel. I DO agree with an earlier comment that the System V STREAMS & the Solaris IPC implementations should be seriously considered for being added to the kernel (and maybe a BSD-specific implementation of DBUS could be built on top of them). But there is no way that DBUS should be done in the kernel.

 

[A. D. Sharpe Sr.]

For god's sake, please check your so called "facts" before making such false statements.

SystemD is not providing Dbus services, it uses them. Dbus-broker runs in userland, not kernelspace!

There was once made an effort by the systemd guys called kdbus to integrate dbus into kernel space. When it was handed over for integration, Linus Torvalds ran a profiler on userland dbus and came to the conclusion that it was just a piece of badly written, slow garbage - to quote him "pure shit user space code". And instead of creating a kernel component in the hope to make dbus quicker, they should instead improve userland dbus because that's not what a kernel is there for. Just performing quicker if it could be also done in user spaceis not reason enough to accept something into the kernel. To quote again: "We don't merge kernel code just because user space was written by a retarded monkey on crack."

So dbus-broker is the result of kdbus not being merged into the Linux kernel: a new userland implementation of dbus, which is being optimized for speed and should overcome the problems of the old implementation. Furthermore dbus-broker does use extensively kernel features which are only available under Linux right now. Since its authors are coming from the systemd crew, they don't care much about portability.

This pretty much confirms my understanding that we need to write an implementation of DBUS that does for us what dbus-broker does for Linux.

 

[ralphbsz]

I DO agree with an earlier comment that the System V STREAMS & the Solaris IPC implementations should be seriously considered for being added to the kernel (and maybe a BSD-specific implementation of DBUS could be built on top of them).

Perhaps. But only if we identify a good use for them. And only if manpower for it can be found, without disrupting more urgent problems.

As a general rule, a lot of functionality that 20 years ago was put into kernels is moving into userspace today. Why? Because it's faster: user/kernel transitions are expensive, and the bulk of the code runs in userspace anyway. That includes locking, RDMA, zerocopy networking, and storage IO. So before FreeBSD seriously commits to adding queuing mechanisms to the kernel, it should consider alternatives.

 

[A. D. Sharpe Sr.]

Perhaps. But only if we identify a good use for them. And only if manpower for it can be found, without disrupting more urgent problems.

As a general rule, a lot of functionality that 20 years ago was put into kernels is moving into userspace today. Why? Because it's faster: user/kernel transitions are expensive, and the bulk of the code runs in userspace anyway. That includes locking, RDMA, zerocopy networking, and storage IO. So before FreeBSD seriously commits to adding queuing mechanisms to the kernel, it should consider alternatives.

Even in microkernels (where most things are pushed into userspace), IPC is typically a part of the kernel itself.

 

[msplsh]

For god's sake, please check your so called "facts" before making such false statements.

I apologize for mortifyingly offending you by making an error while making a point that didn't depend on it.

 

[shkhln]

a native DBUS implementation should be written

Here you go: devel/dbus.

 

[Vull]

This pretty much confirms my understanding that we need to write an implementation of DBUS that does for us what dbus-broker does for Linux.

How soon can you start?

 

[zirias@]

I think you have misunderstood my meaning. My point is that our port should be scrapped & a native DBUS implementation should be written -NOT that DBUS should be in the kernel.

As I said, regarding putting stuff into the kernel, I was referring to the post you quoted. But maybe I didn't make one point clear enough: If in-kernel is off the table (and thankfully everyone seems to agree on that), what's the point of having a custom implementation? Something like d-bus can be implemented portably, just needing POSIX IPC mechanisms. If there are issues with the current code, they should be fixed, but not in a FreeBSD-specific solution.

 

[msplsh]

If there are issues with the current code, they should be fixed, but not in a FreeBSD-specific solution.

Nobody is doing it because it's not a problem for the majority of users because they use a different implementation.

 

[ct85711]

The fun part is, there's been several write ups on what can be done that can be done to improve dbus; but they all hit one part. The project has in it's mind that it can't just change how it is because it would break API/ABI. The fun part is; half the changes can easily be done without changing the API/ABI; because that only covers the public interface level. Internally, the API/ABI does not prevent any of the changes. Second, there is already a agreed convention when you intend to break API/ABI (AKA change the major version). The other part is also commonly used, is depreciating methods/functions and slow shift everything over. While that isn't in the C/C++ language directly, there are numerous workarounds to document that.

 

[Jose]

The fun part is, there's been several write ups on what can be done that can be done to improve dbus; but they all hit one part. The project has in it's mind that it can't just change how it is because it would break API/ABI. The fun part is; half the changes can easily be done without changing the API/ABI; because that only covers the public interface level. Internally, the API/ABI does not prevent any of the changes. Second, there is already a agreed convention when you intend to break API/ABI (AKA change the major version). The other part is also commonly used, is depreciating methods/functions and slow shift everything over. While that isn't in the C/C++ language directly, there are numerous workarounds to document that.

The problem is, the API is crap too. It's based on defining interfaces using XML files:

D-Bus API Design Guidelines

Mercifully the XML hype has dissipated and no one uses it in anger anymore*. XML turned out to be hard for both computers and humans to use, and brought along some lovely security fails too. You would have to come up with an equivalent interface definition scheme and convince GNOME, KDE, etc., to use it.

Furthermore, D-Bus specifies a complex wire message format:

D-Bus Specification

This sort of approach unnecessarily constrains the types of payloads you can send over the bus. Kafka brokers, for example, work with opaque arrays of bytes:

Apache Kafka

Apache Kafka: A Distributed Streaming Platform.

kafka.apache.org

This frees you to use whatever message format works for you. Marshaling and unmarshaling arbitrary payloads is a problem that has been solved and solved well multiple times at this point:

Apache Avro

Apache Avro™ Learn More Download a data serialization system Apache Avro™ is the leading serialization format for record data, and first choice for streaming data pipelines. It offers excellent schema evolution, and has implementations for the JVM (Java, Kotlin, Scala, …), Python, C/C++/C#, PHP...

avro.apache.org

Protocol Buffers

Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data.

developers.google.com

https://msgpack.org/index.html

GitHub - capnproto/capnproto: Cap'n Proto serialization/RPC system - core tools and C++ library

Cap'n Proto serialization/RPC system - core tools and C++ library - capnproto/capnproto

github.com

Lastly, this approach of having a single daemon do multiple things shows why the Unix philosophy is so wise. It would be better have a separate IPC protocol, message broker, and serialization format. The last one is a solved problem, as I've pointed out above. For the message broker, I'd leverage Fifolog to write something similar to Kafka. I have no ideas for the GUI IPC interface specification. I'm not a front-end type. I just don't know the space well enough to not embarrass myself.

* XMPP is a notable exception. They use XML to great effect.

 

[drhowarddrfine]

I never had any issues with XML. Interestingly, so many web frameworks and languages use XML without calling it XML now. React, Angular, etc.

 

[ct85711]

The API being crap or not isn't the point (and I agree the API needs to be replaced); the point is that you can not hide behind it because you don't want to acknowledge the issues and/or that it is "too hard". Programming as a whole is a ever evolving process, so trying to hide your head in the dirt doesn't stop the process.

 

[Cath O'Deray]

… I noticed some failed calls to dbus as well, that could have been a factor as well. Not sure how well dbus works with FreeBSD.

As far as I know, it works well enough.

I'm too lazy to re-digest any part of this topic but a recent comment in IRC <{link removed}> was helpful:

{removed}

– that was, in response to a commit that mentioned Plasma Networkmanager:

{link removed}

(I guess, the three bullet points were copied from <{link removed}>.)

 

[Crivens]

I am working with dbus at $DAYJOB, and FreeBSD seems to be faster than linux for some workloads. It has it's place.

 

[Jose]

I read somewhere about the Linux-only replacement for Dbus, complete with a hard Systemd dependency, but I can't find it now. I did find this, though:

FGA: The Desktop Bus Death Rattle

 

[jbo@]

I read somewhere about the Linux-only replacement for Dbus, complete with a hard Systemd dependency, but I can't find it

Sounds like Linux is trying hard to combine the worst aspects of Windows and the worst aspects of MacOS into one system

 

[fr0xk]

Before D-Bus became widely adopted, desktop environments and applications used various methods for inter-process communication and system integration. They relied on direct IPC mechanisms like Unix domain sockets, named pipes, or shared memory, and utilized the X Window System's built-in protocols for tasks such as clipboard management and window event handling. Custom solutions like KDE's DCOP were also common, tailored to specific needs but lacking the standardization of D-Bus. Now desktop applications communicated through configuration files, polling for changes, or using environment variables for session management. Sure these methods somewhat worked, they lacked the level of flexibility, efficiency, and ease of use that D-Bus provides, which standardized IPC across different environments and applications.

 

[zirias@]

utilized the X Window System's built-in protocols for tasks such as clipboard management and window event handling

Uhm, there's just one (extensible) X protocol, which for communication purposes offers some "primitives" (events/messages are builtin for the most basic stuff, but can be synthesized by any client, general-purpose selections with owners and "window properties" are offered, X design philosophy was "mechanism over policy" after all), so most of the protocols you're probably talking about are those specified on top of these primitives (like in ICCCM and EWMH, but also additional ones).

These protocols, when used correctly, work well for all the common interoperability stuff you'd want on a desktop regarding applications that aren't "part" of that desktop -- unless of course your goal is to eliminate the X server.

I already said that I think d-bus conceptually makes sense for tightly integrated desktop environments. It's certainly better than every desktop coming up with its own solution, and a somewhat complex desktop environment certainly has a need for something like that.

But it annoys me that protocols based on d-bus now also attempt to replace well-working protocols based on X11. I don't want to have dependencies on d-bus in every random application. Case in point, for the x11/xmoji utility I'm currently building (which only makes sense on X11 anyways, not on wayland), I plan to add an optional "tray icon". There's an old protocol supporting that based on the XEmbed protocol, and a newer one utilizing d-bus. With the old one, you can simply have a regular X window for your icon and e.g. use XRender on it. I'll only implement that one, although the large desktops nowadays use d-bus "natively". At least KDE offers compatibility through some proxy service, so I don't see why I should pull in even more dependencies just for such a simple and optional thing ... if a desktop only supports the d-bus variant, well then, no icon there.