Solved Building Linux-originating programs could be easier

[sidetone]

The amount of projects I instead just extract the .c, .cpp, .h files from and quickly knock up my own Makefiles...

I was thinking, we need a Makefile overlay for BSD's, that's separate from the Makefile in ports. One where we don't fork the source, except for the Makefile only. Maybe there could be Makefile tools, that tell us what's needed, and what is called for. The Makefile adjustments could be with the same function as a patch, but part of the ports framework, where we tell it where to get the Makefile, or simply have Makefile.bsd in the ports directory, with a full replacement, or replacement per particular line.

Already you can override some settings like GCC from the source Makefiles with CC in the FreeBSD Makefiles, if it builds correctly.

MAKE_ARGS=      CC="cc"

A difficult one to figure out how to make a Makefile to build in bmake is x11/sxhkd.

Maybe we should learn about Meson (https://docs.freebsd.org/en/books/porters-handbook/book/#using-meson) and its dependency of Ninja, and see if those are worth looking into. Meson and Ninja also rely on Python. I don't know how long Python will be relevant. As a math like scripting language it's good, as for the disappointment of the changes in the newer version and because of Mojo possibly replacing it, we don't know how long those will be relevant. There's also Muon is a Meson substitute, and devel/samurai as a ninja substitute.

I always see ninja pulled in when cmake was used. They're not listed as dependencies, and I don't see it in make config, but ninja can be called on in Makefiles according to https://docs.freebsd.org/en/books/porters-handbook/book/#uses-cmake.

devel/scons has a section in the Porter's Handbook https://docs.freebsd.org/en/books/porters-handbook/book/#using-scons. devel/bam which uses Lua is also interesting.


IIRC Makefile.am is a RedHat construct.

 

[zirias@]

I was thinking, we need a Makefile overlay for BSD's, that's separate from the Makefile in ports. One where we don't fork the source, except for the Makefile only. Maybe there could be Makefile tools, that tell us what's needed, and what is called for.

I don't think this is a good idea ... not only will it be a horrible lot of work with really complex software, you also have a very high risk to get something wrong. In most cases, it makes much more sense to get the original upstream build system to work, as is done right now.

Maybe we should learn about Meson (https://docs.freebsd.org/en/books/porters-handbook/book/#using-meson) and its dependency of Ninja, and see if those are worth looking into. Meson and Ninja also rely on Python. I don't know how long Python will be relevant. As a math like scripting language it's good, as for the disappointment of the changes in the newer version and because of Mojo possibly replacing it, we don't know how long those will be relevant. There's also Muon is a Meson substitute, and devel/samurai as a ninja substitute.

USES=ninja already has an option to use samurai instead. Similar could be done for meson vs muon. But it will never work for every port, because of subtle incompatibilities. And just avoiding python isn't a pressing reason nowadays, many build system rely on python (including e.g. the one chromium uses).

I always see ninja pulled in when cmake was used. They're not listed as dependencies, and I don't see it in make config, but ninja can be called on in Makefiles according to https://docs.freebsd.org/en/books/porters-handbook/book/#uses-cmake.

So? cmake can generate files for different tools, ninja is one of them and it's the preferred one because it has the best build performance, which is quite relevant for large projects. Here's another issue with trying to use bmake for everything instead, it would make the build of really large software considerably slower.

IIRC Makefile.am is a RedHat construct.

It isn't. It's the input file for GNU automake, which is a part of GNU autotools and used in almost any GNU software.

 

[diizzy]

Just use CMake or Meson which makes everyone's life easier irregardless of OS and platform, this is where projects in general are going anyway...

 

[bsdcode]

Has anyone experience with xmake? It looks like a sane and lightweight approach to a meta build system.

BTW: devel/xmake is a dead port for an unrelated dead project.

 

[diizzy]

Please just use something common, there's a lot of re-inventing the wheel projects and in 99% of all cases they tend to fall short and die.

Just to mention a few "odd" systems we need to cope with already waf, scons, bazel, b2 (boost), imake (legacy), homegrown Makefiles / configure scripts

 

[kpedersen]

Just use CMake or Meson which makes everyone's life easier irregardless of OS and platform, this is where projects in general are going anyway...

I don't know about meson. I only really see that around Wayland projects. CMake and even the aging GNU autotools seem to vastly outrank it in terms of popularity.

But I do agree on sticking to a common one. Fiddling with someone's niche build system is annoying unless it is dead simple and so Makefiles are pretty much guaranteed to work.

 

[zirias@]

I don't know about meson. I only really see that around Wayland projects. CMake and even the aging GNU autotools seem to vastly outrank it in terms of popularity.

I've recently seen meson used even by several GNU projects. But indeed, both cmake and GNU autotools are more popular.

But I do agree on sticking to a common one. Fiddling with someone's niche build system is annoying unless it is dead simple and so Makefiles are pretty much guaranteed to work.

As I mentioned earlier, I decided to use gmake for my projects (which have a suitable size for that). Of course I need some framework for that, which I wrote myself, and which is the opposite of "dead simple". Still it should integrate painlessly with any sort of package building, everything necessary is overridable as expected. But writing a full build in "make" that gets everything "correct" is indeed a complex task, and I've seen lots of attempts getting it wrong in more or less severe ways and I also have to admit that my own framework at least lacks in sensible documentation

Of course, with more and more projects using cmake (and, some using meson), my motivation to just use gmake becomes less relevant. If you need cmake/meson installed to build much software nowadays anyways, there's no point any more in avoiding build dependencies.

 

[astyle]

I was thinking, we need a Makefile overlay for BSD's, that's separate from the Makefile in ports. One where we don't fork the source, except for the Makefile only. Maybe there could be Makefile tools, that tell us what's needed, and what is called for. The Makefile adjustments could be with the same function as a patch, but part of the ports framework, where we tell it where to get the Makefile, or simply have Makefile.bsd in the ports directory, with a full replacement, or replacement per particular line.

FreeBSD actually has that, it's called /etc/make.conf... It does exactly this already. I'd suggest reading the Porter's Handbook, and see what FreeBSD already has to offer. I think you'll get better results that way (rather than making things more complicated by adding your own concoction). As an example of pretty wild success of THAT playbook, you have to look no further than Microsoft... Yeah, it's a popular bashing target, for good reasons, but if you think about it, Microsoft does make an effort to tell the devs to use the tools that come with the system. And results speak for themselves.

 

[sidetone]

FreeBSD actually has that, it's called /etc/make.conf... It does exactly this already. I'd suggest reading the Porter's Handbook, and see what FreeBSD already has to offer. I think you'll get better results that way (rather than making things more complicated by adding your own concoction).

Please stop giving knee jerk regressive responses. I've used the FreeBSD Porter's Handbook before, and I know about the Ports tree's set of Makefiles and Mk directory. I wasn't speaking about how that's an overlay.

I was speaking along a response I quoted, for use with individual Makefiles, when someone finds it simpler to rewrite it because it works better that way. An overlay on top of an individual Makefile. I wasn't speaking about how Makefile and make.conf and the ports tree already is an overlay. Furthermore, an individual Makefile overlay in the same port directory is simpler and the diff can be exported easily to any other operating system (primarily being BSD's) or build which uses bmake.

I've made an example of an overlay, how in Makefile of a port, how if it builds in CC, to write that in. There's a few complex ways, it can be build in a certain way. When an individual Makefile.bsd can be simpler, and only replace parts of that Makefile in ports. It has to be finetuned, using that in make.conf will fail to build for such that isn't fine-tuned, to keep certain build tools. If it won't build with them, it won't build, so we leave that as is, until it will build with that tool.

When speaking about a suite of applications like how Gecko is a group for Firefox and Thunderbird, then /usr/ports/Mk/Uses is where to go.

not only will it be a horrible lot of work with really complex software, you also have a very high risk to get something wrong.

I was speaking about individual ports or their Makefiles, when someone makes it work a certain way. Not blanket change of build options.


Furthermore, If someone finds an improvement for a way to build it as someone mentioned, so they can share that, while keeping the Makefile as is and leaving it simpler, without needed a change on every line for one file location or option tool. I know they can already share it, with the upstream as mentioned above, plus in the ports tree. In some cases to do that, one would have to know how to make it build with both tools, native FreeBSD build tools and GNU or other tools, to send that upstream, when my concern is BSD build utilities.

Basically, FreeBSD should be able to build everything which uses it, except for stuff made for platforms that include Windows, which cmake is for, plus GNU/GPL specific world. When something claims POSIX or near POSIX compatibility and isn't GNU/GPL code, especially, when it's a lightweight program, it should be built with FreeBSD's toolchain.

Even if I'll have gmake, cmake and a few other makes on my computer anyway. I would like for the window manager, those few tools used with it, to build with FreeBSD's native build system, maybe with a few exceptions for GPL/GNU world utilities. It may take FreeBSD upgrading to using devel/bam build system one day, which is in Lua, or something along those lines.

I believe that even if various and multiple tool utilities will be on a computer, that it should still be kept orderly, and not use another utility for the reason that, odds are that utility will be on a computer system anyway. qmake made this distinction, and qt applications are great and not intertwined with other suites, which is how it should be.


In the far past, I've had the assumption that gmake influences the license of programs it compiles, but at least in the past few years, I've not had that belief. It was aside from that, that the license of the tool affects the style and dependencies of the particular build tools/chain.

 

[yuripv79]

Please stop giving knee jerk regressive responses. I've used the FreeBSD Porter's Handbook before, and I know about the Ports tree's set of Makefiles and Mk directory. I wasn't speaking about how that's an overlay.

I was speaking along a response I quoted, for use with individual Makefiles, when someone finds it simpler to rewrite it because it works better that way. An overlay on top of an individual Makefile. I wasn't speaking about how Makefile and make.conf and the ports tree already is an overlay. Furthermore, an individual Makefile overlay in the same port directory is simpler and the diff can be exported easily to any other operating system (primarily being BSD's) or build which uses bmake.

I've made an example of an overlay, how in Makefile of a port, how if it builds in CC, to write that in. There's a few complex ways, it can be build in a certain way. When an individual Makefile.bsd can be simpler, and only replace parts of that Makefile in ports. It has to be finetuned, using that in make.conf will fail to build for such that isn't fine-tuned, to keep certain build tools. If it won't build with them, it won't build, so we leave that as is, until it will build with that tool.

When speaking about a suite of applications like how Gecko is a group for Firefox and Thunderbird, then /usr/ports/Mk/Uses is where to go.


I was speaking about individual ports or their Makefiles, when someone makes it work a certain way. Not blanket change of build options.


Furthermore, If someone finds an improvement for a way to build it as someone mentioned, so they can share that, while keeping the Makefile as is and leaving it simpler, without needed a change on every line for one file location or option tool. I know they can already share it, with the upstream as mentioned above, plus in the ports tree. In some cases to do that, one would have to know how to make it build with both tools, native FreeBSD build tools and GNU or other tools, to send that upstream, when my concern is BSD build utilities.

Basically, FreeBSD should be able to build everything which uses it, except for stuff made for platforms that include Windows, which cmake is for, plus GNU/GPL specific world. When something claims POSIX or near POSIX compatibility and isn't GNU/GPL code, especially, when it's a lightweight program, it should be built with FreeBSD's toolchain.

Even if I'll have gmake, cmake and a few other makes on my computer anyway. I would like for the window manager, those few tools used with it, to build with FreeBSD's native build system, maybe with a few exceptions for GPL/GNU world utilities. It may take FreeBSD upgrading to using devel/bam build system one day, which is in Lua, or something along those lines.

I believe that even if various and multiple tool utilities will be on a computer, that it should still be kept orderly, and not use another utility for the reason that, odds are that utility will be on a computer system anyway. qmake made this distinction, and qt applications are great and not intertwined with other suites, which is how it should be.


In the far past, I've had the assumption that gmake influences the license of programs it compiles, but at least in the past few years, I've not had that belief. It was aside from that, that the license of the tool affects the style and dependencies of the particular build tools/chain.

That's a lot of theory (which was already replied to as being a giant waste of time), could you provide some real examples of what you have already done to make this reality (just in case we don't understand what you are suggesting)?

 

[sidetone]

I ported, with some help, and maintain x11/jbxvt, x11-wm/fswm and x11-wm/ittywm.

fswm has CC=gcc in the source makefile, and I overrode that in its Ports Makefile with:

MAKE_ARGS=     CC="cc"

Also, where argument flags known for use with GCC which were called for in source makefile, would be converted to the proper Clang equivalent in the Ports Makefile, and that would build with bmake and llvm.

Before, I would find a port which would compile simply by removing gmake, and telling the maintainer that it would build that way with the make that comes with FreeBSD.

The biggest issue would be which has to do with shell scripts with how the source Makefile is written. shebangfix solves some issues when bash is called for.

I believe there's more people who know more about shell scripting than do with C programming. I could learn more about shell scripting. A lot of it is the Makefile.


I also learn about FreeBSD, and post How-to's and Faqs.

 

[astyle]

Seems like the chicken-and-egg question in porting is: How safe is it to make assumptions about other systems?

The way the basic Linux toolchain works is not exactly compatible with FreeBSD's basic toolchain. As sidetone pointed out, there's plenty of situations when a given project has to write its own dev toolchain, incompatible with everything else, just to bridge that gap. An example of that is devel/ncurses... it has to take into account various terminal types, shells, and other details. Ncurses does a pretty decent job of shielding the dev from extra OS-specific complexity. If you write something with Ncurses, it would be relatively simple to port. Writing the same software without Ncurses would make it a huge headache to port to another system. And, Ncurses is not included in the base of either FreeBSD or Linux.

 

[bakul]

ncurses is included in FreeBSD (which uses termcap). The devel/ncurses uses terminfo (came from AT&T's Sys V world). Traditionally BSD used termcap and SYS V (and I think SYS III before then) used terminfo. I think Linus modeled Linux more on SYS V so for linux programs you may need terminfo based ncurses (just a guess). Most of this is legacy and doesn't matter much these days since pretty much everyone uses extended VT52 terminal capabilities (VT100, xterm etc are all in the same family).

 

[zirias@]

What bakul said. Plus, ncurses is "not included in the base [...] of Linux" just because there is no such thing, still I'm pretty sure almost(?) every Linux distribution will have it in it's "base". I'm currently experimenting with trying to build a userland for the Linuxulator from source, and one of the first things I needed to port (after the toolchain) was ncurses. It's needed for a full-featured readline, which is needed for a full-featured bash, which is the default shell in any GNU userland.

 

[kpedersen]

these days since pretty much everyone uses extended VT52 terminal capabilities (VT100, xterm etc are all in the same family).

A slight off-topic hint. If you set TERM=ansi, you can disable all the horrific default colors that most GNU/Linux servers randomly use. Especially the highlighting that vim, cmake uses. Great if you want a calmer experience.