C++ how to setup a proper C++/Python build environment

[kirsche]

Hi, I am new to FreeBSD (my daily driver now) and I need some help.

I did some development in the past on Windows and embedded Linux, but never on a BSD. On FreeBSD 15.0 I was able to setup my RTL8125BG NIC, KDE Plasma 6, my keyboard to neo(bone) and I was able to run Joplin AppImage using Linuxulator (rl9).

I already installed Python 3.12 in my current user account and I am able to set up a venv.

I know that cc and some other development tools are already installed in base system. But env does not show me a CC or a CXX. I had a look to Resources for newbies but did not see much help for this topic.

So I need some ideas / help to get:

1. a proper C++/Python build environment to be able to build Betterbird (Thunderbird-ESR) from sources
2. without cluttering my complete system or my user account

Kind regards
kirsche

 

[unitrunker]

What does "c++ -v" say at a shell prompt?

 

[unitrunker]

You could look at the existing port of thunderbird and use that as a template to create a port of betterbird.

The port files are located at /usr/ports/mail/thunderbird

 

[Maturin]

On unix[like] cc ain't no compiler but it's an alias for the system's C-compiler, which on FreeBSD by default is clang (On Linux gcc, which you have to install extra on FreeBSD if you want to work with that one [Better don't change cc to point at gcc.])
Similar to gcc clang can compile both C and C++. unitrunker pointed out c++, which is also an alias to clang.

a proper C++/Python build environment

What do you mean by that?
By default you have the C/C++-compiler which also comes with the debugger lldb (gdb needs to be installed extra, but for clang lldb was first choice [your root needs to give your user(s) the proper rights, so he/they can use the debugger.]) Python you have installed. You have a shell (sh by default, bash or others need to be installed extra), and an editor (ee and vi by default, others need to be installed extra, which you already have done when you installed KDE: vim, neovim, emacs, geany and I don't know what others. I just need and use one. But I am not a KDE user .)
make is there by default, like many other very powerful tools.
To me what only lacks is a version control to get a "proper environment".
Install, what you like: git, svn, fossil,...
Apart from vim is capable of view and edit hex-files (I'm sure neovim, emacs and others are capable, too.) you may install a hexviewer. For most jobs I prefer hexyl - short, simple, does 99% of the jobs I need a hex viewer for. If you want something more heavy-weight, complex and with a GUI you may als check out imhex.
That's what can be called a PDE - a personal development environment.
Some prefer IDEs, others (like me) PDEs.
You may check it out before looking for and learning a new IDE. (Dig this forums about IDEs.)
If that's what you meant.
P.S.: vim (and neovim) offers a lot of stuff to pimp your editor, and emacs can be bloated beyond overkill.

without cluttering my complete system or my user account

Of course not. Who wants that?
But it would be helpful for helping you if you help us to understand what you mean with that.
You may use backups to restore a messed up /home/. Which I recommend anyway to primarily think of and realize a suitable backup plan before you start any production.
Also snapshots could be very useful. They reset filesystems to a former state.
Maybe this is just answered with the information: unlike Windows you cannot mess up your system over time by just install/deinstall many packages. Just install what you need, and deinstall what you don't.

Think of a way how you structure, and name your /home/ and its subdirectories.
Keep your home-directory in order is best way to not drown someday in a self produced heap of garbage.
See for example How do you name directories? for to get some ideas how others do it, and find your own style what suits you best.
(P.S.: version control also can help on not getting drowned in garbage one day.)
P.P.S.: backup plan!

 

[kirsche]

Hi unitrunker and Maturin, thank you for your answers.

Here is the output:
:~ $ c++ -v
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
Target: x86_64-unknown-freebsd15.0
Thread model: posix
InstalledDir: /usr/bin


Thunderbird-ESR from the pkg completely destroyed my TB profile exported from Windows. It also downloaded and deleted all emails from my IMAP folders. I've had similar problems in the past when migrating from one system to another. That's why I don't want to use this software anymore. As far as I know, Betterbird is a little more save because it's a soft fork with a higher patch level. In my experience, Betterbird runs more stable. So I want to achive "works on my system"-status before I try to or help to add/improve/change a port.

But that's only one reason for having a proper build environment. There's so much I want to do on my FreeBSD machine, but I want to do it right the first time. And to do that, I want to avoid cluttering my storage or breaking my system through misconfiguration. During my journey I will have to use ./configure to set DESTDIR/PREFIX to resonable values. There are a lot of possibilities like chroot which is mostly used as Linux development environment. That's why I'm asking about best practices. But what is the FreeBSD way to get a proper development environment?

 

[JohnK]

If FreeBSD is all set up; start coding. You did it. You're done.
mkdir ~/src/myapp
vim ~/src/myapp/main.cpp
...

If I were you I would read at least the first few chapters of this:

FreeBSD Developers' Handbook

For people who want to develop software for FreeBSD (and not just people who are developing FreeBSD itself)

docs.freebsd.org

 

[Maturin]

Thunderbird-ESR from the pkg completely destroyed my TB profile exported from Windows.

You do it vice versa: first install TB, then kill the default profile and copy your profile into the TB's directory. At least this way it worked for me. But I did it the last times some years ago, and don't know if with today's TB it still can be done. (I am however turn my back on TB anyway, and currently want to switch to mutt [if I do not spend my time in the forums... )

But of course you have a backup of your TB directory, as of your whole Windows drive before you started such a stunt. So, no data lost, because you still have the backup, right?
If not... Dude, first things first: backup plan, and backups before you think of anything else!

But what is the FreeBSD way to get a proper development environment?

There is no such thing as "THE (right) way."
There is only your way.
You may install something large comprehensive like e.g. KDE and IDEs, or do it the unix-philosophy's way by having smaller tools for each single job. Or find your own personal compromise.

I've seen you started to dig this forums (unless it's turned off in the personal profile's options, hovering the mouse over some one's avater will show what thread she/he is looking at. ) That's good! This forum is a gold mine you will learn to value. Tip: use duckduckgo.com instead of the forum's internal search. You'll receive better findings.

You need to start somewhere. You already did. I personally don't like KDE (too large, too heavy, too bloated, too Windows like - but that's my very own very personal taste. Others will agree with me. Others will disagree, and tell you KDE was a good choice. Doesn't matter. Anyway it's a start. You can stay with it, but you don't have to.
Same is with development environment.
What I described above is a full, and proper development environment. And not a minority does it exactly this way.
Unix comes per se as an development environment by it's very nature. You "only" add more tools.
If this does not suits you - or maybe not yet, because you're (still) used to other development environments (IDEs), well okay, you may browse for which IDEs, or other solutions there are, and pick one of those. You will also find users doing it this way either.

Anyway, be prepared to adapt your system to your needs. Both will change over time simply because of your experience. That's exactly what FreeBSD is meant for: Tailor your own personal system, and adapt it to changed needs.
There is not THE way.

 

[kirsche]

Okay, after thinking about your answers for a few days, I would like to clarify my requirements and hope for your swarm knowledge. So let's get to my actual question.

Let's assume I have set up a working development environment on my BSD machine. I have compiled my stuff. To test it, I now need to copy my compiled artifacts to the appropriate directories. This may overwrite important libraries of already installed packages/ports, which I don't want.

How can I prevent my operating system (and secondarily my user data) from being damaged? What is the easiest way to achieve this goal (static linking, chroot/jail, Bhyve, Port, ZFS snapshots, etc.)?

 

[JohnK]

Ah. So, you probably want a jail (easy to (re)setup and copy to/from for all kinds of testing).

For example (from host):
# doas cp ./path/mything /usr/local/jails/containers/mytest/home/me/bin/
(or you can ssh into, or, etc, etc. pick a method to get stuff on/in jail; easy stuff).

The problem is that you may mess up your jail a few 100 times and you don't want to spend time fixing a "temporary testing jail". So the workflow I like to use is to create a jail, add/do stuff, when it gets broken, delete it and remake it. There are a ton of jail managers but I made myself one that allows me to keep my jails in a recipe of sorts that I can just delete and/or create with one command (each). So, I keep all the packages, configs, etc. in my 'recipe' and just remake the jail when I need to.

I set up some very complicated examples in my examples directory but my real scripts and config files are one or two lines long (add a few packages, and set the name and ip).

GitHub - JohnKaul/jcreate: Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities

Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities - JohnKaul/jcreate

github.com

My jail "manager" allows me to destroy and recreate jails in a more automated method (e.g., from a makefile) so, if I were to "make" my project I can build a binary and deploy it to my test jail with one command.

 

[ralphbsz]

Okay, after thinking about your answers for a few days, I would like to clarify my requirements and hope for your swarm knowledge. So let's get to my actual question.

Let's assume I have set up a working development environment on my BSD machine. I have compiled my stuff. To test it, I now need to copy my compiled artifacts to the appropriate directories. This may overwrite important libraries of already installed packages/ports, which I don't want.

How can I prevent my operating system (and secondarily my user data) from being damaged? What is the easiest way to achieve this goal (static linking, chroot/jail, Bhyve, Port, ZFS snapshots, etc.)?

So you have created an artifact named foo. Now you want to install it, for example into /usr/local/bin/foo, or /usr/local/libfoo.a, or /usr/local/lib/python3.11/site-packages/foo.py (or ...foo/a.py and so on). You are worried that any one of these files already exists, and is part of another package.

Simple answer: In your install script, or in the "make install" part of the make file, if the target file already exists, and is not clearly part of (an older version) of your package, print a warning, and either quit, or get user input.

Harder answer: package your code into kit, to be installed by the standard package management tool (such as pkg for FreeBSD as a whole, or pip for python, or whatever the CPAN format is for Perl). Then use standard install tools, so your thing is installed with "pkg install foo". Warning: on many OSes (I don't remember whether current FreeBSD is among them), pip is locked out, and you can only use the OS's package installed (yum, apt, pkg ...) to install python packages system wide, except in virtual environments. Another case of good intentions gone sideways.

 

[skhal]

Ah. So, you probably want a jail (easy to (re)setup and copy to/from for all kinds of testing).

For example (from host):
# doas cp ./path/mything /usr/local/jails/containers/mytest/home/me/bin/
(or you can ssh into, or, etc, etc. pick a method to get stuff on/in jail; easy stuff).

The problem is that you may mess up your jail a few 100 times and you don't want to spend time fixing a "temporary testing jail". So the workflow I like to use is to create a jail, add/do stuff, when it gets broken, delete it and remake it. There are a ton of jail managers but I made myself one that allows me to keep my jails in a recipe of sorts that I can just delete and/or create with one command (each). So, I keep all the packages, configs, etc. in my 'recipe' and just remake the jail when I need to.

I set up some very complicated examples in my examples directory but my real scripts and config files are one or two lines long (add a few packages, and set the name and ip).

GitHub - JohnKaul/jcreate: Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities

Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities - JohnKaul/jcreate

github.com

My jail "manager" allows me to destroy and recreate jails in a more automated method (e.g., from a makefile) so, if I were to "make" my project I can build a binary and deploy it to my test jail with one command.

Per-jail ZFS dataset may speed up the process: (A) snapshot the jail when it is setup and ready for use (typically network setup, install packages, SSH, PAM, etc.), and (B) rollback to the snapshot when things go off.

 

[unitrunker]

How can I prevent my operating system (and secondarily my user data) from being damaged? What is the easiest way to achieve this goal (static linking, chroot/jail, Bhyve, Port, ZFS snapshots, etc.)?

There are three tiers for executables and libraries.

/bin + /lib
/usr/bin + /usr/lib
/usr/local/bin + /usr/local/lib

Components of the base operating system are found in the first two.

Ports/packages are normally installed to /usr/local/bin and /usr/local/lib. There's also a /usr/local/etc for system-wide configuration.

hier(7) more details.

The point is ... avoid touching /bin+/lib and /usr/bin+/usr/lib. Worst case is you boot single user mode and use the statically linked commands under /rescue to put your system back together.

ZFS and jails make for a very powerful combination.

 

[JohnK]

Per-jail ZFS dataset may speed up the process:

Sorry, a bit like compairing apples and zebras. My tool at heart wants to be a plugable automation type tool (call it from near/far/within/etc).

With my script, I can launch it on an already existing container (if "the rquirements" change) or no container at all (where it extracts/creates one). So, if while writing my widget(s) I decide I want to switch to XYZ instead of ABC I update my configuration and relaunch/recreate depending when I need to. Instead of me keeping an actual notebook with a list of packages to install for "widgetone" I write down what I need in a directory that gets read like a recipe for creating/maintaning jails.

I even gave a dumb/entry level script writer (so even that aspect can be somewhat automated too) but it's very, very crude.

GitHub - JohnKaul/jcreate: Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities

Script to create a FreeBSD jail.conf style jail with setup script, mounts, copy-in abilities - JohnKaul/jcreate

github.com

Snaps are great but I'm not destroying/making jails/containers every time *I* build (you may, I don't know) so I'm not sure what speed up there would be for me--I suppose the build process could restore the snap, but if that's not needed then why bother? I mean, of course everyone keeps snaps, but you still need something to snapshot; my thing is about saving/reusing/automation (e.g. if you and I start collaborating on "ACME-Widget2.5" are you going to send me a snapshot so I can build/test [I could send you an actual jail]?)

 

[kirsche]

Thank you all for your answers.

ralphbsz, I am able to adjust my own "make install". But I don't see myself being able to master third-party applications (for example a Thunderbird).

JohnK, I like your approach, but I don't quiete understand it. I see the advantage to be able to fast set up a jail. But I don't see, how your scripts will prevent me from overwriting system critical data.

Summary: First, use a jail to achieve “works on my system.” Once the software is stable, generate a port. ⇒ Okay. Now let's get into the details for step one.

As far as I understand, I can use nullfs to map a path to anywhere (thin jail). If I mount a path with "ro", I can read all files from inside of the jail but I cannot write anything. If I mount with "rw", every write goes directly to the underlaying directory of the mount - which is not what I want.

I like the idea of using nullfs for mapping my system directories and adding/stacking my compiled stuff with unionfs without overwriting system files. I tested unionfs by stacking 2 directories (I think I will need to stack at least 2 layers to be able to test a software). It seems to work as expected. Unfortunately mount_unionfs() says:

THIS FILE SYSTEM TYPE IS NOT YET FULLY SUPPORTED (READ: IT DOESN'T WORK) AND USING IT MAY, IN FACT, DESTROY DATA ON YOUR SYSTEM. USE AT YOUR OWN RISK.

Is there a functional and safe alternative?

 

[JohnK]

I'm not an expert (aka "I don't use") thin jails so please don't take what I say on those as any sort of advice. The whole mountings aside (and assuming you go the thin jail route), I'd think you would want to do what skhal said and keep a snapshot you can roll back with (at least that's how I read the handbook too) but I'm not sure how you'd deal with the "ro" mounts either.

However, I use (proposed) thick jails. Thick jails are essentially just mini systems so you can screw them up as much as you want. My scripts allow me to keep my setups written down in directories so I can recreate (thick) jails in minutes (i.e. each jail, I keep a list of packages, name, ip, and a setup script. which means I keep my jail notes/setups in directories, like I keep my software projects). I can describe how you could use my scripts to build thick jails, but I don't want to steer you down the wrong path (I'm not sure how/what you're developing). You could try a think jail: it would be easy/quick to setup networking, ssh and play around a bit while you decide/learn/read. ...? I guess, I'd build a (thick) jail with all the packages you'd need and send the code there to test it.

​

 

[kirsche]

Thank you all for your insightful response. I will start a new thread with more specific questions.