ELI5: Guys, I have a few questions about programming and FreeBSD

No need to escalate this further. I don't know when exactly this started, but nowadays, many people "feel" disagreement as a personal insult, if not even "violence".
I don't think You should bother about that. I think this started with the watering-down of (higher) education. People nowadays think they have an indefeasible right on having some opinion, however unfunded it might be.
In the old times any higher education would consist of speaking the common language of the western civilisation (i.e. Latin), and to read the philosophers. And there one learns that one can have a lot of opinions, but in order to articulate them, they need to be properly derived.

In computing, a bit of this has been preserved - because, when you write some code in one way or the other, at least it should make some sense and have some rationale. In other realms this has apparently gone completely astray by now.
 
There is some nice shell code to look at in /usr/share/bsdconfig and /usr/local/share/poudriere.

I've not looked at poudriere, but /usr/share/bsdconfig is perhaps the only substantial suite of programs in the base system written 100% in sh(1), using only base system utilities.

I thought I had a pretty fair handle on sh before I dived too deeply into bsdconfig :)

edit: To clarify, the program is /usr/sbin/bsdconfig and while most of the code (as *.subr) lives in /usr/share/bsdconfig/, there are also includes in /usr/libexec/bsdconfig/
 
I've not looked at poudriere, but /usr/share/bsdconfig is perhaps the only substantial suite of programs in the base system written 100% in sh(1), using only base system utilities.

I thought I had a pretty fair handle on sh before I dived too deeply into bsdconfig :)
Code:
f_isset()
{
        eval [ \"\${${1%%[$IFS]*}+set}\" ]
}

wat...

There is a lot of extra in there that I wouldn't have thought of.
 
1- What would be the best way to learn posix with focus on BSD's, shell and Perl?
FreeBSD documentation is a a good starter of quality.
AbsoluteBSD is a good book too.
2- Is there another language for sysadmins besides shell and Perl?
marginally, awk and sed.

Shell-scripts are 70 % of character string shelling and awkwardly do it with regular expressions because it's stricky.
Prolog programming language and its object-oriented flavor, logtalk are candidates to replace all this stuff because of its convenient ability to deal with grammars via DCGs.

3- What is the best way to navigate the FreeBSD source code?
Certainly a crossref html compilation : http://fxr.watson.org/

In practice, find . -name "*.c" -o -name "*.h" -exec grep -e ... {} /dev/null \;
4- What are the languages most used in FreeBSD development?
C
PS.: I'm not a developer or a programmer, just a script kiddie trying to discover new toys ;-)
 
1. I'd think that a good approach to that would be to learn about BSD's, with a focus on POSIX compliance. Yeah, a bit different with how it was phrased in the OP... What I mean by that is, learn about a BSD feature, and ask if the implementation is POSIX-compliant. As in, figure out which parts of the POSIX standards apply to that BSD feature, and how they apply.
2. Straight C comes to my mind. Also, Python, sed, and awk.
3. Start in /usr/src. If you installed kernel sources (source.txz in the installer), that's where the FreeBSD source code lives. Well, it will be the code of whatever you installed. If you read the Handbook, it will provde more detail.
4. Offhand, I'd say C - most of the files I looked at in FreeBSD's cgit kernel repo are definitely in C.

Try compiling the kernel and to time that, maybe?
1- That actually makes sense after reading ralphbsz and zirias@ comments about the importance of sh in the FBSD base.
2- C really? I was thinking about learning a bit o C 4fun, but I can't see how would be better to a sysadmin in opposition to interpreted languages.
3- Fair enough.
4- 🤔

Compiling FreeBSD it is in my list of stuff to do after I finish to plan my backup setup.

I know that "books" (paper with printing on it, bound on one edge) is getting old fashioned. But about 20-25 years ago, O'Reilly had a really good set of manuals about Posix standards. I had the ones for the C library, real-time (threads + async IO Posix.4, book is by Bill Gallmeister) library at home, and used them as reference manuals when programming. If you only read documentation that's tuned to Posix instead of BSD- and Linux-specific man pages, you will likely end up with Posix-compliant code.

There are also Posix standards for the shell, including standard utilities such as sed and grep (don't remember about awk).


Actually, the primary language is shell, because it is always there (on any machine, even in single user mode). It is also what a lot of the system's infrastructure is coded in. A good programmer and good sys admin has to be quite conversant in shell (and preferably not tcsh or bash, but real sh).

I would describe Perl more as a rapid development, interpreted programming language, not a scripting language. Today, a lot of that function is also taken by Python and Ruby.

Most sys admins do not program in C.


The kernel source code? Start by reading the "black daemon book": Design and Implementation of the FreeBSD, by Kirk McKusick et al. Only after reading the book, and understanding the structure, does it make sense to go into the source.

But the code itself is not very enlightening. You don't see the design principles and overall architecture by looking at lines of code, at least not right away.


For the kernel itself, most utilities (sed/grep/...), and a significant fraction of packages: C. But as said above, also significant amount of sh.
Wow, thank you for that. Bookmarked this massage.

"Learn POSIX" is a bit difficult to define, because POSIX is a standard for operating systems, including all sorts of things like standard APIs for e.g. files, sockets, IPC, threads, a standard set of tools with defined options and behavior, a standard for the shell, etc. Note that perl is not part of POSIX (read: you can't expect perl to be available on any POSIX-compliant system), but offers a module to access POSIX APIs.

So, given that, I'd say starting to better learn the shell (standard POSIX/bourne shell, NOT extensions like e.g. bash and certainly not completely different shells like e.g. csh) is probably a good way to "learn POSIX". If you want to go deeper, write C programs using the POSIX APIs (they're defined in terms of C).
Use whatever does the job. A sysadmin normally writes scripts specific to his scenario, so there's no need for portability, it only needs to work on that one machine. Of course, if you want portability to other POSIX systems, the shell (/bin/sh) is your primary choice. You can't use perl (as explained above), you may use awk which is also specified by POSIX.
I want to focus on posix programing because I use 3 different OS's at the same time (FBSD, OBSD and Slackware). I agree with you, /bin/sh it is probably the best option for that.

Clearly C. Both the kernel and FreeBSD's own userland tools are written in C. If you add third-party software that's integrated in base (e.g. llvm), you will also find C++.

Second place (with quite some distance) is most likely shell. E.g. all the init-scripts are written in shell.
Ok, shell is the king for sysadmins. Nice, I actuality like shell. I wanted to find a new one (new for me) because sometimes one may need a "more powerful" tool, I usualy use Perl for that, but the majority of good Perl modules are mostly abandoned and very outdated. So I'm looking for a new toy for the long run.

As for the purely technical aspects of looking at C code, you can run exctags at the root of the local source tree:
exctags -R --langmap=c:.c.h .
, then run gvim and navigate through the functions with Ctrl-] , Ctrl-T
I found etags, but not exctags on my system and in ports.

There is some nice shell code to look at in /usr/share/bsdconfig and /usr/local/share/poudriere.

I use 'findgrep' to search for stuff in files. It's a wrapper to the command find $dir -type f -name $name -exec grep $opt '$pattern' {} \;
I've not looked at poudriere, but /usr/share/bsdconfig is perhaps the only substantial suite of programs in the base system written 100% in sh(1), using only base system utilities.

I thought I had a pretty fair handle on sh before I dived too deeply into bsdconfig :)
That's is very good to know, I like to use my OS's to learn other stuffs too.
Similar to this are the OBSD rc scripts, looked like black magic to me for a long time.

Ruby. Ruby is the AFAIK only script language that is truly OO.
Perl, python etc had OO only grafted upon at a later time - and that's how it feels when using it. I once had to write Perl OO, and it was ugly. With Ruby I learned how OO is supposed to work, and why (and when) it makes sense.

In some way Ruby is similar to FreeBSD: there is beauty in the philosophy right from the beginning. Usually when scripting something, you do the 20% to put the idea into code, and then you do (or sometimes do not) the other 80% to handle the corner cases. And in that phase the code tends to grow significantly. With Ruby the code often rather shrinks at that point, because the language itself promotes elegance.

Ruby is japanese, and there is some similarity to japanese lifestyle, where many things are done as an act of meditation, with much care for the details. This is not so much a language for fast time-to-market production and "just make things work". And as You seem to be not foreign to the arts of the mind, You might give it a try.
Cool, I will take a look at Ruby, thanks.

You just shouldn't consider it the supreme wisdom
I don't, I just want a common ground between my setups (FBSD, OBSD and Slackware).

Large-scale operations these days are mostly about Terrible (Terraform + Ansible), if not Kybernetes. A useful language for that is Golang. Python's limitations really opened a gap for Go.
For devops yes. But I like to learn "old-school system administration" for my homelab setups.

All I have left to say is: If you're violating a standard you claim to respect, you're rightfully blamed for the breakage you cause:
View: https://twitter.com/8bitsound/status/1644397186862522372

(I'm definitely not the only one "blaming" here of course)
"This Felix dude kicks ass!"
- Beavis

And there one learns that one can have a lot of opinions, but in order to articulate them, they need to be properly derived.
"Opinion it is the wilderness between ignorance and knowledge."

Code:
f_isset()
{
        eval [ \"\${${1%%[$IFS]*}+set}\" ]
}

wat...

There is a lot of extra in there that I wouldn't have thought of.
I cannot even read this, nice.

FreeBSD documentation is a a good starter of quality.
AbsoluteBSD is a good book too.

marginally, awk and sed.

Shell-scripts are 70 % of character string shelling and awkwardly do it with regular expressions because it's stricky.
Prolog programming language and its object-oriented flavor, logtalk are candidates to replace all this stuff because of its convenient ability to deal with grammars via DCGs.


Certainly a crossref html compilation : http://fxr.watson.org/

In practice, find . -name "*.c" -o -name "*.h" -exec grep -e ... {} /dev/null \;

C
1- Yup, that's one of the reasons I'm here ;-).
I have the last edition of Absolute FreeBSD, great book, love reading it.
2- Yup, awk and sed are awesome tools. Wait prolog and logtalk are languages? I didn't know them.
3- That is just beautiful, tnx.


Thank you everybody, I already got the knowledge that I was looking for.
 
Code:
f_isset()
{
        eval [ \"\${${1%%[$IFS]*}+set}\" ]
}

wat...

There is a lot of extra in there that I wouldn't have thought of.

Yes, exactly. Some of it seems overwrought, almost showing off, but apart from a couple of minor but hard to find bugs (eventually due to changes in pkg over years), it actually works for the part I worked on, being 'bsdconfig packages'.
 
Lisp is not meant to be used at runtime.
Mmmph. The original Yahoo Store ran on Lisp. There's another well known few sites or software I can't recall right now that run on Lisp.

I've never been able to get into it but I'm not so sure about your statement.

EDIT: Grammarly and Hacker News for two
 
Java programmers around the globe are in a panic today over a Wikileaks press release announc(ing) that they will re-release the source code for thousands of Open Source Java projects, making all access modifiers 'public' and all classes and members non-'final'.

I think Jose was just messing around linking to a blog like that. It reads like a humor column, poking fun at how people get worked up over access to code, release/re-release procedures, and language design. It does take knowing quite a bit about the topics covered to be able to see the humor. 🤣
 
Mmmph. The original Yahoo Store ran on Lisp. There's another well known few sites or software I can't recall right now that run on Lisp.

I've never been able to get into it but I'm not so sure about your statement.

EDIT: Grammarly and Hacker News for two

I meant the real point about Lisp is compile time computing. You have the full extend of the language both at runtime and at compile time. Not some crippled sublanguage like templates or the C preprocessor.

The compile-time computing ensures of course that Lisp is the only language that long-term-use programs should be written in. Because it is the only way to keep single assumptions in single places in the source code.
 
This statistic refers to the FreeBSD base (kernel and userland),
Yep, and they don't distinguish contributed/3rd-party stuff from FreeBSD's own code. I'm pretty sure C++ is only used in 3rd-party...

Also, I wouldn't call Roff and Makefile "programming languages". Well, Makefile might be a corner case, but Roff certainly just describes documents. So, what's left is indeed C and Shell ... ok, plus some Assembly, can't have an OS without that I guess ;)
 
C 62.2%
C++ 24.5%
Roff 4.9%
Shell 2.8%
Assembly 1.8%
Makefile 1.3%
Other 2.5%

source: https://github.com/freebsd/freebsd-src
This statistic refers to the FreeBSD base (kernel and userland), without the ports. C++ is used in userland, not the kernel btw.

C++ is mainly the LLVM compiler, which has a full import in the FreeBSD base tree. So it is a bit misleading. You rarely mess with the compiler when hacking FreeBSD. And it carries around support for all kinds of architectures and OSes that are inactive in the FreeBSD tree.
 
There are many programming languages. But in the end you only use a few of them.
In my case : racket-scheme,chicken-scheme,sbcl-lisp,ocaml,f#.
None of which is used by FreeBSD.
For FreeBSD it helps to have a good understanding of "make" and "sh" and "C".
 
From what I've read so far, I think you're not directly interested in starting to make a career out of programming and your choice isn't influenced/mandated by a particular work environment. From a sysadmin POV sh(1) and some accompanying sed(1) and awk(1) is always useful to get some basic feel for and skills in.

Another helpful way of looking at your next (first?) programming endeavour could be to look at a language that you'd be comfortable with and enjoy developing code with; that is kind of a chicken-and-an-egg problem I admit. Also, perhaps you've already got some ideas what (problems) you'd like to program. For example, if your deepest wish is to get into systems programming (even kernel programming) then, within the FreeBSD context, C would be almost mandatory.

If you've already got some kind of problem/solution in mind you'd be interested in programming then you could use that as a guide to find a suitable (first) programming language for. I'd focus on a programming language/environment in combination with no or a very light IDE; after some practice, you'd have a much bigger chance of having fun in using it.

Have a look at the (main) website (or join a forum) of a particular programming language and find out what the helpful books (websites?) are for beginners to learn that particular programming language. Apart from that, try to find what the helpful books are for beginning programmers (books that happen to be in that particular language); note the difference between this sentence and the previous one.

Given the possibility to apply to a very broad type of programming problems from sysadmin stuff to full-fledged applications, I'd have a look at Python.

___
* Perhaps after a reasonable amount of time and experience with your first programming language you could broaden your horizon with something like Seven Languages in Seven Weeks if you feel like it.
 
Back
Top