Tutorials for FreeBSD system programming?

[helmet1080]

If I read The Linux Programming Interface, is it OK? I has a couple of chapters on System V... The man page is too hard to learn how to program in this environment, and I don't know how to use it.
man 3 strlen ? I need a guide to follow, to learn everything no idea what exist and how to search it. I've never programmed anything, nothing like system programming before, and in reallity I don't have much time programming. I know The C Programming Language, and I want to learn kernel programming. A friendly and complete guide on how to use system calls, and how the system works would be ideal. Thanks.

 

[SirDice]

You never programmed anything yet you want to learn something as complex as a kernel? Maybe, just maybe, you should start at the beginning? If you don't know the basics something as complex as a kernel will be totally over your head.

A friendly and complete guide on how to use system calls, and how the system works would be ideal.

This has nothing to do with kernel programming. The example you gave regarding strlen(3) has nothing to do with the kernel either. Those are standard C library functions.

C standard library - Wikipedia

en.wikipedia.org

 

[Minbari]

The GNU C Reference Manual
The GNU C Programming Tutorial

 

[tommiie]

Hey helmet1080: I agree with SirDice. Can you explain the project you want to start? There are big differences between programming an application or a kernel. When programming an application, there are again quite some differences: will it be a CLI program like cat(1) or grep(1) (i.e. with a real UI) or will it be a program needing CLI windows (i.e. require ncurses(3)? Will it be a GUI application running on KDE, gnome...?

The more we know about your goal (project) the better we can assist in providing a tutorial, book, or other starting point.

 

[ProphetOfDoom]

Agreed. I wouldnt attempt kernel programming and I've been programming since I was six.
Good first projects are like... a command line program where the computer selects a number and the user has to guess it... Or maybe a program to print out the multiplication tables from 2 to 12.
Dull maybe but necessary.

 

[helmet1080]

You never programmed anything yet you want to learn something as complex as a kernel? Maybe, just maybe, you should start at the beginning? If you don't know the basics something as complex as a kernel will be totally over your head.


This has nothing to do with kernel programming. The example you gave regarding strlen(3) has nothing to do with the kernel either. Those are standard C library functions.

C standard library - Wikipedia

en.wikipedia.org

I mean I've only copied the snippets that appear on books for C programming. I've programmed a couple of data-structures like linked list, stack, queue, binary tree. I was reading Advanced Programming in the UNIX Environment, 3rd Edition, and I'm on chapter 4, and I understand very well what it says. But don't know if I should read/take a course in operating systems like Operating System Concepts 10th Edition by Abraham Silberschatz then any course on operating systems. Maybe I should follow the guide

C standard library - Wikipedia

en.wikipedia.org

and then ask again. Thank you for the reply.

The GNU C Reference Manual
The GNU C Programming Tutorial

The GNU reference manual. I've read that FreeBSD doesn't use the GNUlibrary, uses libc that is a different library. Thank you for the reply.

Hey helmet1080: I agree with SirDice. Can you explain the project you want to start? There are big differences between programming an application or a kernel. When programming an application, there are again quite some differences: will it be a CLI program like cat(1) or grep(1) (i.e. with a real UI) or will it be a program needing CLI windows (i.e. require ncurses(3)? Will it be a GUI application running on KDE, gnome...?

The more we know about your goal (project) the better we can assist in providing a tutorial, book, or other starting point.

Not sure about what project goal to have in mind. But my goal could be to stay envolved, to particapate, to learn anything related to the inner working of the oparing system, and the kernel seems very good place to stay. I'd like to program drivers, I like the kernel, I like the desktop manager KDE Plasma, I like Unix. It's deficult to know where to start if you don't what exists, but for now kernel.
Thank you for the reply.

Agreed. I wouldnt attempt kernel programming and I've been programming since I was six.
Good first projects are like... a command line program where the computer selects a number and the user has to guess it... Or maybe a program to print out the multiplication tables from 2 to 12.
Dull maybe but necessary.

I've been reading about python the basics as well, but I like, I prefer compiled programming languages like C. Thank you for the reply.

 

[helmet1080]

"Advanced Programming in the UNIX Environment" and "UNIX Network Programming" by the late W. Richard Stevens which you're already reading.

If you want a really good book on operating systems, there is Operating Systems: Three Easy Pieces which is free and used at the University of Wisconsin. It explains the concepts and has code snippets, but not full code of any operating system.

"Operating System Design: The XINU Approach, Second Edition" by Douglas Comer, used at Purdue University is excellent. The code can run on either a Beaglebone Black, Intel Galileo, or instructions on the website show how to get XINUpi running on the Raspberry Pi. The entire XINU OS is only 8,000 SLOC and no line is over 80 characters wide.

"Operating Systems Design and Implementation, 3rd Edition" by Andrew Tanenbaum is on the design of Minix3 and also includes all of the source code in the book.

Thank you for the reply. What order should read them all in? Yes, they're all interesting to me. The book: Operating System Design: The XINU Approach. is the one that teaches you how an OS works, and how to build an OS from the ground up, which makes it the coolest book. What are the prerequisites to read this book? I'm interesting in Minix as well, and I wanna learn it. I read it's a Unix-like and it's a POSIX compliant. I have all day to have a tones of fun, so I wanna learn everything I need. Thank you again.

 

[helmet1080]

I have worked as a UNIX system administrator for 25 years, with AIX, Solaris, and Linux (I do not like it). I've worked for one of the world's largest online retailers, the largest online stock trader, US DoD, among other companies, and am presently working for the world's largest transportation company.

But I don't consider myself an expert. To me, people like Adrian Cockroft, currently the Cloud architect at Amazon AWS and authored "Sun Performance and Tuning: SPARC & Solaris." Richard McDougall co-authored "Solaris Internals" while at Sun Microsystems and was CTO at VMWare and is currently in Cloud & Infrastructure Engineering at Google. Or "Solaris Internals" other co-author, Jim Mauro who is currently a performance engineer at NVIDIA. Another is Bryan Cantrill who was a former Sun Microsystems kernel developer and creator of DTrace, and is now the CTO at Joyent; who also contributes code for SmartOS periodically. Those are experts.

The last person I will mention whom I consider an expert is Brendan Gregg. He previously was a kernel developer at Sun Microsystems and was the primary author of "DTrace: Dynamic Tracing in Oracle Solaris, Mac OS X, and FreeBSD" and is the author of the book "Systems Performance: Enterprise and the Cloud." He also developed the ZFS L2ARC at Sun. After Sun he was the Lead Performance Engineer at Joyent. Presently he is Senior Performance Architect at Netflix.

Brendan Gregg has said he wanted to be an expert in performance so he picked up "Solaris Internals" (volume 1 because vol 2 wasn't even thought of at the time) and read it cover to cover. But this is the caveat. He stated when reading the book, he would not turn the page until he knew "everything" on the current page. It took him one year to finish reading it.

I only relayed that to show that it doesn't really matter if you read the XINU book, the Minix3 book, the FreeBSD internals book, one of the Solaris internals books. If you study it well and really learn the material, it will all be applicable to another operating system.

Having said that, I'd probably go with XINU if you also install it on the Beaglebone Black, Galileo, or Pi. That way you will learn all aspects of an operating system, and all code is in C. It is a lengthy book of somewhere around 900 pages. If you followed Brendan Gregg's advice to only turn the page when you know everything on the current page, you would be very knowledgeable.

After XINU, if you wanted to, you could read the others. "Three Easy Pieces" is really good.

I try to apply, use, all the time everything, to really learn. By using "often" what is learned, to get that deeper in my mind. This is a very good method to me. Thank you for the information and advice sir.

 

[ralphbsz]

I would start with Tanenbaum's book. It is sort of an undergraduate level books for the basics. Don't bother with Minix as an implementation though. Then I would go to the Dinosaur Book - the Silberschatz. It's more advanced, more background and concepts, less practical. I haven't read Remzi's book, but heard about it second hand. I also hear that the XINU book is probably the best for hands-on practice with an OS.

The Stevens books are very good, but they are not programming books, nor kernel books. They are excellent concept and reference books for parts of the programming interface: how to use it, not how it is implemented.

Reading C library reference manuals or Gnu documentation won't help at all. This is reference documentation, which you read when you are debugging a problem.

You claim to know the C language. How many ten thousand lines of C have you written, which went into production, ran bug-free and which were reviewed by colleagues?

For learning how a real-world kernel works, I don't think there is a better book than the daemon book: Design and Implementation of BSD, by McKusick and friends. Get it, and read it 10 times. Try to get your copy autographed by Kirk (mine isn't, which really annoys me).

But all that will not make you a kernel programmer. Programming in the kernel requires first considerable C programming experience, and then a lot of special attention and care. If you are serious about it, and have met the preconditions, then download the source, and start improving something random. For example, find bugs on the FreeBSD bug tracking system, and fix them.

 

[helmet1080]

I would start with Tanenbaum's book.

Me too. If it's more easier, I'll enjoy for sure.

You claim to know the C language. How many ten thousand lines of C have you written, which went into production, ran bug-free and which were reviewed by colleagues?

I wanna write a lot of code, but don't know how to end up as an expert, and I feel kind of blind walking to nowhere. I'm a self-taught programmer, who want one day get hired by a company. Though, I don't to think that the only way to become an expert in this, is by getting hired by a company. Somewhere should be all kind of path in, to get there, as a self-taught.

For learning how a real-world kernel works, I don't think there is a better book than the daemon book: Design and Implementation of BSD, by McKusick and friends. Get it, and read it 10 times. Try to get your copy autographed by Kirk (mine isn't, which really annoys me).

I'll try to get it and read it forever, untill It's useless.

But all that will not make you a kernel programmer. Programming in the kernel requires first considerable C programming experience, and then a lot of special attention and care. If you are serious about it, and have met the preconditions, then download the source, and start improving something random. For example, find bugs on the FreeBSD bug tracking system, and fix them.

C programming experience? Writing... software that run on top of the kernel? What kind of software do you recommend to start, and follow through with? I hope the path it's not that's so long, that takes forever... I wish I could build an instant messaging app in C one day as well. To me it's interesting in C.

Thank you for the reply.

 

[helmet1080]

I would be remiss if a few other additions are not added. They will be considered outside of the mainstream, but are excellent as teaching tools.

Xv6, a simple Unix-like teaching operating system is used at MIT for their operating systems course. You can build it from source and there is a free book available. Russ Cox was instrumental in developing Xv6 and he also worked on Plan 9 at Bell Labs.

Another excellent resource is Notes on the Plan 9 3rd edition Kernel Source which was written by Francisco Ballesteros at Rey Juan Carlos University, Spain. It follows in the tradition of the Unix V6 Lion's Commentary. Plan 9 is an excellent teaching operating system because it is small and the code is very well written.

"Principles of Operating Systems: Design and Applications" by Brian L Stuart is also an excellent book on operating systems. It uses Inferno which was an embedded OS derived from Plan 9 and can run bare metal or hosted; and also Linux, to teach the principles of operating systems. If you do the exercises you will learn a lot. Dr. Stuart uses Inferno to teach his operating systems courses.

Plan 9 and Inferno came from the creators of Unix.

As an addendum: Going to Joyent is a good motivational and inspirational read about one person's desire to become an operating systems engineer, who states he hardly knew C when they began their journey.

I read the read. Very inspirational read.
So many options, and I'll probably take the free book. Thank you.

 

[ralphbsz]

I'll try to get it and read it forever, untill It's useless.

Thank you for your sense of humor.

Writing... software that run on top of the kernel? What kind of software do you recommend to start, and follow through with?

That's a good starting point. But beware that there is a lot more to the craft, art and science of software engineering than programming in C. I would actually recommend learning how to program in a much easier language (python, java, or one of the simplified programming languages used for education). That gives the basic skills of writing good and structured code. Then learn OOA&D. Then learn software engineering workflows and engineering practices. Only start programming in C once you have developed the good taste of knowing how to structure the code.

 

[helmet1080]

Thank you for your sense of humor.


That's a good starting point. But beware that there is a lot more to the craft, art and science of software engineering than programming in C. I would actually recommend learning how to program in a much easier language (python, java, or one of the simplified programming languages used for education). That gives the basic skills of writing good and structured code. Then learn OOA&D. Then learn software engineering workflows and engineering practices. Only start programming in C once you have developed the good taste of knowing how to structure the code.

I know the basics of Python and the OOP. It's a good language. I'm gonna keep going with Python and aside, learn Operating System Concepts. Just to make the journey more interesting and fun. Thank you for your advice.

 

[SirDice]

I wanna write a lot of code, but don't know how to end up as an expert, and I feel kind of blind walking to nowhere.

You become an expert by doing it. Not reading about it, actually doing the work. Solve real world problems. Run into issues, debug code until the sun starts coming up again. Ponder on solutions for days on end. Nobody became an coding wizard just by reading books. Books provide lots of theoretical information, experts know how to translate that theory into something practical that works in the real world.

I'm a self-taught programmer, who want one day get hired by a company.

That's good. Start at the bottom and work your way up. That's how you become an expert at something.