Low-level programing languages & kernels

sidetone

sidetone

Low level programming languages can be used as the basis of kernels and operating systems. Rust and Zig are two modern low level programming languages, which a few operating systems are made from. Zig is in the C family, while Rust is in its own family.

The most advanced operating system from these is Redox which uses Rust. Its capabilities have improved in the last few years. Redox now has drivers for USB, but now they are for input only. Redox also lacks wifi, but it does have Ethernet capability now. Uses basic video drivers. It's available for Intel and AMD 32 and 64 bit. It supports older Arm boards, but not the latest. Redox hasn't reached version 1 yet. It's based on Minux, Plan9, BSD, sel4 and Linux. It has server and desktop versions.

  • Tock is an OS for IoT devices offering security in from potentially distrustful applications.
  • Teaching and experimental - IntermezzOS, Theseus, Moros, bkernel, Felix.
  • RustOS, QuiltOS, Hermit, TwilightOS are notable attempts to become general purpose operating systems.

There's not many Operating Systems written in Zig yet. Zig OS is one which is in its early stages. There are a few kernels written in Zig as well.

It makes sense to use Zig in the base of existing operating systems written in C. Adding Rust to an existing base system written in C would give it too much redundancy, because it's in a different family.

Other languages like Swift, V lang, Julia, C++ and Golang are high level programming languages. So these aren't often used for building kernels and operating systems from the ground up.
 
The zircon microkernel (underlying Google's Fuchsia OS) is built in C++. Other OSes in C++: BeOS, ReactOS, Serenity OS, Haiku OS etc.

There is vinix written in V. In V you can use GC or do your own memory management (the latter would be typically needed for a kernel).
 
Someone involved with Symbolics Genera gave a lightening talk at the last Bay Area Lisp & Scheme Users Group. Seemed like they're trying to revive it.... Too bad Unix never had a decent Window system[1]. [BTW, Michael Toy also gave a talk on CMUCL].

Didn't Sun write their "spring" OS in C++?

[1] Not counting NeWS & what MacOS has. The context of my random window system complaint here is that Lisp Machines had already quite advanced graphics and GUI in mid '80s.
 
It shows the status of Redox, which its capabilities have improved as an OS. Years ago, it didn't have networking at all, and it didn't have USB capability. This has improved that it has wired Ethernet capability now (still no wifi), and that it works with USB only for input. So now, it's a workable operating system. It shows that aside from C, and for modern OS's Rust has an OS which is nearly complete.

It's also to see what programming languages are capable of in terms of being the basis of kernels and operating systems.

And to show the status of operating systems with work on SBC's, like Raspberry Pi, pertaining to Arm architecture.


It's offensive to suggest people are trolls, especially those who wrote a handful of FreeBSD howtos and FAQs, and who've been around a long time, on most topics directly related to configuring FreeBSD and all else on software and computers. These are the ones who hang out in the off-topic sections and hardly have activity anywhere else. I've hardly seen the how to section light up on instructions on configuring applications to make a better FreeBSD system, but off-topic lights up all the time.
 
bakul said:
Someone involved with Symbolics Genera gave a lightening talk at the last Bay Area Lisp & Scheme Users Group. Seemed like they're trying to revive it.... Too bad Unix never had a decent Window system[1]. [BTW, Michael Toy also gave a talk on CMUCL].
Click to expand...

Cool.

Revival hinges on the copyright of Genera, which an employee won in the bankruptcy proceedings and who unfortunately hates Open Source. Would love to see some progress there.

Are you talking about Raymond Toy by a chance?
 
Since my post was deleted (I don't know why, because I did neither implied sidetone might be a troll, nor an AI. I know he is a very constructive, productive, and long year active member of this forums I respect very much.])
I again underline the question of what the purpose of this thread was:

At first I don't understand the distinction about what "Low Level Languages" are made here, since to me every programming language besides Assembler is HOL - including C.

And second I think the core idea "what languages are capable to be used for programming OS'" is putting the cart before the horse.
I like to expand the statement:
drhowarddrfine said:
A lot of people trying to make a lot of operating systems out of the latest, new languages.
Click to expand...
to: Any language can be used to programm an OS.
The question is: For what it will/can be usable?

Recently I've even seen an OS written in brainfuck.
The question is:
What is the target the according OS aims at?
Or, even more general: What is an OS, what is not, where do you draw the line?
Or, to be more productive for this thread:
What OS' we are talking here?

To me the question works the opposite way around:
1. What shall an OS be capable of?
2. Which languages are capable to reach those targets?
So, define the OS' specifications, then look for languages suitable to do the job.
Not vice versa.

And, as I already said in another post in another thread, again I like to remind to also not only to discuss the technical aspects of programming languages, but also consider, and respect social aspects, like:
How many people know how to program in a certain language?
How hard is a language to learn?

Especially for an open source project (are we talking this here?), particulary long term targeted, like kind of an multipurpose, or single-user desktop, or server OS to be taken as a serious alternative for real production, it's only comprehensible to prefer a language more easy to learn above one you already need to be a very good programmer in C, C++, and maybe several other languages before you even dare to think of to start learning it.
For example D, or Rust are definitely no beginners languages. And yes, you may object right: 'Beginners are of no good use for OS programming, anyway!' Of course. That's not my point. My point is: Where is the larger pool you may recruit people from? A language already well known by hundreds of thousands, maybe millions of people, or one that is known by some few hundred people, maybe even less?

Of course, that's not the only point to decide for a language. But to view the technical points, only, are too short-sighted, either.
As a company paying well you can recruit people for a certain language. Especially if you're a big tech leader people are willing to learn certain languages, if they knew good wages come with it.
But within an open source project lot (most?) of the work is done by volunteers, often for free.

So, I think in the case of open source you also need to see the "social aspects of a programming language", too.
 
Documentation is always "key" in projects like this. In the old days you could "self" search the Unix/*BSD/etc man pages and find an interesting system or user space call to make (largely in "C"). You would then be able to write a complete working program quickly (minutes? hours?) and see the results. Where I think many of these projects fall short is in (a) good documentation (b) ease of use when calling into the finished product. I realize "good documentation" is relative :cool:

Where Unix/*BSD went "right" was that the documentation is easy to reach in a simple text format -- on the command line or available on the web, etc. Wrapping documentation into complex wikis, file formats, SharePoint, cloud, etc might be easier for the authors but it can make it challenging for the audience you are trying to attract.
 
we had fortran iv in high school on our 'mainframe' (romanian clone of https://en.wikipedia.org/wiki/CII_Iris_80)
github.com

GitHub - dancorl/felix-assiris: Aproximate emulator for a subset of Felix-C-256 computer jobs with Assiris assembler

Aproximate emulator for a subset of Felix-C-256 computer jobs with Assiris assembler - dancorl/felix-assiris
github.com github.com
worked with punch cards.
was a kind of compiled basic, no pointers, no dynamic allocations
no way to write an os with that
when a runtime error/exception occurred you've got a core dump on the (drum) printer :)
 
Maturin said:
Since my post was deleted (I don't know why, because I did neither implied sidetone might be a troll, nor an AI. I know he is a very constructive, productive, and long year active member of this forums I respect very much.])
Click to expand...
The accusations by other users are no longer there. It was said directly.
Maturin said:
I again underline the question of what the purpose of this thread was:

At first I don't understand the distinction about what "Low Level Languages" are made here, since to me every programming language besides Assembler is HOL - including C.

And second I think the core idea "what languages are capable to be used for programming OS'" is putting the cart before the horse.
Click to expand...
I was trying to understand myself, which modern programming languages were optimal for writing kernels and the basis of operating systems, in comparison to C. Also, the status of the most advanced and specialized operating systems from modern programming languages, which are aside from in C.

I used to be a fan of Minix: the description of how its base was engineered and its small install size. Minix is a stale project, and there's no point in going for that, except for learning purposes or to fork it. Redox is based on it, and there are similarities in their history of driver support, both don't have Wi-Fi, and at one point both couldn't use USB at all. Redox looks like the logical successor of Minix, until a comparable OS comes out written in a low level language in the C family.

I was also looking at which OS's could run on SBC's (Raspberry Pi boards) for desktop, command-line, NAS and router/firewall purposes. Recently, I looked into which BSD derivatives specialized for NAS and router/firewall were available for Arm64 architecture, and none were. FreeBSD, NetBSD and OpenBSD are available for Arm64, but these aren't specialized network distributions. This is where to look for these purposes.
Maturin said:
I like to expand the statement:

to: Any language can be used to program an OS.
The question is: For what it will/can be usable?
Click to expand...
Maturin said:
Recently I've even seen an OS written in brainfuck.
The question is:
What is the target the according OS aims at?
Or, even more general: What is an OS, what is not, where do you draw the line?
Or, to be more productive for this thread:
What OS' we are talking here?
Click to expand...
It seems that low level languages are intended for operating systems and kernels. It's been pointed out that other languages can be used for writing operating systems. It's nice to know whether that's optimal, and it's interesting to see discussion on what extent they can achieve.
Maturin said:
To me the question works the opposite way around:
1. What shall an OS be capable of?
2. Which languages are capable to reach those targets?
So, define the OS' specifications, then look for languages suitable to do the job.
Not vice versa.
Click to expand...
I was looking at Redox as an almost complete operating systems, and what it was written in. And comparable modern language which have potential to achieve similar.

Additionally, I've been questioning myself if a kernel could be written in Julia, and if it could be done in entirety, even if for no other purpose than running Julia and/or Python. From Internet searches, it appears as if it can't be used, but based on discussion that unique languages have been used to write a kernel before, is that possible?

Maybe discussion would provide better insight than what's available through searches or hard to find information.
 
Now I got a better idea of what this thread is all about.
Yeah, this kind of discussion for sure is not a bad idea at all, even if it's very general, and theoretical, and if I still think it's either a very large field, maybe too large to come to useful conclusions within a thread in a forum, but you can collect, and discuss ideas, sure.

I think, we're talking open source and free to use OS, which again underlines my point of "social aspects", and
of course as CShell pointed out documentation is determinant - crucial!

Correct me if I'm wrong:
All I know is most languages were created for to do certain things better for special purposes as existing ones, but only few ones were designed as OS programming language (C clearly is one.)
:-/ ... maybe it was a start to first collect languages (serious ones, not brainfuck, or likewise), designed for OS programming, which rules out all scripting languages, and languages that need a runtime environment, or VM, because you already need kind of a basic OS (kernel) to provide that first, before you can build anything with such upon it.

Additionally speed is of some importance (who wants a slow OS?) Providing/supporting concurrency, parallelism, and multithreading would be also very nice, if not mandatory today. And my personal favor (there not so many OS really providing this) real time computing was something, especially if you point into the direction of minix.
There are not so many languages you can put into that pool.
 
sidetone said:
I was trying to understand myself, which modern programming languages were optimal for writing kernels and the basis of operating systems, in comparison to C.
Click to expand...
That is indeed an interesting question, and a part of what is called "systems research" (the research into how to build operating systems). That used to be a rich discipline, with two major conferences (SOSP and OSDI), and lots of research work going on, both at universities, and at the industrial labs (Bell Labs, HP Labs, IBM Research, and smaller ones like Xerox or Fujitsu). Starting in the 90s and early 2000s, the field underwent a transition. The tip of that iceberg might be Rob Pike's "Systems Software Research is Irrelevant", which has an interesting graph: How many new operating systems are presented in papers at SOSP? It's a step function, going from roughly 10 per (biannual) conference to roughly 2. There just is no point writing a new OS from scratch any longer, because Unix and its bastard stepchild Linux has occupied the field. A corollary of that change is that there is little point studying new languages for use in systems, since all production use is using systems written in C, for better or for worse.

One of the factors is that a lot of the R&D activity has moved from R (in academia and labs) to D (in production companies). For example, the new OSes being built inside places like Google (Android and Fuchsia), Apple, Nokia, Cisco ... are intended for production use, and supposedly quite interesting, but there is little public disclosure and discussion about them. At least L4 and seL4 are a standalone company with more ties to universities, so more is known about them.

Another factor is that a lot of "systems" research work has moved to specialized disciplines (and therefore specialized groups of people, and specialized communication venues like conferences and journals). The two big pieces here are file and storage systems (with conferences like FAST, MSST ...) and networking (NSDI...). The questions in those subfields is much less "what language to implement the system in", and much more "what should the system do and what qualities should it have".

sidetone said:
Minix is a stale project, and there's no point in going for that, except for learning purposes or to fork it.
Click to expand...
Minix was intended as a teaching tool, not for production. The fact that it can actually be useful (both for desktops and for embedded) is testament to Tanenbaum's skill, but that is not its purpose.

sidetone said:
... both don't have Wi-Fi, and at one point both couldn't use USB at all.
Click to expand...
Building a full WiFi or USB stack is incredibly tedious. And not terribly intellectually challenging. If the purpose of an OS project is to teach fundamentals, or to research how to build OSes, then putting the years of effort required to build a production-worthy WiFi or USB stack is completely wasted. Do the minimum possible to enable research or teaching use.

sidetone said:
I was also looking at which OS's could run on SBC's (Raspberry Pi boards) for desktop, command-line, NAS and router/firewall purposes. Recently, I looked into which BSD derivatives specialized for NAS and router/firewall were available for Arm64 architecture, and none were.
Click to expand...
People who do serious embedded and IoT work (production worthy commercial) usually use specialized OSes, see above for a discussion of L4 and seL4. Same with network servers, both routers and NAS. In many cases, network and NAS servers use a general purpose OS as a loader, which then puts the production code into memory and gets out of the way. Famously, NetApp and Juniper use FreeBSD for that, but that does NOT mean that FreeBSD is in the data plane or control plane; it just loads a specialized (privileged) executable that then does all the work. Similarly, IBM's storage array box (the Shark) used to boot using AIX, and later transitioned to Linux; EMC's Symmetrix and Cisco routers use a home-grown OS for that (and I'm ashamed to not know what the other major players like Hitachi and Extreme do).

Hobbyists use general purpose OSes, mostly Linux.

sidetone said:
Maybe discussion would provide better insight than what's available through searches or hard to find information.
Click to expand...
Go to the nearest big university, and sign up for the operating systems class. Or download the papers from the last 10 year's worth of SOSP and OSDI. This information is not hard to find, it is just published in specialized places.
 
Now that I think about it, Redox is about where NetBSD was 10 years ago. Back then, NetBSD hardly had Wi-Fi, it lacked GPU acceleration then. Unsure how to compare USB capabilities. Even with what is lacking in Redox, that's a major step. At that time, Redox didn't have wired network, or USB capabilities at all. NetBSD ported GPU acceleration in part from FreeBSD more recently, and it ported its own for VideoCore on Small Board Computers. Of course those video drivers are largely derived from Linux. Now Redox is useful as an OS.

ralphbsz said:
Minix was intended as a teaching tool.
Click to expand...
Everyone who knows about Minux knows that. It has a good design, which gave it potential for much more.

ralphbsz said:
Go to the nearest big university, and sign up for the operating systems class.
Click to expand...
I've taken a college class for Linux/Unix before. It was using a Linux distribution on a virtual machine on top of Windows, for purposes of teaching about Unix on the command line. I don't remember much about it, but it was using shell and logging in to a main computer through ssh. There's a need for other classes too. Minix would make a good teaching tool in such classes.
ralphbsz said:
Or download the papers from the last 10 year's worth of SOSP and OSDI. This information is not hard to find, it is just published in specialized places.
Click to expand...
Good advice.
Maturin said:
Now I got a better idea of what this thread is all about.
Click to expand...
Mostly, about programming languages optimal for kernels as you've written. And also, further curiosity about how far higher level programming can go in regards to kernels and operating systems and whether they need to be supplemented. And to show how far Redox has come along in terms of a useable system for as a desktop and/or server, and for modern hardware architectures.

Looking at which OS using a modern programming language is the closet to production use. Comparing Redox now to its past. Additionally, now I understand Redox to be at a point now where NetBSD was about 10 years ago, give or take a few capabilities.

If a US government recommendation was for people to move past C to Rust or other modern or safe languages for code, how that would be feasible for the majority of a computer system, for the state of OS's are based on a modern programming language.

FreeBSD needs to stay as is, or be in something in the C family. Anything outside of the C family would be a fork, which would be great, but it would no longer be authentic FreeBSD. Even if it stays as a legacy system in that regard. Eventually, I predict that FreeBSD won't continue as a legacy system, and will be in the C family for a safe and modern language.
 
ralphbsz said:
Fortran? PrimeOS!
PL/1 and PL/S? Most IBM mainframe operating systems (plus some assembly)
Click to expand...

Nope. The z/OS (MVS) kernel is not written in PL/1. That's a high-level language. Some of the MVS kernel is written in PL/S but PL/S is not a compiler per se. It's a translator. It translates PL/S into Assember. Then one uses Assembler F or Assembler H to assemble the kernel into binary objects.

JES/2 and JES/3, the primary subsystems, (I worked on both as well) are written in Assembler, directly.

BTW, PL/1 is a horrible language. It's inconsistent with its own language constructs.
 
You people underestimate how difficult writing an operating system is. Let alone a stable, performant one. FreeBSD isn’t going to be rewritten in another language, period. Too much engineering, innovation, and historical momentum went into the system.

There are already ongoing efforts that address mUh SeCuRiTy anyway.

RedoxOS will probably displace Linux at some point IMO.
 
Many OS projects start out as one person projects, to scratch a particular itch. Most don’t produce anything useful. Some continue if more people get interested and pitch in. Even the initial version of unix was a one person project!

It is perfectly normal to not have a fully formed OS idea in your mind and you just want to explore. Even if you have a fully formed OS in your mind, often you may find things evolve in unexpected ways.

I think learning about an OS (or for that matter anything else) by building an even toy OS is a grand idea! Just reading code doesn't give you a good enough mental model.
 
sidetone said:
I've taken a college class for Linux/Unix before. It was using a Linux distribution on a virtual machine on top of Windows, for purposes of teaching about Unix on the command line. I don't remember much about it, but it was using shell and logging in to a main computer through ssh. There's a need for other classes too. Minix would make a good teaching tool in such classes.
Click to expand...
That is a class about "how to use an operating system", in this particular case Linux from the command line.

The class I was thinking of is: what is an operating system, what is its purpose and requirements, how is it built, what parts does it have. Here is a class description from a nearby university:

A detailed study of the issues involved in operating systems design and implementation. Readings cover current research topics and systems of historical significance. Topics include (but are not restricted to) process and memory management, protection, security, synchronization, performance evaluation, file systems, distributed systems. Students complete significant programming projects that extend or modify existing operating systems.
Click to expand...

The classic textbook for this would be the dinosaur book, a.k.a. Operating Systems Concepts by Silberschatz. More hands-on and more modern would be Tanenbaum, using Minix as an example. Recently, people have told me good things about using the Arpaci-Dusseau book (Three Easy Pieces) instead, but I've never read it.
 
I learned about the existence of Perk and C3. C3, Perk, Hare and lang/zig are modern substitute candidates for C. These are safer languages than C. Also they have syntaxes similar to C to have some type of compatibility to it, including ability to compile it.

devel/hare as a small programing language seems useful as the basis of a base system meant for higher level specialized programming languages on top, such as Julia, Lobster, Carbon, Beef or Go. It uses
devel/qbe, rather than LLVM, as a backend for its lang/harec compiler. Hare fits on a 3 1/2 floppy disk, and it can be used to make compilers. Helios is an operating system written in Hare. This Operating System supports x86_64, while Arm64 is a work in progress. Installing Helios isn't a straightforward.

On FreeBSD, Hare's dependencies use GNU components, when languages like this would be expected to be first class citizens in the BSD world: qbe uses gmake, harec uses binutils and is under AGPL3. It turns out that Hare is a Linux Foundation joint, where the standard library is mandatory dual licensed under MPL2. Qbe is also non-virally licensed as under MIT.

I didn't recognize that Rust was in the extended C family, but it's not meant for C compatibility to the extent of Zig, Perk, Hare or C3. Cargo doesn't make sense, as it looks like package redundancy over the packages that the Operating System uses. I saw Rust having redundancy of cargo packages with compiling in addition to ports on FreeBSD. A package system for a programming language only makes sense for containing libraries which header files call for, not for entire programs.

Beastie7 said:
FreeBSD isn’t going to be rewritten in another language, period
Click to expand...
It's definitely not going to be rewritten in a language which varies greatly from original C. FreeBSD's definitely not going to be rewritten in Rust, for instance. It's going to be made safer, either how you referenced in C, or as you may disagree in a safe language very close to original C.


A lot of references to resources for OS design. Something like Hare seemed like a good programming language for learning.
 
covacat said:
no way to write an os with that
Click to expand...
But FORTRAN is till now the standard language for calculations, for engineers, found in supercomputers.

In this thread we see how central is a specific programming language for a specific OS.

What open source OS offers its own implementation of the programming language?

You see for example that plan9 offers it, also original unix.
 
hruodr said:
But FORTRAN is till now the standard language for calculations, for engineers, found in supercomputers.

In this thread we see how central is a specific programming language for a specific OS.

What open source OS offers its own implementation of the programming language?

You see for example that plan9 offers it, also original unix.
Click to expand...
My understanding is "what programming languages are used for writing OS kernels". I think FORTRAN (at least 77 and older spec) is not at all good at it.
FORTRAN is good at writing number crunchers (intensive schientific numeric calcurations), especially for any calcurations requiring complex.
 
You must log in or register to reply here.
Back
Top