How (not) to build .NET coreclr on FreeBSD

I am just hoping that it of course wont drag some more naive companies away from i.e Linux and into the Microsoft prison because they have tied themselves too closely to .NET by bad decisions (and Microsoft's EEE strategy).
I am positive this is a Microsoft strategy. I am positive it is happening because I see Linux people on amateur forums--specifically reddit--who laud over Microsoft as if they are the savior of Linux.

Not that I think professionals, overall, visit reddit on a regular basis but the reddit script kiddies sometimes influence their elders.
 
My company runs two highly performant web sites that you have visited and probably still go to at least occasionally. We continue to do everything in C

I realize this party is over but this is really interesting. All C or C++ web applications are fascinating. I’m currently trying to figure out Beast, which is a header only library, not a framework. It’s not going very well. I haven’t done much web development. Nginx is extremely user friendly, though.

Just curious do you guys use any JavaScript at all in your websites?
 
I realize this party is over but this is really interesting. All C or C++ web applications are fascinating. I’m currently trying to figure out Beast, which is a header only library, not a framework. It’s not going very well. I haven’t done much web development. Nginx is extremely user friendly, though.

Just curious do you guys use any JavaScript at all in your websites?
Beast looks interesting but because it is tied up and stuck inside the murky swamp known as Boost, I will never touch it ;)

A C++ web development architecture that a colleague of mine is showing great results with is using mongoose (https://github.com/cesanta/mongoose) as a combined http and websocket server (single .c and .h file; much less tightly bound than boost::Beast).

His C++ server does very little more than host the html / Javascript files, and then provides a simple RESTful API using both AJAX and Websockets. It is extremely fast and doesn't require any dependencies on a big fat server like Apache. In general though, the majority of the web app is powered by the Javascript running on the client's machine. Better distribution of resources.

I have used Mongoose with many of my projects but only for websockets. I cannot accept CSS as a GUI library so have tended to shy away from full blown web development ;). I find it really good. Especially the older version which is multi threaded. My only complaint is that it deletes client connections at will and functions via callbacks making it very hard to wrap in safer languages such as C++ or Lua.
 
the murky swamp known as Boost, I will never touch it ;)

Boost can definitely get murky hah. So many different versions, “missing this library”, “missing that library”, etc. Apparently one of the key ingredients behind Beast is its core dependency Asio, the async library based on the proactor design pattern (concurrency without threads). Supposedly it can handle a ridiculous amount of connections, if I can ever make it work! Maybe in another lifetime.

Mongoose looks pretty nice. Thanks for the heads up. I’ll try it out.
 
Yes! Asio is such a terrible dependency to have. When looking for any decent networking related libraries, I always see the C++ ones have sodding Asio as a dependency or overly utilize some C++20-only standard async architecture that will never work on older or C++ compilers 25 years from now.

I even decided to write my own (https://github.com/osen/libws) just to avoid Asio in a couple of projects haha.

I stopped looking for C++ libraries long ago and decided to just find C99 ones and polish them up a little for safety and C++.
 
Very impressive. Work project?
Thanks :)
Originally written as part of a contracting job but I then decided to clean it up and add Windows support.
One day I plan to turn it into this (http://websocketd.com/) but without the non-futureproof choice of Google Go as a language.

A word of warning: Mongoose is better in most ways. Mine is not entirely standards compliant (especially in the standard HTTP). It also supports only plain text websocket streams, not binary.
 
I think most of the issues raised regarding the sustainability of the .net stack (namely porting to new platforms), have been solved by Mono.

Mono has no such issues to port mono to new platforms. They are even already able to target web assembly (even though this is still a work in progress). Mono even supports llvm. So with the open-sourcing of the .net framework and the way Mono has been integrating Microsoft's code where it makes sense, I see no downside to using .NET (as long as one uses the Mono implementation - .net is a standard that anyone is free to implement).

Moreover, thanks to language services such as omnisharp, one can now conveniently use emacs to develop. This brings intelliense, autocompletion, refactoring and the like straight to emacs without any vendor lockin/creepy licensing.

To me, besides the features of csharp, a key reason to use .net/mono for a business is its vast standard library and plugins ecosystem. Things such as the extensive network, text, and regex libraries all the way to linq make a huge difference in developer time.

Another key reason that shouldn't be overlooked are build times. Builds are incremental, but are also fast to the point that building after every file change often is a workflow (while it is not even an option in c++). You can work on multi-million lines projects and build in 5 seconds or so. All of this combined makes for a real productivity booster.

FreeBSD + Mono + Emacs/Omnisharp constitute a great technological stack that I think has a brillant future. Mono isn't some sort of poor child, it is a great engineering work having a very interesting architecture.

I personally prefer to spawn two more instances if it can maximize productivity and save me from spawning two extra technological stacks. This makes a lot of difference when it comes to growing a business. One can always rewrite for high-performance down the line when clients are closed and money is in the bank, if there is ever a bottleneck.

I am obviously not advocating for rewriting parts of FreeBSD using .net, I am only saying that it adds plenty of value in the community. The existence of a credible mono port in the ports tree largely motivated my decision to choose FreeBSD.

Once this is said, I do not think this community should spend too much efforts trying to dance .net core tango until Microsoft gets its act together. I personally do not plan to use it in the foreseeable future because .net core looks too much like .net framework 1 at the minute. It looks like the old-days microsoft abominations, with poor documentation, non-stable APIs etc... it may take an additional decade before it really becomes a credible alternative in the Unix world. For time being, mono fits the bill, and probably always will considering its far ranging support for platforms (including web assembly, the future of client programming on the web), while Microsoft made clear that .net core focus will be the web server scenario. So time invested on mono is time well invested.

The performance hit between mono and core is not so terrible and is not even important for the vast majority of people. And if someone is really serious about performance, he can always tweak and optimize mono compile options. However optimizing compile options and building from source is not a mindset commonly found in the windows world which help explain the general feeling. Few people probably know that it is even possible to change the garbage collector algorithm used by mono or write a drop-in replacement (GC being one of the main bottleneck in vm-based languages).
 
Builds are incremental, but are also fast to the point that building after every file change often is a workflow (while it is not even an option in c++). You can work on multi-million lines projects and build in 5 seconds or so. All of this combined makes for a real productivity booster.

Building after a file changing in C++ is actually more efficient than in .NET. In .NET, you have to compile and generate the entire assembly (and then generate the binary) whereas in C++ the compiler is instructed to just compile the units that changed and then relink. If you correctly forward declare in headers, then this has never really been a problem.

For tiny projects or non-native compiles, .NET can be faster to generate bytecode but for larger projects compiled to run on bare metal, the build design of C++ greatly wins out.

That said, C++ and .NET have never really competed. .NET has always just piggybacked and consumed C++ binaries via the use of (often out of date) bindings. Java and .NET are the main competitors to one another and whilst I trust Java more (yes, even with Oracle at the wheel), I do see that .NET has more technical merit; especially when it comes to stack / boxed functionality.

Its deprecated now (like all things Microsoft) but a closer competitor would be Microsoft C++/clr:safe and C#. I wish Microsoft had open-sourced that compiler rather than just take it to the grave with them. Bunch of arses ;)
 
Frankly the one thing I hate most in C# is, that it changes way too often. The second one is some fundamental classes and interfaces they dropped in .Net Core (IDataReader and DataTable no more? Really?!?)
 
Building after a file changing in C++ is actually more efficient than in .NET. In .NET, you have to compile and generate the entire assembly (and then generate the binary) whereas in C++ the compiler is instructed to just compile the units that changed and then relink. If you correctly forward declare in headers, then this has never really been a problem.

For tiny projects or non-native compiles, .NET can be faster to generate bytecode but for larger projects compiled to run on bare metal, the build design of C++ greatly wins out.

That said, C++ and .NET have never really competed. .NET has always just piggybacked and consumed C++ binaries via the use of (often out of date) bindings. Java and .NET are the main competitors to one another and whilst I trust Java more (yes, even with Oracle at the wheel), I do see that .NET has more technical merit; especially when it comes to stack / boxed functionality.

Its deprecated now (like all things Microsoft) but a closer competitor would be Microsoft C++/clr:safe and C#. I wish Microsoft had open-sourced that compiler rather than just take it to the grave with them. Bunch of arses ;)

I was actually talking about non-native compiles (I am not saying this to reject your whole comment, this is just a remark).

I don't know, it looked like people were saying that low-level non-interpreted languages (specifically c/c++) should preferably be favored over c# in the unix world, so I wanted to offer a different view.

Frankly the one thing I hate most in C# is, that it changes way too often. The second one is some fundamental classes and interfaces they dropped in .Net Core (IDataReader and DataTable no more? Really?!?)

Honestly this is not my experience, the language evolves but is always 100% backward compatible. If they allowed themselves to break backward compatibility, the first thing they would have done is making null references impossible, which what they often say is their biggest regret which is too late to fix due to backward compatibility. This is where Microsoft is very different than say Google who does not hesitate to break things and discontinue products abruptly.

The second problem you mention relates to the standard library and not csharp per se. .NET ECMA 335 is a technical standard defining how code written in multiple high-level languages can be run in a single VM able to be executed on multiple platforms. C# is only one of these languages among dozens (F#, IronPython etc...) which is a little known fact. The .NET Framework implements the specification, and so do Mono and .Net Core.

These 3 frameworks are 3 different implementations of ECMA 335 and as such come with different standard libraries. To provide API compatibility, the so-called ".NET Standard" defines an API that the standard library of all implementations must implement. Mono always aimed to be API-compatible, but Microsoft formalized things with .NET Standard when they intensified their efforts on .NET Core. This API definition should have existed a long time ago but back in the days Microsoft didn't care about cross-platform (unless it was to sabotage it).

Unfortunately, the data access classes you mention are not part of .NET Standard, and therefore are not required to be implemented by ECMA 335 implementations, and as such Microsoft has decided not implement them in .NET Core for time being. They may be present in Mono though, I am not sure. But the point is, this is not a C# issue.

If you ask me, I personally find ECMA 335 (and the .NET ecosystem it defines) a masterpiece of software architecture and reviewing it is what convinced me that this stack was the future. I find it impressive that they were so spot on, so early (2001). The open-sourcing of the Microsoft's windows-focused implementation proved me right. So does seeing Mono target Web Assembly. But one can only regret that Microsoft waited so long to implement a vision that they had long laid down.

Once this is said, what you are saying about DataTable is exactly the reason I stay away from the .NET Core implementation for time being and only care about Mono (which has superior and more mature tooling in addition to having a much more comprehensove standard library). Right now, there are way too many easter eggs like this one in .NET Core. The only point I am trying to make is that the ecosystem goes far beyond .net core and it is important to understand the big picture because there is in my opinion huge business value in this ecosystem, whether it is in terms of time-to-market, productivity and maintainability. And today, you no longer need to sell your soul yo the devil through obscure licensing to benefit from it.

I think that FreeBSD will see a lot of people coming from the Windows world in the next decade. These people will want to enjoy their new freedom in an OS being no less consistent than the Windows they were used to. Right now we are in a transition period, people do not fully realize yet the vast extent to which their possibilities have changed.
 
Frankly the one thing I hate most in C# is, that it changes way too often.
Under the guidance of a relative, who is manager of a large Microsoft shop in Houston, I started my company using all Microsoft .NET stuff for which he got us a lot of software for free under the table. We built a complete system that was ready to launch when...something changed. It all came tumbling down. Microsoft didn't support that anymore and we were to upgrade and re-learn parts we just finished learning and weren't sure we were good at. "Switch to Linux!", my relative told us. Thanks so much.

We lost $50K in funds as we had to rewrite a lot of code and it almost sunk us. We knew Linux wasn't for us the first few days we looked into it. I pulled some old floppies out for a FreeBSD install and you know the story from there.

I will never touch a Microsoft product as long as I can help it.
 
Honestly, I won't even try to argue that you can't be burned by Microsoft. You definitely can, and it's too easy to be, even today. I think ECMA 335 and Mono (and maybe .net core 15 years from now) are the exception rather than the rule. I wouldn't even put my money on .net core right now.

I do not touch any Microsoft-related tech apart from that. That would have been quite a huge baby to throw with the water. But even with Mono you need to be wise about which library you want to depend on since they are resource limited. 99% of the time this means thinking twice before using the Microsoft frameworks (such as Winforms) they have ported.

it's possible not to be burnt by studying Microsoft's trajectory / internal politics / where they are going. Just like you would before using any open-source project. But this needs to be done even more carefully with them, especially back in the days when .net was closed-source. Most of their stuff are still closed-source today.

For example we stopped using SQL Server long ago. We use mono with open-source libraries or build our own. No ASP.NET, ASP.NET MVC, Webforms, WCF, WPF etc... we don't touch any of that. This is a huge BUT that I should probably have added before. All we are interested in is mono implementation of ECMA 335 and the high-level languages of the ecosystem (csharp, fsharp, etc...). For the rest beware. Business is business... you'd better know Microsoft well before adding their libraries/product as a dependency.

We have long decided that being a Microsoft shop wasn't a good idea. However, ECMA 335 is a good idea. that's the positive I take from our involvement with Microsoft. It may take focusing hard not to throw it with the rest but this is a real deal. I briefly highlighted some technical merits, but there are also business considerations such as access to a huge talent pool etc...
 
Any technology or company we have to tip toe around is not one I will get involved with. My company turned down work if it ever involved us having to touch Microsoft anything. If MS products were involved, the client had to deal with it and hand us a POSIX compatible environment or interface we approve of.
Running Mono on UNIX/BSD/Linux is like trying to force fit round pegs into square holes. To this day, I will never understand why professional software is written for and runs on a consumer product like Windows.
 
I don't know, it looked like people were saying that low-level non-interpreted languages (specifically c/c++) should preferably be favored over c# in the unix world, so I wanted to offer a different view.

The main benefit Limbo, Java, .NET have that I can completely accept and understand is that they can provide (non-realtime) safety in terms of memory because everything is checked. Perfect for a business usecase because it opens you up to faster deployment and (hate to say it) cheaper programmers because they don't need to be so darn experienced or skilled.

However, In the world of UNIX and Linux, FOSS / open-source (which lets face it... powers UNIX these days), it isn't so much about money. There is no real time or financial pressure. Especially since FOSS developers do a lot of it for fun and recognition the benefits (safety / speed / cheapness) do not actually apply to 99% of open-source projects which make up FOSS. So if you aren't using .NET for these reasons, what other benefit are you using it for? Fun? Doubt it. They are so darn safe that they are boring languages to write software in. Especially since most of your time will be spent making / fixing bindings for native C libraries.

Then you have the other part of FOSS where because it is a hobby, we want to do stuff right! Unlike in our day jobs, we don't just wan't to blitz out crap or stuff that could potentially break in a few years (if Java 1.6 or Mono 1.x is removed from common repos for example). We are quite happy to use a "harder" language because it isn't always about us (the developer), it is about our users and how our software integrates with their workflow. Having to download a massive JVM or CLR just to run our code is quite a disadvantage. This I feel is why the majority of Linux binaries are C/C++ and libraries are still C (not even C++). God, some users bitch about having to install both Qt *and* Gtk. Or Python 2.7 *and* Python 3.x. Those are tiny things compared to Inferno, JVM, CLR.

As it stands C++ is still demonized in the UNIX world (for quite a few good reasons). However, one day if C++ finally overtakes C... then perhaps another 25 years later, Java or .NET might have a chance. If they (or we) are still alive by then ;)
 
it's possible not to be burnt by studying Microsoft's trajectory / internal politics / where they are going.
Not for me, because I'm too naive I guess, and because, y'know, they lie, keep secrets, and play dirty. I'm like Elmer Fudd vs. Bugs Bunny when it comes to anticipating their next moves. They have much more time and resources for planning their next duplicity than I have for anticipating it, so IMO it's better if I don't even try.
 
FYI - the build steps at the start of this thread no longer work. Microsoft pulled the pre-built FreeBSD SDK binary required for their build script to run. Anyone pursuing this should seriously consider Mono instead.
 
And then they'll pull the FreeBSD version of Mono somehow.
The technical issue is - the core-sdk is written in C#. You need to compile the SDK to get a working toolchain. You need a toolchain to get a working SDK. Chicken and egg. You have to cross-compile from Windows or Linux (with FreeBSD tweaks) to start.

It's a bit like building clang or GCC from source - without a working C compiler.

I suspect it is comically plausible that one could use mono to boot-strap core-sdk.
 
I'd strongly recommend using mono for the foreseeable future rather than .NET Core. Mono will not go anywhere, it's been around forever now, and is stable and has very mature tooling. And it is the backbone of plenty of things at MS (Xamarin, Android and IOs compatibility etc...) so there is a permanent commitment to it. It is like the .NET framework at this point, it can't go away.

In terms of memory footprint, mono may be less optimized than .NET Core out of the box, but let us remember that the fundamentals of the VM are the same because they both implement the same specification : if you do not touch an AppDomain, it does not get loaded in memory. And if the VM is still too heavy for you, if you are truly at the scale where every megabyte matters (unlikely unless you are operating at a huge scale, of have aggressive hardware constraints), it likely won't be a big deal for you to tweak mono's compiler options to trim it and optimize it as much as you need.

The memory footprint of the VM of an headless web application on mono is about 60MB to 200MB, most applications would use about 100MB (putting aside load and the specific performance profile and resource usage efficiency of the application). This is negligible most of the time, unless you plan to run on an embedded device having 256mb of ram installed, in which case you'll still be able to play with mono's compiler options. And I am not the type of person considering that running an electron-based text editor using 2GB of RAM is always reasonable.

I bet .NET Core has been optimized for web scenarios to take the number closer to the one of a LAMP stack (even though this is comparing apples to oranges, since .NET/Mono give more isolation and features to every application - you can import DLLs, and do all sort of things in a secure context, so this necessary involves resource usage duplication where on a LAMP stack everything uses the same PHP instance. This is without even mentioning the sandboxing that .NET provides you - for example I'd never expose my filesystem straight to the web - I'm always shocked to see how much of a direct access to the file system PHP apps use and the tweakings with .htaccess files, scary...).

LAMP can run multiple websites by using well under 100MB of memory largely thanks to the fact that everything uses the same PHP instance and lazy execution. But you pay the price at execution time with much slower execution. But if you are hosting 1000 websites and 99% of them get less than 1000 hits per day, then these maths are arguably very advantageous. The right tool for the right job.

For if you are the marketing department of a company having 3000 websites (blogs, review websites, and all sort of content marketing schemes) to promote a range of specific products using an in-house methodology, then if you are sophisticated, these 3000 websites will likely be powered by a single application, having an integrated dashboard for content editors, and a transparent engine adding styling and graphics assets, and content modules, based on the entity being served (every website would be represented as a well-thought-out entity in your domain model). Then you would for example put a reverse proxy in front of your application to transparently map "marketingdomain1.com" to your backend "http://localhost:<port-to-your-app>/marketing-schemes/entity-name-for-domain-1/" and you have your 3000 websites powered by a single application using 60-200MB of RAM memory.

And the maths work here too. And the thing is, in addition to the huge security benefits coming for free, and to the huge performance benefits at execution time, .NET/Mono and their family of languages (csharp, fsharp etc...) make it very convenient to create and maintain such complex domain models. The experience of creating a complex, yet scalable and maintainable domain model in .NET and seamlessly integrating it with the infrastructure code, while having access to a broad range of expressive languages (csharp, fsharp, but much more), and a wealth of open-source packages for your infrastructure, this experience is quite honestly seamless. Once you have experienced that, you can't fiddle with Python and PHP if you are serious about your business. And I am not talking about Microsoft toolings (which we do not use anyway), I am talking about core concepts and the core features of the languages, and of the ecosystem and of the way everything plays together to foster productivity, quality and shareholder value in an optimal way.

So honestly it really depends of what you are doing, and whether or not you know what you are doing... And to be honest, most .NET shops wouldn't be able to pull out the technological scheme I have described. They need the click-click-click hand holding of Microsoft's tools to ship anything. And I am just talking about infrastructure code here, I am not even talking about the fact that very few teams even know what a domain model is... they know about POCOs.... and glue.... and spaghetti.... and mapping and ETL-ing stuff all over the place.

So it really comes down to what you are doing, and whether or not you know what you are doing. And what I am saying is that in many - if not most situations - if you are a company only deploying your own products, ECMA 335 is quite a huge baby to throw with the water... You're missing on a lot of value if you dismiss it or do not know about it (which most people do, especially in the UNIX community - but for legitimate historical reasons, but times have changed), and you're probably paying a premium without even realizing it, as you'll have to constantly re-implement here and there many complex and expensive stuff coming for free with ECMA 335 (in addition to introducing codebase complexity, less expressive programming languages etc...).

And once all of this is said.... I just saw a one-year old question on stackoverflow of someone saying that their small .NET Core application (20 pages or so, this was actually a headless JSON-API....) was using 350MB of RAM.... So bottom line is, honestly, nowadays there is not even a practical need for .NET Core if you know what you are doing.

Old is not always true forever. Change is not always meaningful or positive, but sometimes it is. And new is not always better. For those of us able to take advantage on Mono on Unix, .NET Core is a solution to a problem faced by Microsoft more than it is a solution for us, since there is no actual need in this area. However, we indirectly benefit since they significantly scaled their investment in open-source because of this program. For example, they multiplied the headcount of developers working on Mono while keeping everything open-source, they even open-sourced the proprietary division that allowed Mono to make money. They also open-sourced the .NET framework, MSBuild, all their compilers, and all sorts of things, so the Mono team has been able to do things such as swapping less performant code for code developed by the 6-figure-salary full-time PhDs paid by Microsoft where it made sense.
 
The technical issue is - the core-sdk is written in C#. You need to compile the SDK to get a working toolchain. You need a toolchain to get a working SDK. Chicken and egg. You have to cross-compile from Windows or Linux (with FreeBSD tweaks) to start.

It's a bit like building clang or GCC from source - without a working C compiler.
Which exemplifies why I always say people should never use this stuff. It will always be fitting square pegs into round holes. FreeBSD has their own tools that are just as good or better.
 
They also open-sourced the .NET framework, MSBuild, all their compilers, and all sorts of things, so the Mono team has been able to do things such as swapping less performant code for code developed by the 6-figure-salary full-time PhDs paid by Microsoft where it made sense.

This made me chuckle. Often code from PhDs isn't always the best for long-term viability. Just check out the garbage collector alone from clr (https://raw.githubusercontent.com/dotnet/coreclr/master/src/gc/gc.cpp)
This sheer volume of over-engineered code is not really maintainable (this alone is bigger than the entirety of TCC: https://github.com/TinyCC/tinycc) and they certainly haven't followed KISS principles.
For an exercise; try getting Mono 1.0 building and running on a current platform. It will take you over a lifetime of manhours to do it. I stay away from "technology" like this.

Compare it to C where it is possible to write a compiler in about a week. Someone much smarter than me could probably even make it production ready in that time.

Even though I mentioned language names; it is the underlying platforms that I am really talking about. VB.NET and CSharp are probably fine languages but until they can drop the runtime complexity; I wouldn't recommend them. But again, that is typical of PhDs, the ideas are often valid but the execution and implementation is often atrocious (this is certainly the case for my thesis). If they can find a better (more portable) solution than a garbage collector and develop a compiler that outputs native binary from CSharp code (similar to dotgnu, unfortunately not what AOT does) then I see CSharp having a much longer life. Oh and just drop VB.NET and give us back VB[6]; it was vastly superior! ;)
 
Back
Top