Leaving FreeBSD with broken heart

MrBSD said:
Good luck. Ill take your place. I have been using arch linux for 13 years now and i recently switched to freebsd and i love it. There is a lot to learn here and i see this forum is full of extremely knowledgeable people along with devs actively participating. Its good to be here.
Click to expand...
Indeed! To be frank, I think BSD may be complicated for junior Linux users or Ubuntu users that just want their browsers and games running (what is totally fine). But man... for advances Linux users and developers I think the BSDs distributions are the places to be.... And as you are starting now, when reading the FreeBSD codes, don´t forget to pay attention to the history of computation imprinted everywhere :)
 
kpedersen said:
Lack of CUDA is a shame but when expediting competencies, I am vehemently against locking myself into a single vendor. OpenCL is a bit shite but will luckily remain being shite long after CUDA has disappeared. Possibly it is one to look into instead?

Also, if you are just now playing catchup with ML/AI, you might want to instead put your feelers out at what the next hype may be and prepare to cash in on that instead?
Click to expand...
you are totally right! and to be frank, it seems an entire industry is falling into that issue. Currently I have some pressure from ongoing projects.
 
omardevonlittle said:
This must be some kind of bait. If OP really joined FreeBSD a year ago and loved it and "didn't know how he lived without FreeBSD", why didn't he join a year ago? Why didn't he ask beginner questions, or CUDA questions?

I've been active on some topics regarding CUDA and AI on FreeBSD, and I don't even have a strong Nvidia card! That's because I joined the FreeBSD community for real a little over a year ago and I really don't know how I managed to get by without it.

If this guy was for real, he would get one server with Nvidia cards on Linux, and a workstation with FreeBSD, but that is not interesting or gets attention in this forum like ragebaits :(
Click to expand...
Are you saying you managed to have CUDA running on FreeBSD ?
 
omardevonlittle said:
This must be some kind of bait. If OP really joined FreeBSD a year ago and loved it and "didn't know how he lived without FreeBSD", why didn't he join a year ago? Why didn't he ask beginner questions, or CUDA questions?

I've been active on some topics regarding CUDA and AI on FreeBSD, and I don't even have a strong Nvidia card! That's because I joined the FreeBSD community for real a little over a year ago and I really don't know how I managed to get by without it.

If this guy was for real, he would get one server with Nvidia cards on Linux, and a workstation with FreeBSD, but that is not interesting or gets attention in this forum like ragebaits :(
Click to expand...
BTW, I am not a absolute beginner. I started my career developing SW in AIX, what makes FreeBSD somehow nostalgic to me.
 
Maturin said:
So, by principle it's always (way) more secure to blacklist/block everything by default, and whitelist/allow only explicitely selected things you know what it is and why you allow it.
Click to expand...
It's just pretty intimidating to start writing jail rules from scratch every time. Or are there general/per-app templates for FreeBSD jails?
 
canihazsekooriti said:
It's just pretty intimidating to start writing jail rules from scratch every time.
Click to expand...
I was just referring to a general principle.
It can also be said about firewalls, or operating systems, and other things.

Another principle: If you write everything completely new by scratch everytime you produce something similar, you are doing it wrong.
If you search the net, or this forums you will find a lot of "templates", for jail usage, and many other things.
But of course I cannot tell, if there are any you can just copy-paste to fit your particular situations exactly, or how far you need to adapt them.
 
startrooper said:
Are you saying you managed to have CUDA running on FreeBSD ?
Click to expand...

Some people have run CUDA Linux programs in the Linuxulator. Requires careful juggling of NVidia driver versions with in the Linux environment and the FreeBSD machine.

I'm not sure about development tools, but I don't see why they wouldn't run in the Linuxulator.
 
startrooper said:
I thought hard about dual boots and VMs, but I have a HW with limited capacity to start with.
Click to expand...
There is another solution to this conundrum. You're not limited to having a single PC on your home network. It's how I started too, first just one PC, dual booting FreeBSD and Windows (think it was 95 or 98). Got sick and tired of rebooting to switch back and forth. Got a second PC. Very limited box, old hand-me-down. It was cheap (as in free). But it allowed me to run FreeBSD 24/7 on it, could let it built stuff while I rebooted my desktop to play games. Fair warning though, this homelabbing might prove to be quite addictive, and you'll end up with a crap ton of equipment running all sorts of stuff :D
 
startrooper said:
Are you saying you managed to have CUDA running on FreeBSD ?
Click to expand...
Example here in PR 287895.
Corresponding x11/linux-nvidia-libs[-304|-340|-390|-470|-devel] is needed in addition to ports listed in Comment 8 there.

Note that I myseld don't using CUDA. Installing x11/linux-nvidia-libs[-devel] just for confirming that the isntallation goes fine or not on maintaining. But at least benchmarks/clpeak ran for me on my Quadro P1000 (notebook).

And problems on RTX50* series seems NOT limited with FreeBSD.
But at least for FreeBSD, (I myself cannot confirm as above-mentioned P1000 is my only nvidia GPU, though,) candidate for fix seems to be existing.
 
SirDice said:
There is another solution to this conundrum. You're not limited to having a single PC on your home network. It's how I started too, first just one PC, dual booting FreeBSD and Windows (think it was 95 or 98). Got sick and tired of rebooting to switch back and forth. Got a second PC. Very limited box, old hand-me-down. It was cheap (as in free). But it allowed me to run FreeBSD 24/7 on it, could let it built stuff while I rebooted my desktop to play games. Fair warning though, this homelabbing might prove to be quite addictive, and you'll end up with a crap ton of equipment running all sorts of stuff :D
Click to expand...
Just read my life in those line :D
 
canihazsekooriti said:
I would be leaving FreeBSD because with FreeBSD's jails everything is blacklisted by default and you have to know what to whitelist whereas with Linux' firejail everything is whitelisted and you can just blacklist because you know what there is to blacklist.
Click to expand...
That is how security is built - same with firewalls - you first block everything - and then only allows what is meant to be allowed.

Maybe that will help :)

chatgpt.com

Firejail‑style App Sandboxes Using Free Bsd Jails (fjail)

A conversational AI system that listens, learns, and challenges
chatgpt.com chatgpt.com
 
Just my 2 cents here, but has OP heard of Zluda? it's basically a translation of CUDA code that can run o AMD GPUs... Zluda is a WIP, which is last year's info. There was a thread on the Forums about that: Thread local-llms.93502, see post #12.

It's kind of my bucket list item to play with that stuff on FreeBSD when I find the time. 🤤
 
Model runs on the GPU on FreeBSD,


ollama ps
NAME ID SIZE PROCESSOR UNTIL
phi4:latest ac896e5b8b34 10 GB 100% GPU 4 minutes from now
x@myfreebsd:~ $
 
I've stated it in other threads but CUDA and lack of md-raid are the reasons I do most of my work in Linux. freeBSD for me is about embedded (specialized) systems, nostalgia, and bit more grey matter in the bsd forums than the linux ones.

I'm on the fence as to whether I'd ever bother to write a "so long and thanks for all the fish" post, myself, cause in the end, what does it accomplish?
 
I have only had a quick play with LLMs so far but ollama is just a tiny Go wrapper around llama-cpp (C/C++ underpins the entire industry as usual...).

This underlying middleware should support Vulkan. The backends list is here. Potentially this can be an option instead of CUDA?

I note it also supports OpenCL but potentially only on Adreno GPUs (ARM/SoC stuff). I am certain this will improve in future.
 
My ollama "db",
NAME SIZEGB
medllama2:7b 3.8GB
devstral:24b 14GB
sqlcoder:15b 9.0GB
mathstral:7b 4.1GB
nuextract:3.8b 2.2GB
qwen2.5-coder:3b 1.9GB
phi3:14b 7.9GB
mistral:7b 4.4GB
codellama:13b 7.4GB
 
I have ollama running on a laptop with CPU-integrated AMD GPU. It's not very fast of course, but it does use the GPU.

But I suspect the OP wants to do more than ollama. OP?
 
I'm currently using ollama "medical". Very interesting.
PS: I asked for a 2by2 backtracking tic-tac-toe. But the code generated is null.
So on language it's ok. On problem solving it lacks very much.
 
kent_dorfman766 said:
I've stated it in other threads but CUDA and lack of md-raid are the reasons I do most of my work in Linux. freeBSD for me is about embedded (specialized) systems, nostalgia, and bit more grey matter in the bsd forums than the linux ones.

I'm on the fence as to whether I'd ever bother to write a "so long and thanks for all the fish" post, myself, cause in the end, what does it accomplish?
Click to expand...

I don't do serious compute with CUDA and applications, but I've seen some games use it (DLSS in Diablo 2 Resurrected on Linux/Wine).

I'm not sure if leaving an OS can be a permanent thing to write about. Not having adb on FreeBSD means I can't update my phone the way I'd like and a game partially didn't work; while I'm on Linux currently, I'm still interested in checking out other OSs and FreeBSD again! If I had reason to install Windows 98, XP, or 7 today, I'd do those too :p Even macOS is on the table if I ever feel like deciphering ACPI.

FreeBSD might not handle CUDA well today, but I believe that'll change with enough interest in the future!
 
startrooper said:
I hope that will change. I work with linux in many projects and products and time to time I tell people, hey! freebsd seems more stable and reliable. Hope NVIDIA will find a positive business case using freebsd, :)
Click to expand...
They will continue to sell chips for all the big companies that need to put LLMs everywhere for the forseeable future.
We'll have to wait until that bubble pops.
 
startrooper said:
Good morning FreeBSD community!

I am traditionally a Linux user and moved to FreeBSD a little more than a year, and man.... I loved it! It is some of those moments we say "how have a lived until now without it?".
But it is with broken heart that I am planning to move back to Linux as I currently have a urge to expedite my competences in ML/AI and the lack of CUDA integration by NVIDIA
has been a obstacle difficult to get around. I thought hard about dual boots and VMs, but I have a HW with limited capacity to start with.

Anyhow! I may be leaving but I joined this forum to remain tightened to FreeBSD looking forward to make a comeback!

Cheers!
Click to expand...
That's exactly the reason why my "AI" server at home is Linux and Ubuntu on top of everything (don't like Ubuntu I would prefer a clean Debian). Still my other servers and storage are FreeBSD.
 
If I ever got into AI I was thinking about buying a dedicated box for it like Jetson Orin Nano (although if local AI is like GPU mining I'd just buy some x86 hardware :p). I like my server and workstation/laptop running the same OS, but specialized devices can run whatever.
 
You must log in or register to reply here.
Back
Top