Ollama

[cracauer@]

My posts are derived from NapoleonWils0n and his projects on podman with cuda and cracauer@ saying it's possible to run cuda in the linuxlator.

Of course for the Linuxulator you also need to compile llama.cpp into Linux binaries.

But I can imagine making a port linux-llama-cpp with CUDA support.

 

[grandpa]

Of course for the Linuxulator you also need to compile llama.cpp into Linux binaries.

But I can imagine making a port linux-llama-cpp with CUDA support.

That would be nice yes. It would require deep knowledge of the uvm modification and how the linuxlators translation layers work so I can't do a port like that.

Besides I have 3 other ports committed and not yet found a reviewer with some spare time.

Also the maintenance when Nvidia changes something that affects the uvm modification would probably be a nightmare.

I just got curious since I don't normally run the linuxlator so there was a learning opportunity for me when I bulldozed this through.

/grandpa