ollama, GO_DEFAULT, golang binaries in ports and packages right now

cracauer@

cracauer@

Developer
I'm a bit puzzled right now about some things:

1) I can install the go-1.21 pkg but I can't build the go-1.21 port because it says "Building Go cmd/dist using /xcarb3/usr/ports/lang/go122/work/go-freebsd-amd64-bootstrap. (go1.20 freebsd/amd64)" - "This program can only be run on AMD64 processors with v2 microarchitecture support.". This is on an Opteron 6174. Fine, but why is it that the pkg seems to work on this same machine?

2) the ollama port says it uses go-1.22, but GO_DEFAULT in Mk/ is on 1.21. How is the pkg built then?

3) the ollama package starts fine but fails to work in some mysterious way. I had that on another machine, too, and fixed it my rebuilding the ollama port. In this case I can't since I run into issue #1. "Error: llama runner process has terminated: signal: abort trap" is what the client gives.

I didn't have coffee yet, so maybe I'm just thinking square. I'll update this later but maybe somebody with caffeine can shed some light in the meantime?
 
OK, so I wasn't thinking straight.

go-1.22 is a separate port, so no problem installing it.

I brought the too-high CPU level on myself by not realizing it is Go config, not llama config. I can now build everything.

It still doesn't let me start a LLM. Still the abprt trap.
 
Do you have a custom kernel disabling any of the COMPAT_FREEBSD_* options?

Also try compiling with Go 1.23. I have `DEFAULT_VERSIONS+= go=1.23` in /etc/make.conf
 
Ah. Got my claws on an actual error message.
Code: 
LLVM ERROR: Cannot select: 0x3cda676a7460: f32,ch = load<(load (s16) from %stack.8), anyext from i16> 0x3cda676b3620, FrameIndex:i64<8>, undef:i64
  0x3cda676aed90: i64 = FrameIndex<8>
  0x3cda676a7cb0: i64 = undef

Apparently it isn't trying to use the CPU directly but instead does Vulcan through llvmpipe with a target of CPU, which seems to not work.
 
rbranco said:
Do you have a custom kernel disabling any of the COMPAT_FREEBSD_* options?
Click to expand...

Not much customized and COMPAT_* is all in.

Also try compiling with Go 1.23. I have `DEFAULT_VERSIONS+= go=1.23` in /etc/make.conf
Click to expand...

Good idea. But as far as I understand it now I don't need to set the port-wide default go, I can just set it in ollama's Makefile.

I'll be back.
 
rbranco said:
Also try compiling with Go 1.23. I have `DEFAULT_VERSIONS+= go=1.23` in /etc/make.conf
Click to expand...

Go 1.123 builds fine and ollama on top, too and the previous error message disappeared.

Now I am running into a timeout when `ollama run`. Maybe too low per-core speed on this machine. The timeout is hardcoded in a go file, so recompiling.
 
hi mate

i made a video tutorial about using ollama on Freebsd
with emacs as a front end with the gptel package

i have a dell xps 15 2019 with 16 gb of ram
and a NVIDIA GeForce GTX 1650

i can run deepseek-r1:7b

github.com

cerberus/ollama/ollama.org at master · NapoleonWils0n/cerberus

cerberus code library. Contribute to NapoleonWils0n/cerberus development by creating an account on GitHub.
github.com github.com

i start ollama like this ( which is from the ollama pkg message )

Code: 
OLLAMA_NUM_PARALLEL=1 OLLAMA_DEBUG=1 LLAMA_DEBUG=1 ollama start

you can also start ollama like this as well

Code: 
ollama serve

once the server is started you can pull a model

Code: 
ollama pull deepseek-r1:7b

list the models

Code: 
ollama list

run the model

Code: 
ollama run deepseek-r1:7b

sometimes when you run the model it will fail with an http error
if that happens kill the server and retry

Code: 
pkill ollama
OLLAMA_NUM_PARALLEL=1 OLLAMA_DEBUG=1 LLAMA_DEBUG=1 ollama start
ollama run deepseek-r1:7b

try not to use anything like a video player that uses the gpu when running ollama
it doesnt like it

NapoleonWils0n

Thread 'Ollama set up on Freebsd to run llm's and use Emacs with gptel as a front end'

Ollama run large language models on your computer including deepseek-r1, deepseek-coder, mistral and zephyr

In this video i install Ollama on Freebsd 14.2 on a Dell XPS 15 2019 with a NVIDIA GeForce GTX 1650 gpu with 16 gig of ram
on Freebsd 14.2 quarterly release with the 550.127.05 Nvidia driver


I also cover running the ollama lama server and pulling and running models and setting up Emacs with the gptel package as a front end

As well as taking a look at Google Gemini and how to create an access token you can use with a authinfo file for Emacs

ollama...
  • Thanks
  • Like
 
You must log in or register to reply here.
Back
Top