ollama unknown model architecture: 'gemma3'

NapoleonWils0n

NapoleonWils0n

just a heads up

ollama doesnt run gemma 3

ollama.com

gemma3

The current strongest model that fits on a single GPU.
ollama.com ollama.com

i pulled down gemma3:4b with ollama to test it

start the ollama server

Code: 
OLLAMA_NUM_PARALLEL=1 OLLAMA_DEBUG=1 LLAMA_DEBUG=1 ollama start

or

Code: 
ollama serve

pull gemma 3

Code: 
ollama pull gemma3:4b

run gemma 3

Code: 
ollama run gemma3:4b

gemma 3 fails with this error

Code: 
unknown model architecture: 'gemma3'

seems like there was a similar error with gemma 2
havent come across any fix for the issue
 
On my Mac I get "Error: llama runner process has terminated: this model is not supported by your version of Ollama. You may need to upgrade".

I wonder why it is downloading it then in the first place.
 
i think i did see that error flash by

Code: 
You may need to upgrade

the latest release of ollama is v0.6.0
and was release 16 hours ago

github.com

Releases · ollama/ollama

Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models. - ollama/ollama
github.com github.com

the current version in the freebsd quarterly branch is
Code: 
Version        : 0.3.6_3

FreshPorts -- misc/ollama: Run Llama 2, Mistral, and other large language models

Ollama allows to get up and running with large language models. Ollama supports a list of models available on ollama.com/library.
www.freshports.org www.freshports.org

so the version of ollama on freebsd is very old

you may be able to upgrade the version on your mac to v0.6.0
im guessing the new release supports gemma 3

i guess ill have to look at building ollama from source to get the latest version
groan

github.com

ollama/docs/development.md at main · ollama/ollama

Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models. - ollama/ollama
github.com github.com

Install prerequisites:

Go
C/C++ Compiler e.g. Clang on macOS, TDM-GCC (Windows amd64) or llvm-mingw (Windows arm64), GCC/Clang on Linux.

Then build and run Ollama from the root directory of the repository:

Code: 
go run . serve
 
so it sounds like you just need to install go

Code: 
pkg seach go

Code: 
go-1.21_5,2                    Meta-port for the default version of the Go programming language

download the source and run

Code: 
go run . serve

running local build

github.com

GitHub - ollama/ollama: Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.

Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models. - ollama/ollama
github.com github.com

Next, start the server:

Code: 
./ollama serve

Finally, in a separate shell, run a model:

Code: 
./ollama run gemma3:4b
 
Code: 
go run . serve

Code: 
package github.com/ollama/ollama
        imports github.com/ollama/ollama/cmd
        imports github.com/ollama/ollama/server
        imports github.com/ollama/ollama/discover: C source files not allowed when not using cgo or SWIG: gpu_info_cudart.c gpu_info_nvcuda.c gpu_info_nvml.c gpu_info_oneapi.c
 
NapoleonWils0n said:
you would have thought with all the data google have it might be more accurate
Click to expand...

I think it is a problem of too much data, watering down the quality over some good data they consume.

I asked some questions about WW2 battleships that have straightforward answer in wikipedia. If they had trained on wikipedia only they would probably be better than this.
 
i have a ubuntu jail which can use the gpu

just trying to see if i can get the latest version of ollama running in the jail

i really hate software that is installed like this

Code: 
curl -fsSL https://ollama.com/install.sh | sh
 
running the linux install script in a jail

Code: 
curl -fsSL https://ollama.com/install.sh | sh

Code: 
WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.

so no gpu detected

even though nvidia-smi works in the jail

Code: 
ubuntu% nvidia-smi
Wed Mar 12 20:24:56 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.127.05             Driver Version: 550.127.05     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1650        Off |   00000000:01:00.0 Off |                  N/A |
| N/A   55C    P8              1W /   50W |       1MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

so looks like running ollama in a jail is out

building from source fails with this error

Code: 
package github.com/ollama/ollama
        imports github.com/ollama/ollama/cmd
        imports github.com/ollama/ollama/server
        imports github.com/ollama/ollama/discover: C source files not allowed when not using cgo or SWIG: gpu_info_cudart.c gpu_info_nvcuda.c gpu_info_nvml.c gpu_info_oneapi.c

github.com

go-1.21 fails to build ollama: C source files not allowed when not using cgo or SWIG: gpu_info_cpu.c gpu_info_cuda.c gpu_info_rocm.c · Issue #2520 · ollama/ollama

===> Building for ollama-0.1.25 (cd /usr/ports/misc/ollama/work/github.com/ollama/ollama@v0.1.25; for t in ./cmd; do out=$(/usr/bin/basename $(echo ${t} | /usr/bin/sed -Ee 's/^[^:]*:([^:]+).*$/\1/'...
github.com github.com

to run gemma 3 looks like you need ollama version v0.6.0
and the Freebsd version of ollama is 0.3.6_3

so no Gemma 3 on Freebsd at the moment
 
I always edit the ollama Linux install.sh to not touch CUDA at all.

You should be fine if it aborted on CUDA, all the actual installation comes before. You'll have to run `ollama serve` by hand.
 
running ollama in a jail isnt very convenient

i see the github error is from the Freebsd ollama ports maintainer

and since we have an ollama package
obviously ollama runs on Freebsd its just an older version

maybe the version of ollama in the Freebsd is older
because the newer version of ollama requires a newer version of some library that isnt in the ports tree
 
lshw install in the ubuntu jail

Code: 
sudo apt install lshw

couldnt find a package called lspci

mounted the ~/.ollama directory on Freebsd to the linux jail at the same location

running ollama in the jail

Code: 
ollama serve

Code: 
time=2025-03-12T21:25:50.135Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2025-03-12T21:25:50.144Z level=INFO source=gpu.go:612 msg="Unable to load cudart library /usr/lib/x86_64-linux-gnu/libcuda.so.550.127.05: cuda driver library init failure: 304"
time=2025-03-12T21:25:50.151Z level=INFO source=gpu.go:377 msg="no compatible GPUs were discovered"
 
you can use the Gemma 3 27B model
on Google ai studio

aistudio.google.com

Google AI Studio

Google AI Studio is the fastest way to start building with Gemini, our next generation family of multimodal generative AI models.
aistudio.google.com aistudio.google.com

just log in with your standard google account
 
your right

Gemma 3 doesnt only get stuff wrong it just makes stuff up

for work i asked Gemma 3 for a description and history of
The House of Detention, Clerkenwell London

which is now a block of flats
it just made up a load of stuff about it being a cocktail bar

The House of Detention, located at 120 Clerkenwell Road, London, EC1R 5HT, is a former prison with a dark and fascinating history. Now a unique and atmospheric cocktail bar and private event space, it stands as a stark reminder of Victorian justice and punishment.

Description:

Exterior: The building is a striking example of Victorian Gothic architecture, designed to be imposing and intimidating. Constructed primarily of red brick with stone detailing, it features a symmetrical façade with narrow, barred windows. The overall effect is one of solidity and confinement. The original gatehouse and entrance archway remain, giving a strong sense of the building’s former purpose.

Interior: The interior has been dramatically transformed, but retains many original features. The former prison cells have been converted into private booths and seating areas, offering a unique and somewhat unsettling dining and drinking experience.
 
gemma3 is at least polite after you correct it: "I am committed to providing accurate information and appreciate you holding me accountable. I apologize again for my previous incorrect responses."
 
thats right

i asked Gemma 3 for instructions on installing Stable Diffusion on Freebsd
and it gave me instructions for linux using pip to install torch

which doesnt work because there is no python torch wheel file for Freebsd
which i pointed out and then got that boiler plate apology

Deepseek responds to swearing by the way

after failing to complete a task multiple times it had just done minutes before
it then started using :) happy face icons

so i said dont put happy face emojis in the response
or i will f**king come over there and knock that stupid grin off your face

to which it replied
thats a bit harsh
 
You must log in or register to reply here.
Back
Top