The #OpenClaw and #Ollama local #AI #agent combo is working fairly well.
-
The #OpenClaw and #Ollama local #AI #agent combo is working fairly well. The setup is an absolute nightmare, but I won after many hours of tweaking stuff. Running the #GPT-OSS:20B model with 32k context window on a 7900XTX. The OpenClaw install is in a VirtualBox VM running Linux, running on a Windows 10 host with a 7950X and 128GB of DDR5. The OLLAMA is running on the bare metal.
Responses take about a minute, give or take.
@Lydie What are you using it for?
-
The #OpenClaw and #Ollama local #AI #agent combo is working fairly well. The setup is an absolute nightmare, but I won after many hours of tweaking stuff. Running the #GPT-OSS:20B model with 32k context window on a 7900XTX. The OpenClaw install is in a VirtualBox VM running Linux, running on a Windows 10 host with a 7950X and 128GB of DDR5. The OLLAMA is running on the bare metal.
Responses take about a minute, give or take.
@Lydie
Why win10? Everything on Linux seems more straightforward. -
"The setup is an absolute nightmare"... true! I gave up on it
-
The #OpenClaw and #Ollama local #AI #agent combo is working fairly well. The setup is an absolute nightmare, but I won after many hours of tweaking stuff. Running the #GPT-OSS:20B model with 32k context window on a 7900XTX. The OpenClaw install is in a VirtualBox VM running Linux, running on a Windows 10 host with a 7950X and 128GB of DDR5. The OLLAMA is running on the bare metal.
Responses take about a minute, give or take.
@Lydie Did you try other models, like gemma3:32b?
-
@Lydie Did you try other models, like gemma3:32b?
@maikm I tried QWEN3.5 and it struggled - very slow, seemed to overflow to sysram. 20b seems a good model size to fit the context window.
-
@Lydie
Why win10? Everything on Linux seems more straightforward.@bitzero See my profile for a note on that...
-
"The setup is an absolute nightmare"... true! I gave up on it
@sheepfreak I almost did. Needless to say, I made some solid backups!
-
@bitzero See my profile for a note on that...
@Lydie
Ah ok. Got it. -
@maikm I tried QWEN3.5 and it struggled - very slow, seemed to overflow to sysram. 20b seems a good model size to fit the context window.
@Lydie I've used those before too (and also deepseek-r1:70b) and settled on gemma3:27b (sorry, not 20b) for it's nice balance of speed and quality.
I run these on a MacBook Pro M1Max with 64 GB, which supports up to 48GB models, don't know if the 27b will fit in your GPU.
-
@Lydie I've used those before too (and also deepseek-r1:70b) and settled on gemma3:27b (sorry, not 20b) for it's nice balance of speed and quality.
I run these on a MacBook Pro M1Max with 64 GB, which supports up to 48GB models, don't know if the 27b will fit in your GPU.
@maikm I have a Strix Halo tablet that can do a similar trick, I should give it a go. The thought of using a tablet as a remote LLM host is
-
@Lydie What are you using it for?
@davep Eventually, to automate monotonous daily tasks for work. E.g. collecting the latest daily NWS weather forecasts and summarizing them hyper-locally to deliver to my field colleagues.
-
@davep Eventually, to automate monotonous daily tasks for work. E.g. collecting the latest daily NWS weather forecasts and summarizing them hyper-locally to deliver to my field colleagues.
-
R relay@relay.infosec.exchange shared this topic
