<![CDATA[gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU.]]>gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU.

Q8_0 (~8GB), 17 tokens/secs. real nice.

i need to read google's paper on this and the novel compression method they used.

maybe these new datacenters can eventually go fuck themselves after all

]]>https://board.circlewithadot.net/topic/40d595ae-71d7-476f-af51-32dfc0cff200/gemma-4-e4b-isn-t-half-shabby-but-i-didn-t-think-it-would-run-in-llama.cpp-vulkan-in-ubuntu-on-this-lenovo-yoga-laptop-with-an-amd-radeon-860m-gpu.RSS for NodeFri, 01 May 2026 03:55:14 GMTTue, 14 Apr 2026 22:22:44 GMT60<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 10:46:58 GMT]]>okay. there is an experimental llama.cpp flag "--webui-mcp-proxy", not to be used in production, which puts a proxy in front so the CORS interface (?) that the browser needs is available.

you also have to check "use llama-server proxy" in the webui, and then it connects properly.

Link Preview Image
]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413983517813789https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413983517813789Thu, 16 Apr 2026 10:46:58 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 10:36:42 GMT]]>aha. the reason why the llama.cpp webfrontend can't reach the MCP server seems to be that for reasons beyond me, the communication with the server is attempted in the browser client rather than in the backend, and neither firefox nor chrome seem to support this out of the box (?)

Link Preview Image
Does not work with browser based MCP clients due to lack of CORS support · Issue #840 · PrefectHQ/fastmcp

Description Does not work with browser based MCP clients due to lack of CORS support Example Code Version Information Should allow for CORS header config to support browser clients Additional Context No response

favicon

GitHub (github.com)

according to this, you have to install something called "starlette" and then modify the mcp server so it works and i am already exhausted by this fuck shit stack.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413943145739866https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413943145739866Thu, 16 Apr 2026 10:36:42 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 10:01:46 GMT]]>@neo you're not wrong.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413805773812442https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413805773812442Thu, 16 Apr 2026 10:01:46 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 09:37:07 GMT]]>@lritter It's called a "job". Some people have to do that to earn money so they can pay their bills. 😉

Also it's not that bad. Actually shipping something can be quite satisfying. You should try it one day. 😜

]]>https://board.circlewithadot.net/post/https://soc.psynet.me/users/neo/statuses/116413708831672971https://board.circlewithadot.net/post/https://soc.psynet.me/users/neo/statuses/116413708831672971Thu, 16 Apr 2026 09:37:07 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 06:54:00 GMT]]>@neo *looking at the list* fascinating that the question "what am i doing with my life" is not coming up once for you - with or without bots.

i guess i could do this for money. in a "row row row your boat" kind of way. i would have to be careful to pretend to take this seriously and not see it as the cube-esque dream logic that it is.

i mean it's one thing when life itself is this way; that can't be helped. but living inside a sentient bureaucratic monster takes new levels of acceptance.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413067464936782https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413067464936782Thu, 16 Apr 2026 06:54:00 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 06:38:01 GMT]]>@dunkelstern all "knitted with a hot needle" as we say around here

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413004631035285https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116413004631035285Thu, 16 Apr 2026 06:38:01 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Thu, 16 Apr 2026 06:35:04 GMT]]>@neo what can i say. the mcp runs, the port is 8000, the protocol is http, llama.cpp connects to http://localhost:8000 but then complains about errors in the protocol itself. i haven't even gotten to taskIng the model yet.

i'll try again with the docs here. there are details the other example didn't have.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116412993035052818https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116412993035052818Thu, 16 Apr 2026 06:35:04 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 23:32:16 GMT]]>@lritter Hehe. 😄 Here is my fleet of MCP servers. Six of them with about 200 tools in total I vibecoded over a couple of days using https://github.com/modelcontextprotocol/python-sdk. 🙈

My current challenge is to tune the tool descriptions in a way to nudge agents into using the correct tools more intuitively (since I like to write rather abstract queries that often require a combination of tools from different MCPs to be used). Weird way of software development, but pretty much how imagined the future to be. 😂

Link Preview Image
]]>https://board.circlewithadot.net/post/https://soc.psynet.me/users/neo/statuses/116411330467472135https://board.circlewithadot.net/post/https://soc.psynet.me/users/neo/statuses/116411330467472135Wed, 15 Apr 2026 23:32:16 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 23:25:08 GMT]]>@lritter that’s absolutely the same thing i am feeling on this. I have been tasked at work to build a copilot/teams “Agent”. The official SDK from microsoft is version “alpha 50”, documentation is completely wrong and the “usecase” is built on hope and prayer and does not work in 60% of all cases. Copilot apparently does so much “Reasoning” in the background it burns through the token context window in no time and then starts to do weird things. Tool calls work in 80% of the cases but in the 20% they do not work it does hilarious things and wrecks the workflow completely. Doesn’t help that copilot has at least 4 ways to build “Agents” where 1 is basically just a prompt, 1 does not work and 2 need copilot pro subscriptions and ms developer accounts which cost you an arm and a leg.

I completely do not understand why anyone wants to use this or how this should replace workers…

]]>https://board.circlewithadot.net/post/https://corteximplant.com/users/dunkelstern/statuses/116411302477503260https://board.circlewithadot.net/post/https://corteximplant.com/users/dunkelstern/statuses/116411302477503260Wed, 15 Apr 2026 23:25:08 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 23:01:10 GMT]]>apparently MCP servers are now the replacement for openai tools protocol. you can sort of convert the existing scripts for it. wrote one, ran it with a thing called fastmcp (a cornucopia of 986175 dependencies), connected to it in llama.cpp: doesn't work. these cardboard & sharpie solutions are begining to annoy me.

i notice that most users seem to be happy *when* it works, never asking many questions about *how* it works which is how all these abysmal security failures happen.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116411208225541060https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116411208225541060Wed, 15 Apr 2026 23:01:10 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 20:32:07 GMT]]>@StompyRobot if you have a subscription, you go and let them write docs for my stuff. let's see if you're not exaggerating.

but no cheating!

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410622139808324https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410622139808324Wed, 15 Apr 2026 20:32:07 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 19:48:37 GMT]]>@StompyRobot i have no use for a technology that emulates the flaws of people

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410451091822896https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410451091822896Wed, 15 Apr 2026 19:48:37 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 19:40:34 GMT]]>@lritter for Gemma, no idea!
For the frontier models, yes, it can actually be good.
Think of it as an intern you can foist off certain tasks to when you specify them well, while you do higher thought work.

Creating docs and tutorials are a great test case actually -- I'd expect the good models to do above average-human quality on that.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410419428306612https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410419428306612Wed, 15 Apr 2026 19:40:34 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:54:32 GMT]]>@StompyRobot and this is supposed to be good?

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410238427896379https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410238427896379Wed, 15 Apr 2026 18:54:32 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:53:23 GMT]]>@lritter
What I'm saying is that that's not at all as certain as with people.
Or, to put another way, the prompt is a hash function into one of billions of possible programs stored in the model, and you'll get different bugs with a different prompt.
Getting the same model to work on the same problem in three different ways absolutely increases the rate of correctness, especially if you make a "best two of three" kind of setup.
It's really quite counter intuitive that it should work!

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410233881876970https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410233881876970Wed, 15 Apr 2026 18:53:23 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:43:05 GMT]]>@StompyRobot did you just lazily outsource your rebuttal to the machine? 😉

you know what i mean. if the machine makes mistakes generating, it will make mistakes verifying (whose output is also generation)

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410193357685824https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410193357685824Wed, 15 Apr 2026 18:43:05 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:40:01 GMT]]>@lritter
Models aren't conscious, don't have volition, and aren't trained to have self preservation behavior. They are surprisingly OK at diagnosing their own output when given specific instructions!

Programming them is a whole new way of thinking, but they *can* be made into a useful part of a useful system.

As you note, we're still being much in a "batteries not included" early stage, despite boosters claiming it's all done.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410181322244628https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410181322244628Wed, 15 Apr 2026 18:40:01 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:28:42 GMT]]>@StompyRobot but you see the problem in asking a politician to investigate their own dealings yes?

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410136823416968https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116410136823416968Wed, 15 Apr 2026 18:28:42 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 18:26:41 GMT]]>@lritter Gemma is a very small model.
Did you try asking opus to write a tutorial in the same repository?
And, because it's computers, then ask it to verify and correct itself?
(That's currently the state of the art in how to get useful stuff out. Why can't it do it automatically? IDK!)

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410128881044695https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/StompyRobot/statuses/116410128881044695Wed, 15 Apr 2026 18:26:41 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 17:14:18 GMT]]>@allo well it turns you into a bit of a CEO. so it would be logical that you get the same problems as one. which predicts an eventual coke habit 😉

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116409844299066740https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116409844299066740Wed, 15 Apr 2026 17:14:18 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 17:07:41 GMT]]>@lritter
No idea, butI think it is plausibel that doing more even with a tool is more stressful than doing less by hand. I think it was particularly about coding work.

]]>https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409818262643460https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409818262643460Wed, 15 Apr 2026 17:07:41 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 17:05:55 GMT]]>@lritter
Be the zero, its not affected by multipliers! 🙂

]]>https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409811312146068https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409811312146068Wed, 15 Apr 2026 17:05:55 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 17:04:42 GMT]]>@allo all this sounds more like mythbuilding to me than truth.

]]>https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116409806507773282https://board.circlewithadot.net/post/https://mastodon.gamedev.place/users/lritter/statuses/116409806507773282Wed, 15 Apr 2026 17:04:42 GMT<![CDATA[Reply to gemma 4 e4b isn't half shabby, but i didn't think it would run in llama.cpp-vulkan in ubuntu on this lenovo yoga laptop with an AMD Radeon 860M GPU. on Wed, 15 Apr 2026 17:04:17 GMT]]>@lritter
The AI assisted 10x engineer, I guess.

]]>https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409804878456943https://board.circlewithadot.net/post/https://chaos.social/users/allo/statuses/116409804878456943Wed, 15 Apr 2026 17:04:17 GMT