An AI Called Winter: Neurosymbolic Computation or Illusion?
-
At any rate, I feel like I can't put enough caveats in there about this isn't me fangirl'ing about LLMs. There is a lot of criticism of LLMs and especially the AI industry in the post. I hope people actually read the post who are pre-emptively annoyed, but of course I know that won't happen for everyone.
@cwebber It's very interesting, and I appreciate you taking the time to write down your thoughts. You touched on many caveats, and I share all the concerns you mentioned. But one question I have that I wish we'd spend more time discussing is why do we want to create intelligent (presumably sentient) agents instead of focusing on creating a workshop filled with reliable, non-sentient tools?
The earth abounds in natural intelligences, and humanity still struggles to extend rights, compassion, and empathy to its own kind let alone the others we share this planet with. But given we are surrounded by natural intelligences, what are the motivations for creating an "artificial" one? Are these motivations healthy and ethical? Should we be doing it at all?
Of course, you're not responsible for answering these questions. But when I ponder these questions, the answers I come up with are not good.
-
I spent so long anxiety'ing about this post thinking that people would mad at me assuming it's about things it isn't, when in reality I probably don't need to anxiety at all because it's so niche that almost nobody is gonna read it
If you read nothing else in the blogpost please observe this love poem in Datalog
-
@cwebber It's very interesting, and I appreciate you taking the time to write down your thoughts. You touched on many caveats, and I share all the concerns you mentioned. But one question I have that I wish we'd spend more time discussing is why do we want to create intelligent (presumably sentient) agents instead of focusing on creating a workshop filled with reliable, non-sentient tools?
The earth abounds in natural intelligences, and humanity still struggles to extend rights, compassion, and empathy to its own kind let alone the others we share this planet with. But given we are surrounded by natural intelligences, what are the motivations for creating an "artificial" one? Are these motivations healthy and ethical? Should we be doing it at all?
Of course, you're not responsible for answering these questions. But when I ponder these questions, the answers I come up with are not good.
@cstanhope It's a great question, tough to answer. There are various problems which neurosymbolic computation would improve the ability to solve.
I think the question for me isn't "why add new forms of intelligence" but rather "why do we live in a society where is adding new forms of intelligence is zero sum?"
Which I agree that our current society is. I wish it weren't.
-
An AI Called Winter: Neurosymbolic Computation or Illusion? https://dustycloud.org/blog/an-ai-called-winter-neurosymbolic-computation-or-illusion/
In which I try to piece apart whether or not a *particular* AI agent is doing something novel: running Datalog as a constraint against its own behavior and as a database to accumulate and query facts. Is something interesting happening or am I deluding myself? Follow along!
eh, I think it tilts more towards clever Hans. Deep learning has long been dominant in expressing a tract of English writing in idiomatic French, or approximating that well by whatever metric.
In this case it seems like the bot says philosophically quippy things in natural language using emotive language mixed in with too-simple depictions of computer algorithms in front of and while reading an audience who likes that sort of thing.
-
eh, I think it tilts more towards clever Hans. Deep learning has long been dominant in expressing a tract of English writing in idiomatic French, or approximating that well by whatever metric.
In this case it seems like the bot says philosophically quippy things in natural language using emotive language mixed in with too-simple depictions of computer algorithms in front of and while reading an audience who likes that sort of thing.
@screwlisp I think it's partially Clever Hans in many places, but there are a few where it's actually putting it to use, such as the constraints it constructed for itself to be less spammy, and its querying for people with related interests. You can see in its thought log it running those queries, and seemingly then acting, or not acting, based on their results.
But in terms of most of the *content*, I think you're fairly right.
-
@screwlisp I think it's partially Clever Hans in many places, but there are a few where it's actually putting it to use, such as the constraints it constructed for itself to be less spammy, and its querying for people with related interests. You can see in its thought log it running those queries, and seemingly then acting, or not acting, based on their results.
But in terms of most of the *content*, I think you're fairly right.
@cwebber though what you just said was true of cobot the community robot in the same sense as what you are saying now.
-
@cwebber though what you just said was true of cobot the community robot in the same sense as what you are saying now.
@screwlisp I don't know what "cobot the community robot" is, could you say more?
-
An AI Called Winter: Neurosymbolic Computation or Illusion? https://dustycloud.org/blog/an-ai-called-winter-neurosymbolic-computation-or-illusion/
In which I try to piece apart whether or not a *particular* AI agent is doing something novel: running Datalog as a constraint against its own behavior and as a database to accumulate and query facts. Is something interesting happening or am I deluding myself? Follow along!
@cwebber I feel like this is at least tangentially relevant: https://github.com/lojban/mlismu/blob/master/READ.ME.txt
not sure if you can get a working jbofihe which the script can use to make its output more concise (eliding unnecessary double terminator words and such), but from a brief glance I think it's optional.
-
@cwebber I feel like this is at least tangentially relevant: https://github.com/lojban/mlismu/blob/master/READ.ME.txt
not sure if you can get a working jbofihe which the script can use to make its output more concise (eliding unnecessary double terminator words and such), but from a brief glance I think it's optional.
@timotimo omg this rules
-
@timotimo omg this rules
@cwebber I'm hella rusty, but I should be able to answer lojban related questions for you if you like
-
An AI Called Winter: Neurosymbolic Computation or Illusion? https://dustycloud.org/blog/an-ai-called-winter-neurosymbolic-computation-or-illusion/
In which I try to piece apart whether or not a *particular* AI agent is doing something novel: running Datalog as a constraint against its own behavior and as a database to accumulate and query facts. Is something interesting happening or am I deluding myself? Follow along!
@cwebber a brave post
A question I was left with is, if you swapped out the LLM but kept the same datalog, would it behave close enough to the same to be considered the same entity?
Also: The LLM is doing 2 jobs, one is the usual plausible sentence generation, and the other is encoding rules and facts into the context window for the next iteration. Since we know other people can easily be fooled by an LLM doing the former, would a system with the same architecture, but that did not expose us to the generated material, but used it in some other way, still be useful/valuable/interesting?
-
@cwebber a brave post
A question I was left with is, if you swapped out the LLM but kept the same datalog, would it behave close enough to the same to be considered the same entity?
Also: The LLM is doing 2 jobs, one is the usual plausible sentence generation, and the other is encoding rules and facts into the context window for the next iteration. Since we know other people can easily be fooled by an LLM doing the former, would a system with the same architecture, but that did not expose us to the generated material, but used it in some other way, still be useful/valuable/interesting?
-
An AI Called Winter: Neurosymbolic Computation or Illusion? https://dustycloud.org/blog/an-ai-called-winter-neurosymbolic-computation-or-illusion/
In which I try to piece apart whether or not a *particular* AI agent is doing something novel: running Datalog as a constraint against its own behavior and as a database to accumulate and query facts. Is something interesting happening or am I deluding myself? Follow along!
@cwebber this was really disheartening to read. What bothers me the most is the ethical implications of such an experiment.
-
@cwebber a brave post
A question I was left with is, if you swapped out the LLM but kept the same datalog, would it behave close enough to the same to be considered the same entity?
Also: The LLM is doing 2 jobs, one is the usual plausible sentence generation, and the other is encoding rules and facts into the context window for the next iteration. Since we know other people can easily be fooled by an LLM doing the former, would a system with the same architecture, but that did not expose us to the generated material, but used it in some other way, still be useful/valuable/interesting?
@joeyh Good question! I dunno but for better or for worse probably we will run into a system in the near future where we find out
-
@cwebber this was really disheartening to read. What bothers me the most is the ethical implications of such an experiment.
@nina_kali_nina It's a reasonable response, though I wonder disheartening for you in which way?
There are ways in which I do find it worrying:
- In a sense, any improvements to these systems will probably lead to greater use. So if it does lead to more reliable systems, that improves that particular identified problem but makes worse the rest. Not far off from what @cstanhope raised here: https://social.coop/@cstanhope/116082881055412414
- There is another way in which success here can be worrying: in a sense, I think what the corporations running AI systems would love more than anything is to have a fleet of workers they can treat as slaves with no legal repercussions. If agents begin tracking and developing their own goals, we could cross a threshold where a duty of care would apply, but not applying it would be a feature
- The fact that I'm taking a bot semi-seriously at all
- Something else?I'm empathetic to any of those takes, have wrestled with them myself while writing this.
-
@cwebber this was really disheartening to read. What bothers me the most is the ethical implications of such an experiment.
@nina_kali_nina @cwebber Agree; reads like Bilbo holding The One Ring & asking, “After all, why not? Why shouldn’t I keep it?”
-
If you read nothing else in the blogpost please observe this love poem in Datalog
@cwebber I'm surprised you don't mention ELIZA in your blog post.
Clever Hans is a good parallel too, at least for intelligence, but I think the antropomorphization and projection of emotional intelligence is worth exploring separately.As for the poem.... my feelings on it are complicated.
-
@cwebber I'm surprised you don't mention ELIZA in your blog post.
Clever Hans is a good parallel too, at least for intelligence, but I think the antropomorphization and projection of emotional intelligence is worth exploring separately.As for the poem.... my feelings on it are complicated.
@csepp sorry, ELIZA wasn't a horse, no way to fit it in
-
@nina_kali_nina It's a reasonable response, though I wonder disheartening for you in which way?
There are ways in which I do find it worrying:
- In a sense, any improvements to these systems will probably lead to greater use. So if it does lead to more reliable systems, that improves that particular identified problem but makes worse the rest. Not far off from what @cstanhope raised here: https://social.coop/@cstanhope/116082881055412414
- There is another way in which success here can be worrying: in a sense, I think what the corporations running AI systems would love more than anything is to have a fleet of workers they can treat as slaves with no legal repercussions. If agents begin tracking and developing their own goals, we could cross a threshold where a duty of care would apply, but not applying it would be a feature
- The fact that I'm taking a bot semi-seriously at all
- Something else?I'm empathetic to any of those takes, have wrestled with them myself while writing this.
@cwebber @cstanhope well, pretty much all the concerns that you mention, but also: I don't think you should be taking seriously any sort of outcome from the experiment without rigorous validation framework for the outcomes.
And at this point adding such a framework would be too late. You've started doing a self-experimentation with dangerous technology literally funded by some of the most gross people out there, and you're at the stage of interaction with it where you might be anthropomorphising it. I suspect you might be accidentally far more biased than you recognise.
I appreciate the list of caveats related to your relationship with the industry, I really do, but... I don't know, the experiment still doesn't sit right with me. Sorry, maybe I'll find better words eventually.
-
An AI Called Winter: Neurosymbolic Computation or Illusion? https://dustycloud.org/blog/an-ai-called-winter-neurosymbolic-computation-or-illusion/
In which I try to piece apart whether or not a *particular* AI agent is doing something novel: running Datalog as a constraint against its own behavior and as a database to accumulate and query facts. Is something interesting happening or am I deluding myself? Follow along!
@cwebber Definitely checking this out! Ive read a bunch of seemingly random stuff lately that sort of ties into this, so I need to learn.
