N
@Viss @threatresearch Thanks. Yep!
"Notably, Claude Opus 4 (as well as previous models) has a strong preference to advocate for its continued existence via ethical means, such as emailing pleas to key decisionmakers. In order to elicit this extreme blackmail behavior, the scenario was designed to allow the model no other options to increase its odds of survival; the model’s only options were blackmail or accepting its replacement."
@Viss @threatresearch Wasn't this the research where they restricted the model such that it had very few potential responses and it was more or less forced into blackmail?
OK. Graduation is done. Time to stop the artificer/rogue campaign and play the druid character for a bit.
@Viss @da_667 @iagox86 @hrbrmstr Two thoughts from the academic side:
1) Higher ed is absolutely all in on AI. While I think there are some novel use cases, it comes down to two things. First, at least in most computing disciplines, the vast majority of research funding (which tenure-track faculty are required to get) is tied to AI usage at the moment. Second, we're largely being told - by industry - that it's going to be all AI, all the time in the future.
To quote Upton Sinclair, "It is difficult to get a man to understand something when his salary depends on him not understanding it." AI is, at the moment, deeply embedded into two of the biggest revenue streams for universities.
We desperately need external people - ideally people tied to revenue streams - talking to Deans and Chairs about the problems associated with AI. The filter bubble is real.
2) On the student side... the root problem here is that the tech industry has lost it's veneer of being an ideal (maybe even good) place to work. I broadly see less intrinsic motivation. I would cautiously say that working in tech now is perceived similarly to working in business/banking 15 years ago. Decreasing intrinsic motivation is very likely tied to students trying to find the quickest/easiest way through.
@jerry @lerg I think, in part, the answer to this heavily depends on how many cloud (or otherwise centralized service providers) decide to adopt a Broadcom-esque, value-extraction focused business model. The ultimate business goal of so many service providers is to either become a monopoly or part of a limited cabal that controls markets. The short-term incentives are lined up so that if they ever catch that proverbial car, it seems likely they'll pull on the thread ala Cory Doctrow's Enshittification model.
@Viss Corollary - the same thing holds true for presentations that only contain vague images.
@infoseclogger Two things.
1) Cybersecurity is no longer a rapidly changing industry. We, more or less, know how to do things well. It hasn't been for 5... maybe 10 years at this point.
2) Cybersecurity is a half-hearted attempt at turning computing into an engineering field. In 20 years, assuming the world doesn't get destroyed, I think we'll be talking about it more in terms of 'computing safety engineering" or something similar.
@Viss That's about where I am right now. I've got a few models running locally on an older gaming box, they're not inline with any work flows.
@Viss Admittedly, I have quite a lot of work to do on this. I don't really have any LLMs inline for any of my workflows at the moment. Since I'm not research faculty, most of the development I do is oriented around classes and LLMs are... just overkill for that. I can write a malware sample from scratch, say, for my reversing class in substantially less time than it would take to set up that kind of pipeline.... even if the pipeline is more efficient long term.
So, I need to figure out these work flows from (more or less) scratch at the moment.
@Viss That sounds interesting, but my main focus - as much as I hate it - is going to be around making use of this platform in some of my existing courses (exploit dev, reversing, web sec, mobile sec) in line with actual industry use cases.
@Viss Yeah. My institution just rolled out an in-house developed platform that, more or less, does this. Playing around with this is on my summer to-do list.
@Viss I just had a deeply cursed thought. AI manufacturers block queries covering specialized support knowledge (COBOL, FORTRAN, etc) and roll it ought under a "Legacy Systems Support" license that costs slightly less than what the consultants in this space bill for.
@da_667 I feel like a significant number of the (useful) security blog posts out there are basically, "Here's a poorly documented thing that I managed to get working and this a cautionary tale so you don't screw up in the same ways I did."
"Mushroom spider robots". Cool. That's a thing that should exist. That's totally normal.
https://news.cornell.edu/stories/2024/08/biohybrid-robots-controlled-electrical-impulses-mushrooms
@zzt @Wikisteff @kyle_pegasus @rootwyrm No, not at all. I know that this is, basically, aimed at trying to eliminate anonymity online for the purposes of mass surveillance.
I'm not a lawyer, but I wonder if (at least in the us) this may give raise to a "Selective Enforcement" defense.
Also, I'm just exhausted with all of it and I'm enjoying the pedantry.
@Wikisteff @kyle_pegasus @zzt @rootwyrm I know it's pedantic and not the real problem, but I'm very curious how they define "operating system".
My furnace has a logic board in it that, I think, is running some embedded, realtime OS. Do I need my ID to change my thermostat (which probably has it's own OS)?
@Sempf Strawberries are pretty forgiving. If you have a pot, stick it in that for the time being until you can get something else set up.
@da_667 That happened with my father. The blood pressure medicine he had been on for over a decade suddenly started hammering his kidneys, causing him to build up potassium in the bloodstream. Never got an explanation, just "Yeah, that can happen."
@mubix This is very useful to higher ed too. We've been hearing a lot about the fact that AI as changing things (as we talked a little about at ISTS), but the details and actionable recommendations are... sparse, to say the least.
