The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again.
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye My core cynical view from the trajectory of 'DevOps' since then is that developers don't want to be on the hook that way. I actually can't blame them because I'm not sure management rewarded them for it. If management demands shipped features you get shipped features and damn the torpedoes, full speed ahead.
-
@mhoye My core cynical view from the trajectory of 'DevOps' since then is that developers don't want to be on the hook that way. I actually can't blame them because I'm not sure management rewarded them for it. If management demands shipped features you get shipped features and damn the torpedoes, full speed ahead.
@cks My slightly adjacent cynical view is that as soon as the realization that code had consequences really took root, it basically served as an incentive marketing campaign for SAAS based solutions that let you offload those risks to companies and "devops" rapidly became "contract management in a terminal window".
@owen had some related thoughts about this but a quick search didn't find them.
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye
Some thoughts:
Heston Blumenthal said he could always tell if a chef had eaten at his own restaurant by the butter - if it was cold, and didn't spread, they clearly had never experienced being at the table themselves.Railway and civil engineering however is much closer to the large "team effort" of software. They don't need to "live" their own consequences by attending to every derailment, because a sensible proxy has been set up - professional chartering, striking people off, criminal charges, disciplinary action.
Right now, software does not have this concept. But prior to them having this setup, old bridge builders were invited to stand under it to "prove" it worked.
-
@cks My slightly adjacent cynical view is that as soon as the realization that code had consequences really took root, it basically served as an incentive marketing campaign for SAAS based solutions that let you offload those risks to companies and "devops" rapidly became "contract management in a terminal window".
@owen had some related thoughts about this but a quick search didn't find them.
@mhoye @cks I do wish I remembered where I picked up that framing; it's not mine, but it rung so loudly through me I've had trouble shutting up about it.
However, I might disagree that, as practiced, modern devops has a damned thing to do with what _developers_ (or, for that matter, the few remaining ops folk) want. Most devops implementations I've seen are firmly top-down exercises, facilitated by a few true believers but largely funded by the host organization.
-
@mhoye @cks I do wish I remembered where I picked up that framing; it's not mine, but it rung so loudly through me I've had trouble shutting up about it.
However, I might disagree that, as practiced, modern devops has a damned thing to do with what _developers_ (or, for that matter, the few remaining ops folk) want. Most devops implementations I've seen are firmly top-down exercises, facilitated by a few true believers but largely funded by the host organization.
@mhoye @cks The mean "devops transformation" reallocates work without removing any, and reduces headcount. _Devops teams_, where they exist, are almost universally in a lower total team comp band than the ops team whose work they inherited, and given how underpaid a lot of ops teams were to start with, that's saying something.
That's not to say that individual devops engineers are necessarily underpaid; I've known some very well compensated folks in that space.
Who work, functionally, alone.
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye My cynical take is that most enterprise organizations and their managements don’t really feel the pressure needed to justify the effort to move over to real DevOps.
DevOps was meant to solve the issue of delivering fast AND with high quality.
In practice, management seems happy with SAFe running release trains on cloud platform of choice and delivering larger features every half year to a year.
Most teams are 3 levels removed from anything resembling production or user feedback.
-
@arichtman @benjamingeer @mhoye unfortunately AWS IAM is genuinely the only thing which solves the very hard problem, of “I want near-Turing-complete permissions management schemes to enforce almost any possible rule set”
GCP tags are a joke, for example
-
@mhoye @cks The mean "devops transformation" reallocates work without removing any, and reduces headcount. _Devops teams_, where they exist, are almost universally in a lower total team comp band than the ops team whose work they inherited, and given how underpaid a lot of ops teams were to start with, that's saying something.
That's not to say that individual devops engineers are necessarily underpaid; I've known some very well compensated folks in that space.
Who work, functionally, alone.
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye
I hear the trope of "getting up in the middle of the night to fix an incident" quite often.
I wonder how often this actually happens. In my experience, most incidents occur during working hours.
If something really does break at night, the question arises: what is different?
Do the systems have to be actively kept alive during working hours --> not good.
Are there any bulk cron jobs running at night that endanger the system --> also not good, it's better to run these during the day. -
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye @dymaxion yeah, because I let the petty job I do to pay my landlord and my bills dictate every last bit of my life. God forbid this silly shopping platform isn’t performing at 100% 24/7.
Most things we’re made to do aren’t worth doing to begin with. And especially not worth sacrificing that little free time I’ve got left at my disposal.
It’s just a job!
-
@mhoye @dymaxion yeah, because I let the petty job I do to pay my landlord and my bills dictate every last bit of my life. God forbid this silly shopping platform isn’t performing at 100% 24/7.
Most things we’re made to do aren’t worth doing to begin with. And especially not worth sacrificing that little free time I’ve got left at my disposal.
It’s just a job!
-
Second, like others have pointed out in this thread, a lot of what results in „shit work“ is out of my control. I don’t get to make the decisions of what is to be implemented nor the timeframe nor the tools or technology to be used to reach that goal.
Nor was it my decision to get rid of the QA department.
-
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye Tony Hoare called this “the sword of Damocles method of ensuring programme correctness” and cited an example of a team of programmers who had to calculate strengths of the chains used to slow a ship during a (sideways) launch. The team knew that they would be in a stand on the other side of the river, so would drown if the ship went in too fast.
-
The core insight of DevOps, at the very beginning, was that when people need to carry the pager for the code they write, they write code that won’t wake them up at Oh My God, What The Fuck O’Clock after it falls over in production again. Everything since - DORA, all of it - has been in service of this one idea that aligning software discipline with quality of life consequences makes better software.
It’s an idea that should be everywhere and in everything.
@mhoye This is how I ended up doing devops before it was a thing. Small companies with leadership that understood that if I have to be on call, I get a say in how things are built. Respect the people that do the work. I was an admin on call, catching taxi at 2am to go reboot solaris x86 boxes at a colo.
Later, in large companies with an operations team in a different department, as a lead we made sure that we took personal responsibility for any disruption to the operators on call — including exchanging personal phone numbers and betting on call formally or not, depending on circumstance. Focusing on observability and deployment speed needed to respond and fix (live debugging and patching production via a CL REPL in some cases).
Even when we controlled the tech, we had periods where a database or storage system was having chronic problems that took weeks to resolve. This taught us to favor simplicity over everything, and to truly understand our resource and state management. We called it stupid, not simple, because it avoided the common hackers conceit that clever is simple.
I am sure that this personal responsibility is abused in dysfunctional organizations, but that doesn’t reflect on the origins of the practices, only what happens when they become popular and adapted as ritual or control in organizations that skipped the foundation. Similar to extreme or agile development.
Respect the people who do the work.
Keep it stupid, stupid.
-
R relay@relay.infosec.exchange shared this topic
