When it comes to your self-hosted services, what sort of attitude do you have when it comes to installing updates?

[zak@infosec.exchange]

When it comes to your self-hosted services, what sort of attitude do you have when it comes to installing updates? Do you install them right away? Wait a week or two? Update only when absolutely necessary? And why?

(I'm typically the sort of person that likes to be using the latest release of everything, but I'm open to opinions)

#selfhosted #FOSS #homelab

[vikki@know.me.uk]

@zak depends if it's a security related update I guess.

[rutrapio@piaille.fr]

@zak for non vital, I update them. For more important ones, I read the releases notes, even if only to see the new stuff.

As I check every two or three days, usually there IS already reviews.

[rubenwardy@hachyderm.io]

@zak

I have unattended updates on at the server level. Specific server software is on docker and so requires me to manually go upgrade, helps to avoid breakage

[woe2you@beige.party]

@zak When I get around to it.

[jwildeboer@social.wildeboer.net]

@zak I do updates every Friday and, in the case of security updates, right after them becoming available.

[unknownuniverse@unkn.uk]

@zak once a month I set aside a morning to run all my updates. It used to take me a whole day as my Proxmox host was a mess but now my system is much more streamlined I don't dread doing them like I used to!

[eingfoan@infosec.exchange]

@zak a view (2-3) days delay. Many reasons

Supply chain attack
Bad Updates
...

[daniel@colquitt.xyz]

@zak Security patches installed immediately on everything. For non-security related updates on critical stuff (email, routers, etc.), I usually wait at least a couple of weeks before installing them manually. Most other things are on auto updates.

[p4trick@chaos.social]

@zak Security -> ASAP

Minor -> I like to wait a few days so I don't deploy buggy/exploited releases, I take my time to properly review the changelog

Major -> I stay on LTS as long as possible so I have time to preare for major releases

Personally I value stability over features (unless I really them)

[firecat53@s.firecat53.net]

@zak On NixOS and have a service that updates all my desktops/laptops/homeservers daily. Rarely do I have any breakage. If so, a quick search most often finds the solution (either a config change, or a bug that already has a PR merged into nixpkgs). After running Arch for 10+ years and NixOS for 3+, I've come to appreciate more frequently updating as it tends to overall reduce the cognitive load of having to fix multiple issues all at once.

[mmeier@social.mei-home.net]

@zak I've got a two-pronged approach. I'm subscribed to release feeds for most apps running in my Homelab. If something is a security update, it gets updated immediately.

Otherwise, I've got a regular task to update all apps running in my cluster. I then sit down, go through my list of apps, look at new releases' notes and do the update manually. I quite enjoy that as a Friday evening activity.

Infrastructure, like k8s itself or Ceph, get updated less regularly.

[rachel@transitory.social]

@zak@infosec.exchange everything* is devoted via gitops and I have a renovate CronJob that runs a few times a day so I get emails on updates. A few services auto-update (the automerge step only happens after a few days delay, and never for major versions)

The cluster OS is Talos and I update it when I update k8s, or in theory if they had a critical security update (less likely due to small attack surface)

The small handful of misc services get updated less frequently but I'm not really worried about dnsmasq tbh

[rachel@transitory.social]

@mmeier@social.mei-home.net @zak@infosec.exchange do you manually check each application? Or have a way to track release notes for all of them?

[zak@infosec.exchange]

@rachel @mmeier I've done this myself manually just using my RSS service to subscribe to GitHub release pages. It works.

[rachel@transitory.social]

@zak@infosec.exchange @mmeier@social.mei-home.net I am subscribed to a small number of repos when I am also interested in pre-release/beta versions, but normally I just depend on renovate

recently I've been thinking that I should figure out method to check the age of each deployed image so I cam double-check that renovate is tracking everything correctly. A handful of times the project refactored and changed an image/chart name and the applied version ended behind by a few versions....

[viq@social.hackerspace.pl]

@rachel @mmeier @zak docker images have labels, maybe something got put in there?

[viq@social.hackerspace.pl]

@rachel @mmeier @zak when building custom images in work's CI, I used that to encode e.g. versions of various stuff inside, to make it easy to figure out without having to look inside the container.

[rachel@transitory.social]

@viq@social.hackerspace.pl @mmeier@social.mei-home.net @zak@infosec.exchange yeah but these are upstream images so I don't have that sort of control. Hmmmm, podman manifest inspect ... doesn't get me a creation date

Ah, but podman image history ... should do it! now to see how I can do that against every image in the cluster without needing to pull all of them

then any image over a certain age I'll doublecheck, ez ez

[cory@follow.coryd.dev]

@zak @faisal It depends on the experience I've had with updates in the past, but if major updates have been smooth I’m usually pretty willing to install them.