skarnet.org will be down until further notice - it may take up to 10 days (!)
-
@ska @navi Considering Gmail "too big to fail" is very... optimistic.
Google can and will destroy anything in their control, up to and including Google Search. Look at the Google Graveyard! Is Google really a company you should trust with anything? https://killedbygoogle.com/
I also use Gmail and I'm typing this from Android, but I live in fear of these becoming untenable overnight. I would spend a lot of time, money and effort to achieve stability elsewhere. Google may continue, but will Gmail?
@skyfaller @navi For all intents and purposes of this conversation, "too big too fail" means that gmail will be adequate to receive recovery e-mails, and that if Google kills it, it won't be overnight, and I'll have ample warnings and time to modify my setup.
The lifetime and management of Google projects is something I'm well aware of, but isn't exactly relevant to my current predicament.
-
@navi@social.vlhl.dev @ska@social.treehouse.systems @skyfaller@jawns.club people don't actually use gmail so it getting worse won't do much as long as it still sends and receives emails
if the ui becomes utter garbage people will use external clients like when they did the initial redesign like 15 years ago@navi@social.vlhl.dev @ska@social.treehouse.systems @skyfaller@jawns.club people are still using outlook/hotmail after all
-
@navi@social.vlhl.dev @ska@social.treehouse.systems @skyfaller@jawns.club people are still using outlook/hotmail after all
@navi@social.vlhl.dev @ska@social.treehouse.systems @skyfaller@jawns.club there's also the bit of moving to a new email is a pain in the ass, especially with a ton of sites not allowing you to change email addresses bc it's your login id.
-
@skyfaller @ska considering how much data they get by hosting gmail, yeah -- while both google is alive and email is used by people, gmail will last
it'll get shittier tho, but that doesn't matter much@navi You may be right, but are you saying that Google didn't get data from Google Plus, or Google URL Shortener? Feels like they have killed a lot of useful data sources.
-
@navi Spreading your dependencies is clearly a more resilient setup and you're right to do so.
But the thing is, this incident also confirms me in my belief that it's a good thing for me to have control on as many things as possible, because as soon as I depend on one external entity, they fuck something up. So I want to be self-reliant even more.
And I guess my external line will now be gmail, for the sole reason that they're too big to fail. Which I don't like, but I'm going to use that property.
-
-
skarnet.org will be down until further notice - it may take up to 10 days (!)
I wanted to reboot the server on a newly tailored kernel to avoid CopyFail shenanigans. The shutdown failed right before rebooting (because of a small oversight in s6-linux-init that is easy to fix and will not impact anyone else). Normally that wouldn't be a problem: log in to the web interface to the VPS administration, press the reboot button, done.
Except, Gandi added 2FA since I last logged in, and did not notify their users. And they decided that by default, the verification code would be sent to the registered primary e-mail address.
My registered primary e-mail address is hosted on skarnet.org. Because it was never a possibility that it would be needed for the bootstrap chain. Except that now it is, and I can't access it.
This is the dumbest catch-22 in the history of catch-22s. And the recovery procedure, when you have to prove your identity when writing to the support outside of a logged in account, can take up to 10 days. And tomorrow is a holiday. Fuck me, I guess.
I should have paid more attention, but the consequences of my not paying enough attention should definitely not be so vast, and I have choice words for several of the decisions that were made.
That was way faster than expected! Some person read my emergency ticket sent last night (from logged out), and I seem to have been convincing enough because they restarted the server for me, and now everything is working again. π₯³
Guess I won't have these 10 days of forced vacation.
Now to take measures so that it doesn't happen again...
-
That was way faster than expected! Some person read my emergency ticket sent last night (from logged out), and I seem to have been convincing enough because they restarted the server for me, and now everything is working again. π₯³
Guess I won't have these 10 days of forced vacation.
Now to take measures so that it doesn't happen again...
@ska i guess "please restart this vps that is hang anyway" is a easier ask to "give me access to this account" -
@ska i guess "please restart this vps that is hang anyway" is a easier ask to "give me access to this account"
@navi Absolutely. I had asked to either send a recovery code to my gmail account, or reboot the server. They rebooted the server. Totally fair.
-
@navi Absolutely. I had asked to either send a recovery code to my gmail account, or reboot the server. They rebooted the server. Totally fair.
-
@lanodan @navi https://git.skarnet.org/cgi-bin/cgit.cgi/s6-linux-init/commit/ - not pasting the permanent URL to avoid people getting caught by my LLM blocker, the commit id is 43394581bb3106633c8802b378a7f2e6e53b352a
(Edit: add the commit id)
-
@lanodan @navi https://git.skarnet.org/cgi-bin/cgit.cgi/s6-linux-init/commit/ - not pasting the permanent URL to avoid people getting caught by my LLM blocker, the commit id is 43394581bb3106633c8802b378a7f2e6e53b352a
(Edit: add the commit id)
-
@lanodan @navi I never have problems booting thanks to the static guarantees of s6-rc, whenever I have an issue it is shutting down because after some living, the state may be different from what I assumed it would be.
But I'm learning, and incrementally fixing assumptions about state, this is only the latest case - and hopefully the last one.
-
@lanodan @navi I never have problems booting thanks to the static guarantees of s6-rc, whenever I have an issue it is shutting down because after some living, the state may be different from what I assumed it would be.
But I'm learning, and incrementally fixing assumptions about state, this is only the latest case - and hopefully the last one.
-
@navi Spreading your dependencies is clearly a more resilient setup and you're right to do so.
But the thing is, this incident also confirms me in my belief that it's a good thing for me to have control on as many things as possible, because as soon as I depend on one external entity, they fuck something up. So I want to be self-reliant even more.
And I guess my external line will now be gmail, for the sole reason that they're too big to fail. Which I don't like, but I'm going to use that property.
-
@lanodan @navi You can always force a shutdown until it's on a remote VPS and the shutdown worked just well enough to kill everything including your sshd and your emergency getty and leave you with a brick and you're logged out of the VPS administrative account and the 2FA codes are sent to the brick.
(Edit: accuracy)
-
@navi What's the alternative? Paying for a second VPS hosted elsewhere? Having to manage another e-mail address?
One of the points of skarnet.org is maximum independence. And really, it has worked pretty well so far: I could always recover from mistakes.
Now my new recovery address will be my gmail one. Lovely. I could change and ask for some hosted address somewhere else, but that defeats the purpose of being self-reliant.
Really, it all comes down to "don't cut my lifeline without telling me you fucking clowns".
@ska@social.treehouse.systems @navi@social.vlhl.dev Having a backup MX or a backup DNS server would give options. While mail often can't be run on residential lines, DNS usually can.
A backup MX will receive and store email to be forwarded, and you can just look directly in the mail queue to get the auth code, for instance.
I completely agree that companies' ideas about two factor are broken, are often added without asking and without configuration, and can be highly problematic during emergencies. At least you're here, sharing with others, so others can consider whether they might be in the same scenario should a problem occur.
-
@ska@social.treehouse.systems @navi@social.vlhl.dev Having a backup MX or a backup DNS server would give options. While mail often can't be run on residential lines, DNS usually can.
A backup MX will receive and store email to be forwarded, and you can just look directly in the mail queue to get the auth code, for instance.
I completely agree that companies' ideas about two factor are broken, are often added without asking and without configuration, and can be highly problematic during emergencies. At least you're here, sharing with others, so others can consider whether they might be in the same scenario should a problem occur.
@AnachronistJohn @navi that's the point of the thread: so others can learn from my misadventure
Setting an external secondary DNS server is something I have thought about, obviously, but the answer was always "not worth it" because only this server is under skarnet.org so losing the DNS means losing access to a server that is down anyway.
To have enough redundancy to avoid what happened to me, the external machine would also need to be MX and mail storage. I don't want to install that myself (duplicate work), I don't want to pay for that, and that's a little too much infrastructure to ask friends. I'd rather learn the lesson and not be an idiot next time
-
That was way faster than expected! Some person read my emergency ticket sent last night (from logged out), and I seem to have been convincing enough because they restarted the server for me, and now everything is working again. π₯³
Guess I won't have these 10 days of forced vacation.
Now to take measures so that it doesn't happen again...
Was the procedure to wait for a secret code via postal mail?
-
R relay@relay.mycrowd.ca shared this topic
