#copyfail is why the Forgejo Actions runner service i'm setting up will be using kvm and single shot VMs rather than containers
-
#copyfail is why the Forgejo Actions runner service i'm setting up will be using kvm and single shot VMs rather than containers
@whitequark dunno what copyfail is but we're gonna end up doing the same
currently working on migration to our alpine+libvirt+scripts hypervisor os on main server specifically to allow for stuff like this
eventually plan to make a thing to let us do docker compose style services, but as a full VM per service or a container runtime if required -
@thejpster yep, the plan is to do something like that with microVMs (or the closest approximation if the obvious solution is non-viable)
@whitequark I suspect you won't be using VMware vSphere and the VMware Perl SDK though
-
@janl yes (already underway) and uh want to buy me a motherboard?
@whitequark I have my old box collecting dust while waiting for a crazy enough buyer. Im slowly lowering price, but probably it’s not going to be sold in next few months.
If it’s any use for you in meantime, I can give you direct access to bcm webui so you can do whatever you like with it
We can figure out how much of heads up you need in case someone would be willing to buy it. And I have second low power box that can be used for backups
EPYC 7551P (1gen, 32C/64T) on Gigabyte MZ01-CE1 (with browser based KVM)
RAM 256G: 8× 32G M393A4K40BB2-CTD DDR4-2666
NVME 2T: 1× WD Black SN850X
HDD 12T: 4× 3TB HGST Ultrastar 7K300- 2× Tesla P40 (if you are in a mood for messing with sharing gpu with workers)
-
@whitequark dunno what copyfail is but we're gonna end up doing the same
currently working on migration to our alpine+libvirt+scripts hypervisor os on main server specifically to allow for stuff like this
eventually plan to make a thing to let us do docker compose style services, but as a full VM per service or a container runtime if required@whitequark hoping there is a way to speed up vm launch time though, if you have any resources that might be useful, feel free to send us, medical issues are making it almost impossible for us to focus enough to find resources on how to best do this kinda thing securely and decently
while we'd love to use xcp-ng, the old kernel / xen kinda limits its use on little guys (fucked cpu freq scaling), and proxmox requires so much host configuration for our weird setups which we'd rather avoid, so making our own embedded style alpine distro with A/B boot specifically for this -
@whitequark containers: the convenience of a VM and the security of rawdogging userspace
-
@whitequark hoping there is a way to speed up vm launch time though, if you have any resources that might be useful, feel free to send us, medical issues are making it almost impossible for us to focus enough to find resources on how to best do this kinda thing securely and decently
while we'd love to use xcp-ng, the old kernel / xen kinda limits its use on little guys (fucked cpu freq scaling), and proxmox requires so much host configuration for our weird setups which we'd rather avoid, so making our own embedded style alpine distro with A/B boot specifically for this@chaos firecracker or crosvm is the current plan but i'm not recommending either until i get actual operational experience
-
@whitequark I suspect you won't be using VMware vSphere and the VMware Perl SDK though
@thejpster mercifully, no
-
@whitequark I have my old box collecting dust while waiting for a crazy enough buyer. Im slowly lowering price, but probably it’s not going to be sold in next few months.
If it’s any use for you in meantime, I can give you direct access to bcm webui so you can do whatever you like with it
We can figure out how much of heads up you need in case someone would be willing to buy it. And I have second low power box that can be used for backups
EPYC 7551P (1gen, 32C/64T) on Gigabyte MZ01-CE1 (with browser based KVM)
RAM 256G: 8× 32G M393A4K40BB2-CTD DDR4-2666
NVME 2T: 1× WD Black SN850X
HDD 12T: 4× 3TB HGST Ultrastar 7K300- 2× Tesla P40 (if you are in a mood for messing with sharing gpu with workers)
@alex access would help a lot for testing, yeah!
-
@chaos firecracker or crosvm is the current plan but i'm not recommending either until i get actual operational experience
@whitequark let us know how you get on with em
we did some hacking on crosvm and used it a while ago but knowledge gained never sync'd -
@whitequark let us know how you get on with em
we did some hacking on crosvm and used it a while ago but knowledge gained never sync'd@chaos ack, I will be sharing it here
-
#copyfail is why the Forgejo Actions runner service i'm setting up will be using kvm and single shot VMs rather than containers
@whitequark Wow... that one is pretty bad.
Guess I'm deleting some kernel modules today.
-
@whitequark I suspect you won't be using VMware vSphere and the VMware Perl SDK though
@thejpster i have to say Solaris zones are surprisingly slow for this
-
@alex access would help a lot for testing, yeah!
@whitequark awesome! Gimme some time, I'll prepare everything. How can I reach you to hand over access credentials?
-
R relay@relay.mycrowd.ca shared this topic
-
@whitequark awesome! Gimme some time, I'll prepare everything. How can I reach you to hand over access credentials?
@alex whitequark@whitequark.org, you can encrypt to https://codeberg.org/whitequark.keys
-
#copyfail is why the Forgejo Actions runner service i'm setting up will be using kvm and single shot VMs rather than containers
@whitequark wanna tell me a bit more about that forgejo runner system?
interested. -
#copyfail is why the Forgejo Actions runner service i'm setting up will be using kvm and single shot VMs rather than containers
@whitequark What do you use the orchestrate the VMs? Every time I try to do this I end up trying to reinvent half of libvirt.
-
@whitequark wanna tell me a bit more about that forgejo runner system?
interested.@jorijn it's very much WIP
-
@whitequark What do you use the orchestrate the VMs? Every time I try to do this I end up trying to reinvent half of libvirt.
@ananas nothing specific yet, trying out some stuff
-
@jorijn it's very much WIP
@whitequark interesting nonetheless, does it resemble https://codeberg.org/forgejo/discussions/issues/152?
-
@whitequark interesting nonetheless, does it resemble https://codeberg.org/forgejo/discussions/issues/152?
@jorijn i'm leaving the actual forgejo specific bits until the end, want to see that i can reliably and quickly spawn VMs first
