#DuckDuckFedi: what's a good S3 storage backend that is lightweight enough to not be a pile of overhead for small deployments, but also scales for larger deployments?
-
#DuckDuckFedi: what's a good S3 storage backend that is lightweight enough to not be a pile of overhead for small deployments, but also scales for larger deployments?
MinIO took their repo offline again recently and I'm looking for alternatives.
things I came across or already knew:
- SeaweedFS (small single go binary)
- Garage (seems overkill for this but unsure if it actually is that heavy)
- RustFS (neat MinIO replacement, but very new, unsure if mature enough)
- Ceph (definitely overkill) -
R relay@relay.mycrowd.ca shared this topic
-
#DuckDuckFedi: what's a good S3 storage backend that is lightweight enough to not be a pile of overhead for small deployments, but also scales for larger deployments?
MinIO took their repo offline again recently and I'm looking for alternatives.
things I came across or already knew:
- SeaweedFS (small single go binary)
- Garage (seems overkill for this but unsure if it actually is that heavy)
- RustFS (neat MinIO replacement, but very new, unsure if mature enough)
- Ceph (definitely overkill)@anthropy I've been playing around with Garage and it works okay for me. Not amazingly easy to auto-deploy on e.g. Docker as it needs some commands running to configure it.
My use case is a bunch of workers orbiting an API pulling work from S3/API and putting the results on S3. -
#DuckDuckFedi: what's a good S3 storage backend that is lightweight enough to not be a pile of overhead for small deployments, but also scales for larger deployments?
MinIO took their repo offline again recently and I'm looking for alternatives.
things I came across or already knew:
- SeaweedFS (small single go binary)
- Garage (seems overkill for this but unsure if it actually is that heavy)
- RustFS (neat MinIO replacement, but very new, unsure if mature enough)
- Ceph (definitely overkill)@anthropy SeaweedFS does work remarkably well, and it has some amazingly low system requirements too.
But I've had multiple times where I've lost EC shards when not running the rebalance and rebuild regularly enough, so I'd recommend sticking to replicated storage for where the data is actually important. -
@anthropy SeaweedFS does work remarkably well, and it has some amazingly low system requirements too.
But I've had multiple times where I've lost EC shards when not running the rebalance and rebuild regularly enough, so I'd recommend sticking to replicated storage for where the data is actually important.@ace that is definitely very good to know, I'm definitely looking for reliable storage here
but also interesting because it feels almost like all that would need is a good wrapper to keep track of and manage the shard redundacy
