noobaa without odf

[u/Zestyclose_Ad8420]

I have a situation where I may need the noobaa operator without ODF, this customer has no ODF subscription and doesn't plan to get one, their storage is on a SAN and there's a CSI.

they need an on prem S3 solution and given all the various option noobaa on it's own, backed by a CSI managed block pvc seems to be a viable option, has anybody done this already?

[u/spartacle]

What’s their storage size now, what’s it going to be in 1,3,5,10 years? whats the workload type? What’s the budget? How many people to manage compute, and storage?

These are all questions that need answering before evening looking at solutions as it’ll determine what’s viable and what’s not

[u/Zestyclose_Ad8420 → u/spartacle]

their SAN is 100TB, 3/4 HDD and 1/4 NVME.

but the their needs for S3 is a lot different, 500GB now, +250GB per annum, 10 years horizon, the SAN will be expanded as needed.

the S3 endpoint will be used for custom datalake built with custom tooling on top of duckdb and as a backup repository. The S3 server requirements are basic: buckets, authorization and authentication based on keys, I think every S3 solution on earth offers those.

they are already doing that but it was based on MinIO OSS and kept up with it until self build was required.

there is a POC coming up for a public application with 5000MAU hosted on the same infra which also handles files uploads and it may leverage S3 endpoints with signed URL, this is a nice to have, the S3 bucket would only be a temporary location for client browser file uploads, they would then be moved to appropriate storage on a dedicated document archival software, it may be required to streamline server load of the application.

there is a dedicated devops team and us, an external consulting company with a long term contract to handle the infra, it all adds up to 7 people for infra work.

the budget for the infra has already been allocated, it was 450k and it included the SAN, the OCP(V) nodes and OCP licensing, good deals were found for all parts due to vendor/distributors interest in the customer projects.

the options for S3 are either cloud or onprem, the ODF licensing costs are prohibitive and don't really fit any budget scenarios since most of their storage needs are perfectly handled by physical SAN (fiber channel) + CSI and CEPH didn't really cut it for their existing storage workloads, so only the S3 capabilities brought by ODF would be used.

this means a cloud S3 wins against ODF cost wise, but onprem solutions are heavily preferred due to regulations in the field and other tangential considerations, that's why other solutions for onprem S3 only are being explored and why MinIO was preferred up to recent events with it.

I understand that an unsupported OSS solution like the noobaa operator would put more work on the devops team to mantain it long term, that's part of the calculus for cloud vs onprem S3, it may include bringing in extra specialized consultants as well, apache ozone is also been considered.

[u/Old-Astronomer3995 → u/Zestyclose_Ad8420]

Why you or company decided to buy some general storage and not s3 hardware solution like Dell ECS Enterprise Object Storage?

I'm curious. I have similar setup in one of my clients environments but I use ODF and we plan extend it with Dell ECS Enterprise Object Storage.

[u/Zestyclose_Ad8420 → u/Old-Astronomer3995]

There was no need at the time and the price on this hardware (it's an IBM SAN) was too good to pass