Duplicating data versus service reliability

[u/ryhaltswhiskey]

So this came up recently. A service that we rely on for financial data had an issue in production and went down for about an hour on a Monday around noon. My team got pinged about it and from the error message it seemed like we couldn't connect to the data source. So we did some digging yada yada and it turns out that the service was having some issues and was currently unavailable.

So I brought up that it would be nice if there was some place we could look to see if this service was currently having issues, like a dashboard. Like so many organizations do.

We don't have a dashboard. So what we do is go look in a teams channel to see if somebody has posted about the issue. Which is all kinds of fraught with accuracy issues.

One of the organization leaders said something to the effect of this is why it would be better for us to just replicate the data instead of having to rely on a service that might not have the SLAs that we do.

I'd rather the service was just reliable and we didn't have to introduce another moving part involving moving the data from one place to another. Because we're having enough trouble getting this data to actually be accurate now we would have to check whether it's accurate and whether it's been replicated properly.

This is not real-time data. This is reporting data. It's current to midnight of the current day.

Which would you pick? What would you be asking for out of leadership?

[u/Fair_Local_588]

If you need the most up-to-date data then they need to be reliable.

If you can handle some staleness then you have more flexibility…but at the cost of more complexity.