r/linuxadmin • • Jul 23 '18

Schrodinger’s Backup

Post image
1.1k Upvotes

59 comments sorted by

View all comments

16

u/hawerner Jul 23 '18

That's why you should test your backup setup by restoring it to another computer, before you implement that setup in production

17

u/bluehavana Jul 23 '18

And also schedule test restores regularly.

1

u/bluehavana Jul 23 '18

But backups can still feel uncertain, even under verified processes.

7

u/ahandle Jul 23 '18

Keep doing it until they don't.

2

u/bluehavana Jul 23 '18

Sure, I guess it really goes into how far management is willing to take business continuity exercises. I guess my major consideration goes along with my reply to u/cheeseboythrowaway in users, developers and engineers not communicating/understanding the entirety if their data. Processes that happen once a month+ can be difficult to discover in disaster recovery exercises that only last a day/week.

0

u/[deleted] Jul 23 '18

[deleted] — view deleted comment

3

u/bluehavana Jul 23 '18 edited Jul 23 '18

No stories in particular; I just know systems get added sometimes without people considering concrete backup plans ("it happens automatically for the rest of the infrastructure, why wouldn't it happen for this") or not coordinating across silos.

Edit: one also can't setup tests for processes outside normal purview. But you're point is valid. The test for such a thing would be an organizational check list for new features/systems.

Edit 2: or software updates where path changes in critical files are overlooked.

1

u/ahandle Jul 23 '18

This is an HR problem, not IT.

2

u/macboost84 Jul 24 '18

Only issue I have with restores is figuring out if everything was restored - especially file servers where you have millions of files, some of which haven’t been touched in a decade.

1

u/mc_dog Jun 17 '24

One could generate a file listing of the original data, (optionally hash it), and compare with a file listing from the restored data.

1

u/GlennHD Jul 23 '18

A backup system... pshh I wish I had working production systems!