Robin Gierse
Checkmk Consultant

Today I learned

The fall of an admin

2 min read

Today I learned: About the fall of an admin.

What happened?

A coworker shared this story and it touched me. The story is told very quickly: An admin, which ran three Mastodon instances suffered a total loss of data on all three instances. Why? Because essentially they had backups, but never tested them. The story is a little more complex, but that is not the point of my writing.

Why do I share this?

I share this, because the last words of the announcement get me.

"Being an admin has been one of the most fulfilling things I have done in a long time, and you all have made it such an amazing experience. However, I need to take a step back."

I feel this to the very essence of my self. I am an admin by trade and by heart and data loss is one of the most crippling feelings there is in this profession. No matter if it is your very own data, your company's, customer's or that of your friends and family. Knowing you lost data is terrible, having to tell this your boss, client, a friend or a family member is even worse.

What I am trying to say is this:

  • Always have backups and test them!
  • Do not shame a person that lost data, they most likely suffer enough already.
  • I wish this person the very best and the strength to get back in front of a keyboard and spin up another system! You can do it!

Update

After reading this post some months later, I realize that I want to put one thing into perspective: While I ask to not shame someone for loosing data, I need to make a clarification: This applies to this very scenario, maybe stretching into the private realm. But if someone is paid to keep data secure, and they mess up, it is on them. Especially if the data loss is due to neglect or sheer incompetence. In my initial post I had a private person in mind, running something on the side, without commercial interest and obligations.