Auto-Scheduled Oops

I blew up some active development work the other day. I’d been meaning to implement auto-shutdown on a number of outlandishly expensive virtual servers that are used only as development machines. They’re only used intermittently, so shutting them down after-hours not only saves us billing for the overnight and weekend hours, but also for whatever days folks may not be using them immediately or at all. Now, sometimes, developers have long-running processes (test-runs of large reporting jobs, or something) for which we’ve disabled this auto-shutdown in the past, but they’re definitely anomalies in our process. So, on my timeline and having put it off for so long that I just wanted it out of my head, I enabled the auto-shutdown at 6:30PM local time (an appropriate after-hours time for most of our developers) and made an announcement that could be summarized as “This happened, it’s to save money, if you have any issues, concerns, or adjustments you need made for your personal situation, just let the team know so someone can take care of it” This occurred late in the day (just before 4:00PM or so) and about 4:40PM I finished up for the day, having heard nothing. No more than ten minutes later (as I found out much later) a senior dev (who had the necessary permissions to resolve or avoid this without intervention) responded, decrying this setting, and asking for an exemption for that particular evening, tagging only me. Needless to say I did not see this. Having a strange impulse to check my messages a few hours later, I was greeted with the fallout of this auto-shutdown triggering during a long-running process. I wrote an apology, reverted the settings, and lamented my own haste for applying this with only a small window before it took effect.

I spent a long while afterwards thinking about the implications of such a situation on my own consciousness. Not because this situation was anything particularly profound, but it was something that could warrant a lot of possible responses depending on ones mental setting. For me, I owned it and apologized immediately; for some watching, that could have seemed performative, but I was genuinely sorry. Now, there was still nothing I could do to recover that developer’s time; I offered a hand with developing plans to avoid things like this moving forward with these settings by planning out solutions for any exceptional cases in advance, but of course that would have been better before all of this happened. I do not have a particularly positive history with this developer anyway, so I was concerned about the possible response being more negative than I may have anticipated from another coworker at random. I spent a number of moments finding myself playing through different situations that might arise in my meetings with this person scheduled for the next day; I found that my predicted or simulated responses included feelings of fear and anger rooted in embarrassment but procedurally justified through my own weaponizations of the truth: that there could be nothing done about that lost time; that the only thing we can do is move forward more effectively. Though these are true, it’s no use to throw that at someone who is feeling negatively about something that has happened. For me to do so would be an example of the classically despicable “I know I messed up but I think it’s bad that you’re mad about it” response, which is easy to slip into to avoid further feelings of embarrassment through what we believe are justifiably cold or harsh responses.

I had to pull myself out of my own mental simulations of how this would all play out, typically using a reflection upon my previously noted weaponizations of the truth: that there could be nothing done about that lost time, and that the only thing I could do was move forward, but this time coloring those ideas with a sense of compassion towards the developer. That might sound a touch cheesy, but it was important for me to recognize that if I wanted to continue on the path I had set with a true apology, I needed to follow it with a commensurate level of compassion towards the person I had negatively affected. That required quelling the “being right” desire that fuelled those mental simulations, but also allowed me to remove myself from them sufficiently to recognize that they were not useful in the present, and that a conscious approach to any conversations that may follow as I worked to re-implement this policy more effectively in the future would avoid any of the issues I was concerned about. People could react negatively, but I could not be caught in a mess of those negative feelings I mentioned unless I allowed my own focus to drift away from the facts at hand to the trappings of protecting my own image, or my own sense my capabilities or status as a “good engineer.”

The next day, nothing happened; the developer did not mention it at all, and never responded to any of my messages. I think that’s fine, personally. I have not yet pursued round two, but I think all of my investigation (over-analysis, perhaps) of the situation as it presented in my mind my feelings towards it have given me a good place from which to tackle that. It’s an interesting situation when I can feel myself devoting a bunch of attention to something seemingly insignificant, but I can also feel myself learning from something that may have otherwise been brushed aside. Getting out what I put in, hopefully.

P.S. As I was writing this, one of my personal icons of my industry and interests was working at the bar behind me in the coffee shop. Uselessly glad that he doesn’t know how I messed this one up.