An all-too realistic scenario.
The wuhan test lab (where the companys new test team has been set up) have been testing the latest overnight build, which contains the code you checked in yesterday. The machine logs have filled up with some unknown garbage error messages and then your code crashed both horribly and spectacularly in a way you have never seen before. The testers raise a code red urgent defect report which your boss assigns to you to look at first thing this afternoon, he has been in the conference call in the project war room for the last 4 hours with his second line in new york since he got to work this morning.
The product launch date is in three days time and they desparately need to get through this last test cycle to make the date, so missing it likely to be a "career-limiting" event. Naturally you are completely unable to reproduce the problem on the machine you've been unit testing it on locally here in san jose, and are perplexed as to why its failing. Your boss is starting to get concerned about your soft skills too, maybe you don't understand team dynamics after all, and the HR lady isn't too sure about your EQ either.
The box in wuhan is a headless 2U server in a rack buried somewhere down on the test floor. You discover no-one there has time to go down to the basement from the fourth floor to fix the broken KVM switch, so there is no console access, and anyway they're all off and out of the office for the next three days in a couple of hours time because it's the spring festival national holiday, a fact your boss somehow forgot to mention to you earlier. Luckily you are able to ssh into the box over the company intranet so you can investigate the system; you have about 2 hours left today to get this fixed, while the testers are still at work in wuhan to re-run the regression test suite.
The only editor on the box is vi, because that is what was installed as standard with the operating system, nothing else is installed anyway because this is a stripped-down pre-production build. All of the operating system software on the box is mounted on a read-only flash drive as part of the product security policy, so you can't install any new software. In addition the test lab's security policy prohibits you from scp'ing any new software onto their machines over the intranet link you're using, so you don't have a cats chance in hell of putting nano, pico, teco, or anything else onto it. Since the last guy they fired tried to run a crypto mine on some of those big SMP servers, the test lab security team are being extra strict.
You rummage around and find the problem, some of the perl scripts have been screwed up by a bum install process when one of the testers knocked the power cable out of the box during the install, a fact he somehow forgot to mention in the code red bug report. Your boss is now having serious doubts about your leadership qualities in the daily morning crisis scrum. However the broken scripts are part of a test package that was installed on a writeable partition, so you are at least able to modify the scripts.
You decide to patch the scripts manually by editing them in situ over ssh, you think you can see what's wrong. Thank heavens you learned vi years ago; it takes you about an hour to patch the scripts using vi and then do some basic testing, which leaves enough time to have the tester re-run the test. You tell the tester to try it again, whereupon... it all starts working properly and passes the test suite.
Your boss suddenly decides your soft-skills are actually top-notch world-class and in fact the scrum just wouldn't be the same without you, tells the HR person that your EQ is outstanding and that you're overdue to receive a crackerjack pencil! You breathe a huge sigh of relief and briefly reflect on what a good idea it was to put the effort into learning vi all those years ago.
And this story is a lot closer to real life than you might think!
github.com
