Morlock’s, Microsoft Mail for the Macintosh and Me.
Just a little bon mot about my early career.
In 1995 I followed a fellow named John Clark from the State Department to a law firm, Howey and Simon, where he was taking the CIO role. This was before most law firms *had* CIOs, so it was a very big deal. I wasn’t a big deal, I was just a decent computer nerd that he decided to bring along to work in the firm's IT department.
This was, I would come to understand, a sort of hazing ritual. This job would prove me to be either a complete fool or redeem me in the eyes of my peers who weren’t optimistic. I was young, keep in mind, maybe 23 and I really had no clue on corporate politics. I just didn’t understand that the new CIO guy bringing in his pal and plopping him in the middle of a decently well run department without their review or even much interviewing would be looked at askance.
To get to this job, you’d either take the metro to McPherson Square or drive into the district. I’ll paint the picture of driving to this job: You pull up and into the parking garage under the Warner building. On the first and second subfloors, you would pass the S-class Mercedes, 7 series BMWs and Porches owned by the lawyers, descend the next few floors of C-classes, 3 series and such of the paralegals, then the 5th , 6th and 7th floors of Tauruii, Camrys, Civics and the like of the support personnel.
I would park my beater Integra (3 out of four cylinders firing on a good day) usually in the last spot on the 7th subfloor. Get out, not bothering to lock the car as if it were stolen I’d come out ahead. Then I would open the stairwell door and descend down to the final parking level they had converted into the offices for the support personnel.
On that floor were my friends in reprographics, accounting, IT and a little microwave and cafe for us underground denizens. It was really nicely done, no exposed concrete except for some columns and such here and there, and decently lit.
To be clear, the warner building was *spendy*. Three or four blocks from the white house and super nice. You don’t have your , ahem, helpers take up that kind of real estate. To the basement with you! Morlocks stay below, Eloi up top. I wasn’t claustrophobic and didn’t see how really ominous all the carbon dioxide warning alarms were, and I was just there to make a living messing with computers. I was, well, decently happy.
Anyhow, it was here I was tasked with running the email network for the firm. It was a pretty decent job, technically interesting. I was in charge of 1200 users running on Microsoft mail for the Macintosh across 12 at-the-time-beefy Macintosh servers. I remember a few weeks into the job I had a real problem with performance on the system and I called up the support folks to get some guidance on how to improve reliability and such in the system.
Imagine young me, bright eyed and bushy tailed, calling up a random number in Washington state and asking “Hey, I’m having this issue with my email servers”
A very helpful person on the line asked me if I had tried, well, vacuuming (cleaning up the flotsam and jetsam) the mailbox databases? I had, of course, and would do so according to the schedule given in the docs and by my manager1. I explained that it didn’t seem to help much and the process was always fraught with indeterminate failures and outages.
The support person asked “Well, how many mailboxes are we talking about here, tell me about the installation?”
I explained that we had 12 servers with about 130 or 140 megabytes each of mailbox databases which were largely evenly distributed among the 1200 people we were serving.
“Oh, well that’s your problem. We never tested more than 22servers and 10mb mailbox servers … though we did test more than , I dunno, 300 or so users).... “
This gave me pause. So I was basically running 10x the load and 6x the number of interconnected servers (this doesn’t include the ms mail for pc gateway to partner firms I maintained) and their guidance assumed much much less traffic and database churn.
I would discuss with them what sorts of things I could do both with and without taking down the network and re-architecting the entire system.
At this point I should point out that Howrey was a litigation heavy law firm, with some of the most aggressive lawyers and paralegals I’d ever meet, I remember walking into a common area on one of the floors and hearing about one of the attorneys being praised for getting someone testifying against his clients to break down in tears on the stand. It wasn’t his first time either.
Anyhow, I was very much incentivized to never, ever, allow the email network to go down in any way that the powers that be would want to murder me for, so… in working with the folks in Redmond3 I came up with a plan to keep the machines nice and tidy on the regular/weekly, and bring them all down every three weeks for an hour or so on Friday evenings for a full database cleaning and re-compaction.
It’s funny, I would send a note to all the legal, paralegals, everyone a note saying that this was happening the morning of, an hour and 15 minutes before and even doing so, I’d get a call or page that would tell me I was ruining someone’s life… it largely passed after the third or fourth time I’d done it though, as they got used to that hour of downtime, but it was extremely instructive on how to manage expectations of my customers.
I would do this until I exited some 9 months later for a job in California, which was blissfully above the water table.
That manager, Chuck McKinster, was one of the nicest people I’d work for and I’d think of him often as I grew in my career, but that’s for another article. I’m sure he has no clue that I learned a bunch from him that would come in handy at Google later.
Or three, it’s been a few decades, after all…
I don’t know that’s where the other end of my emails and phone calls were going, but let's go with it