• Technical Debt Pays Our Rent

    I started my business a few years ago with just a couple of clients, as a part-time on-the-side gig. I intended to make just enough from it to fund my hopes and dreams in software development, and maybe on to other, bigger, things.

    That little business, based out of a small town in California, is now paying most of the rent and expenses for three people, and still growing. I should be proud -- and usually I am -- but I'm also disappointed.

    We're troubleshooters. We handle a ridiculously large range of technical problems, from website-related to computer-related to network-related and on and on. We're quite good at it. We're what Geek Squad could be, if they had ethics, principles, and professionalism.

    So, our bills don't get paid unless someone, somewhere nearby, is having trouble with technology.

    This guy, during a meeting with us a while back, asked us incredulously if we thought computers were unreliable. I stammered out a weak, "uh ... yeah" in response; it had seriously never occurred to me that there were people who thought they weren't unreliable.

    I wonder sometimes how it will all turn out in the technology world. I've been working professionally in one aspect or another of I.T., off and on, for a little over 15 years. I've been a hobbyist programmer since the second grade of elementary school. I'm not really sure whether technology is getting better or not in terms of reliability.

    There isn't a single brand of printer that I feel like recommending to anyone, and someone asks us about that pretty often. I don't even feel like I can recommend a specific model of printer; too often, whether the printer is any good or not comes down to chance, whether or not the manufacturer managed to correctly produce a particular machine without defects. This is largely a result of the race to the bottom in printer prices even as their complexity has gone up -- but that doesn't change the fact that printers and their various issues frustrate huge numbers of people every day. As an industry oldtimer at this point, I wonder: whither the HP LaserJet 4?

    My dad approached me, unprompted, this weekend and asked me if I knew of any good search engines. He, like so many people, has been having trouble with Google. I said, no, I didn't know of any; that there was DuckDuckGo, which I like, but is just an aggregator whose results depend largely on other, crappier search engines. Then he wanted to know why it seemed like Google wasn't working well anymore, and I made some lame excuses about spam and that sort of thing, but maybe I should have explained instead that Google just isn't concentrating fully on search now. Now, they're an advertising company; they're a social graph company, trying to compete with Facebook for who-knows-what-reason; they're an email company.

    Google is probably my favorite whipping boy, because I'm such a fan of business case studies, and here we have a company that has done so many things so right: they concentrated on one problem, and solved it, and solved it well, and then they made a serious effort to go out and hire the smartest people they could, so that they could continue to solve that problem well. Historically, it's one of those cases that's right up there with early Ford. And yet, my dad loathes it; I've had clients, out of the blue, tell me how much they hate Google Instant or some other Google feature; and since May 23rd of this year, I've been maintaining a file on my computer called "Google Faillog", documenting many of the various ways in which it irritates me every day. New entries in that file have tapered off because so many of the irritants are just variations on ones I've dealt with before; it's the same problem, over and over and over again. Today, for example, I wanted to search for information related to Apache2's mem_cache module and Wordpress. Every time, I had to tell Google that, yes, I really did mean "mem_cache", not "mod_cache" or "memcache" or something else. I hate that it constantly assumes that I'm a fuck-up, especially when there's no logical reason to assume a misspelling. A lot of other people hate that, too.

    But, I guess that's the best that thousands of the world's smartest people can pull off.

    Which makes me worry a little.

    In my business, I've turned in to a big stickler for documentation. Solve a difficult problem? Document it. Build a server? Document it. Document, document, document. This is a big deal because I hate documenting things, just like I hated doing homework all throughout my school years. But, at some point recently, it really began to dawn on me just how bad most of the how-to's and already-written documentation were. The documentation for Samba is an extreme case of awfulness, so I'll let go of that one and just look at the documentation for setting up a web server. Web server documentation should be straightforward. After all, so much of the software now can be installed and configured pretty darn easily. But, there are all kinds of features and tweaks and tune-ups and knobs to turn, and the documentation for all of that is horrible. Probably one of the most valuable individual items that my business owns right now is documentation for setting up a web server and a mail server. The web server documentation includes an incomplete list of references for eight other websites.

    This, too, is partly because of fragmentation in software platforms. nginx won't deal with Apache's rewrite rules and other directives, so it's not a drop-in replacement for everyone. Apache's configuration and management isn't quite the same from one operating system to another. And so on.

    A lot of people now will say, "So use the cloud!" I'm not sure what to think of a solution that just makes the same problems someone else's responsibility.

    Clients actually ask us to unbox their brand new computers for them and "set them up", which usually means start them up, remove all the garbage installed by the OEM, install all of the software updates that have been released since the manufacturer last created their disk image for this particular system, install a decent free antivirus, that sort of thing. It seems strange to me every time we're asked to do this -- I can't think of a single other piece of equipment, that doesn't come from Ikea, that requires this sort of post-retail work as a "normal" thing.

    We used to also install Firefox and AdBlockPlus by default, but Mozilla has gotten it into their head that what their customers really want is a web browser that's a little bit different every other time they launch it. We've had to deal with some frustration from clients who we convinced to switch to Firefox a year ago or more, who are now really unhappy with it. Attempts to explain this directly to Asa Dotzler have been met with a very polite, "Piss off, we know what's best for our users" response.

    I could go on and on and on like this until your eyes glazed over, if they haven't already. Chris at this very moment is grumbling about Sony drivers for a fresh install of Vista on a laptop. (Vista couldn't partition the new hard drive; he had to do it from Linux.) When I'm done with this, I need to go back to trying to figure out why the most recent version of Wordpress seems to have a regression that makes it incompatible with mem_cache -- a chore which would be much easier if only I had a good search engine to work with. Tomorrow will be like this, yesterday was like this, it's been like this for years now. Every day.

    This entire industry feels to me a lot like trying to build a stable monument, floating on the ocean. Our toolchains are not great; I waste hours every week trying to troubleshoot problems that could have been easily diagnosed with the help of a decent message or two posted to an easy-to-find log file. We see regressions all the time, and long-standing bugs that have never been fixed, and all this extreme frustration with technology from a very wide array of people, and this attitude among our clients that sort of, with resignation, just accepts that this is an inescapable cost of relying on technology.

    All the while, programmers keep producing new programming languages and arguing about frameworks and which language is better and how to prioritize features versus bugs and all this other nonsense.

    And I ... I have given up. Maybe my experience with Samba recently was the straw that broke me. I don't know. All I know is, I don't feel like I can talk to anyone else in my industry anymore, and get them to see what I see.

    The only good news I have with all of this is that my small business has made enough money in the last twelve months that we've begun to do some product development of our own.