Hi. You’re going to start seeing blog posts here from Wavii’s engineering team. They’re a smart bunch, and there’s been a lot brewing at Wavii HQ from which to skim advice. But I’d like you to try your best to ignore at least some of it.
Many software engineers, fortunately, already do a great job of ignoring advice. Some of those engineers even work at Wavii – we wouldn’t be a startup if we weren’t making at least a few rash decisions. But try as we might to fumble about in ignorance, we accumulate good ideas over time. Some of these good ideas are good, and some of them aren’t.
Here are a few reasons why you should sometimes ignore good advice:
You Don’t Work at Wavii
Shame, isn’t it? Similarly, there’s a reasonable chance you don’t work at Yahoo! or Facebook or Google. These companies have solved problems of incredible scope, and in doing so have shared with us the tools and vocabulary we might use to do the same.
When Wavii first got started, we were ardent users (and advocates!) of Hadoop. But we kept asking questions of the Hadoop community for which the most frequent response was “Well, don’t do that.” We needed better support for languages outside of Java. We wanted to make frequent, granular updates to HDFS and be able to immediately retrieve them. We wanted to build sampling experiments with short turnaround. We wanted to do this all reasonably well in the cloud. And we needed an API that was either very uncomplicated, well documented, or at least not in flux.
One day, during a particularly painful session of Hadoop hanging at 99.99% for two hours before crashing, we decided to try running the same job using Chef, a bash script to coordinate fetching data from S3, and a little homegrown network-async tool to partition mapper output. It worked great, and we haven’t looked back since.
Hadoop evolved as a collection of applied architecture principals that are particular to Facebook and Yahoo’s world of gorillion-byte data-warehouses, stored across thousands of real, physical machines. Frankly, many of those principals – things like replication, name-nodes, heartbeat – aren’t as important to cloud-based startups.
Similarly, Wavii has some peculiar problems that probably don’t apply to you. But we are going to share what we’ve learned, and if you aren’t careful, you might take those lessons as rules of thumb. Sorry about that.
You Exchange Ideas Using Words
An idea is tangled up in its encoding, and good encoders win when they evangelize. Why is data science a thing now? Why are we scrumming at work? Why don’t relational databases scale? (or for that matter, why do they?)
All of these ideas have arrived to you via some very talented writers. Effective writing and clear presentation are incredibly important aspects of software development, and I love the influence that good communicators have over our hive mind. But it’s impossible to deny that sometimes the quality of communication outpaces the idea itself, or conversely, we actually come to expect a certain presentational pizzaz when we consider new tools or concepts.
I noticed this problem when searching for metrics software to use at Wavii. I immediately discounted a tool named Graphite because their website looked awful compared to many of the newer, shinier tools available today. Weeks later, after a few discouraging experiences with the shinier tools, we revisited Graphite. Much like the last anecdote, it worked great and we haven’t looked back since.
You Listen To A Few Winners
This is the trickiest one to explain – bear with me. Very few people or companies actually make it in the tech world, and these are the ones whose advice we most often heed. In machine learning we call this asymmetry, but you can also think of it as survivorship bias.
Imagine if you interviewed emergency room patients to understand how to avoid ending up in the hospital yourself. You might notice that they all rode in ambulances before being hospitalized – would you conclude that a solid way to avoid health crises is to stay away from ambulances? Of course not!
But we employ similar thinking when we focus on generalizations from disjoint sets of circumstances that brought some narrow 1% of our field into the limelight. If we aren’t careful, we end up joining a cargo-cult for ideas with mantras that we all repeat: multithreading is harmful, relational databases don’t scale, don’t reinvent the wheel.
The best part of this imbalance is that the winners have already won. Cutting-edge startups, on the other hand, are in the business of getting there. In fact, it’s usually a bad sign if a startup depends too much on a wealth of accumulated wisdom. So our industry creates this churn where the most popular tech aphorisms are dispensed by those who’ve already made it, and the same advice is most often ignored by those who are making it right now.
This means that if you want to be successful in a startup, you should be very suspicious of collective wisdom. The more compelling the idea, the further in the past it was probably conjured. It’s tricky: sometimes you just have to unexpect the unexpected.
But that’s okay! Because you’ve read this now. You’ve learned an important lesson about disregarding important lessons. And we can’t wait to share another lesson with you soon.
