Organic (growth) is good
At MyBuzz we’ve been fighting a “slow” fire for the last couple days. Every time our crawl kicks off, our website becomes very slow, because they’re hosted on the same box. Fortunately, since we’re still in beta and don’t have a ton of customers, this wasn’t a big deal and people were understanding. This highlights a big advantage of organic growth – we never have to fight flash fires, so we don’t have to cut corners.
The problem turned out to be a spike in memory usage. We have a Rails site that kicks off a rake crawl process. As our customer base grew, we began to see a steady pattern of failures, because crawls started to overlap and the combined memory usage maxed out our server. There are two ways to fix this:
- Throw more hardware at the problem
- Re-architect the system
On the surface, option 1 seems attractive. We don’t have to write any code, and a server twice as big costs $50 more a month. Option 2 requires a significant amount of dev time. It ended up taking a couple days. Surely, a couple days is worth more than $50?
Without solving the root of the problem, it’ll come back again as we continue growing – when it’ll matter more, with more customers who’ll expect a non-beta product to be up and running. Knowing that, we took the time and made the crawler run on Amazon’s Elastic Compute Cloud (EC2), which means we’ll be able to scale out infinitely. This comes at a cost savings as well. EC2 costs $0.1 / hour. Because the crawler doesn’t run constantly, we’re paying for about 10 hours a day, which translates to $30 a month. And it’s a more powerful server!
We could have saved the rewrite for a later time. Now that I think about it, we already passed on it once and doubled our memory 
. So this post isn’t about making good decisions. It’s about having the option to make that decision. Instead of spending time and money on acquiring customers, we’re focused on improving the product and making sure existing customers are super happy. And a good product sells it self (mostly ).
