After we launched our first mobile app, Menuboard, we faced a big silence: we saw a few people downloading the application, but we didn’t know what they used it for, or if they used it at all. Since the App Store owned the relationship with our users, we didn’t even know who was using our product. We learned that any time you create an app you should build in feedback mechanisms that can help you learn about your customers and how you can improve your application.
So for the next version of Menuboard and another app in the making, we did a quick research on the solutions we could use for storing mobile app statistics.
We were looking for a logger that is easy to set up and extend, and is preferably hosted. We focused on free or low-cost solutions. Existing loggers didn’t live up to these expectations, they turned out to be expensive and their configuration was too complicated for our needs, so we looked further.
Apache CouchDB
CouchDB is an open source database using JSON to store data and JavaScript as its query language.
As we preferred a hosted solution, we tried Iris Couch with a reasonable pricing of $1/stored GB, 1 cent for 500 reads and 1 cent for 100 writes. Going with this solution, we wouldn’t have had to implement anything, but it might have been difficult to extend it with custom features later.
Iris Couch is a new startup, so we were concerned about the long-time support and maintenance of the product and didn’t want to risk having to do emergency migration and app update to have our statistics, if the company didn’t last.
Checking out the research and opinion of others on CouchDB, we found other concerns that made us vote against this solution:
- "Because batch mode only stores updates in memory until a flush occurs, updates that are saved to CouchDB directly preceding a crash can be lost. By default, CouchDB flushes the in-memory updates once per second, so in the worst case, data loss is still minimal."
source: http://sauceio.com/index.php/2012/05/goodbye-couchdb/
Google App Engine and Heroku
App Engine is a platform as a service solution supporting Java, Python and Go, and is free up to a certain amount of usage per day: 28 instance hours, 1-1 GB inbound and outbound traffic, 50000 reads/writes.
Heroku supports all languages. One instance is free (one web instance in our case), and their dev database holding a maximum of 10000 rows (very small for logging) is also free.
Although these PaaS solutions need a bit more early investment, they seemed to be a better choice for our requirements. They have been around for a while, and they will be available for a long time. We have the flexibility of writing custom data mining solutions and customize logging. Just by starting a worker instance, we can go through any big dataset (that we will hopefully have).
We also like the idea of not having to worry about hosting.
Between the PaaS solutions we prefer App Engine to Heroku, because Heroku would cost more for our use case (Heroku is cheap when you have a small dataset, with a lot of reads).
Our choice
The only advantage of CouchDB over these PaaS solutions is a REST interface, which can be written in a few hours.
Considering our options, we chose to write a small Go service, which runs on top of AppEngine and after a simple authentication, stores the logs in the datastore. Thanks to the built-in Google Apps support, we can easily extend the service with an admin interface and custom data visualization later.
Have you used CouchDB or other logging solutions? How were your experiences?
