Pixable is your mobile photo inbox, where the relevant photos from your friends on Facebook, Twitter and more, come to you. In the same way that Google and Yahoo crawl and index web pages --to add organization and boost relevancy--, Pixable crawls and indexes social photos. But in the case of Pixable, the ranking of photos is fully personalized. As you can imagine, the crawling and indexing requires a scalable backend infrastructure.
In this blog post, Pixable's CTO and VP of Engineering describe how they support a scalable infrastructure with 80+ servers on Amazon Web Services to continuously crawl, process, rank and index new photos from Facebook, Twitter, Instagram and more. The main ingredients are:
- Backend: AWS (with more than 80 EC2 instances), LAMP stack (with a little bit of MongoDB), Memcache layer and Beanstalkd for the job queue.
- Web frontends: CSS5, HTML 5, jQuery with heavy use of ajax calls.
- iPhone/iPad: Objective-C
As there isn't many documented best practices, I hope this post is useful to many tech entrepreneurs that need to build scalable backend infrastructures using today cloud tools and services.