Transitioning from MySQL to Cassandra at Chaordic

Chaordic is the leader in big data based personalization solutions for e-commerce in Brazil. The largest online retailers in the country, such as Saraiva, Walmart and Centauro use our solutions to suggest personalized shopping recommendations to their users. Chaordic’s Data Platform provides a common data layer and services shared among multiple product offerings. Three years ago Chaordic was experiencing an exponential growth. From the beginning, our business was based in gathering as much as relevant information as we could collect. I’ve joined the company when the core database solution - MySQL - was struggling to keep the pace of growth....…

read more...

Optimizing for Cost in the Cloud and AWS

Scaling a cloud operation requires a great deal of design choices, engineering practices, and picking the right set of technologies. Critical to being competitive while scaling is also being cost-efficient. Specially for complex cloud based offerings, infrastructure costs can represent a significant part of the company expenses. At Chaordic, we achieved a factor of 3 of improvement in cost efficiency metrics in the last two years, and this post describes some of the techniques we employed to succeed. To begin, being able to actually know your costs is the basis for intelligent decision making. While this is somewhat obvious, the...…

read more...

Start using Spark with Ignition!

Apache Spark is a cluster computing framework designed to provide ease of use, fast processing and general-purpose pipelines when compared to traditional systems like Apache Hadoop. Here at Chaordic we are using it to build scalable products, like Personalized Emails, replacing a stack comprising of Hadoop Jobs, databases and services like Amazon Dynamo, MySQL, SQS, Redis and internal systems built on Jetty. Even Spark being easy to use and very powerful, we’ve learned better ways to use Spark and condensed them in a open-source project called Ignition, consisting of: Ignition-Core, a library with Spark and Scala goodies, including a command-line...…

read more...

Saldão Black Friday 2014

No último fim de semana tivemos o maior evento do e-commerce brasileiro, a Black Friday. Enquanto você se divertia na caça por promoções, a gente na Chaordic também se divertia mantendo as engrenagens girando para fornecer a melhor experiência de compra aos consumidores, promovendo o encontro entre eles e os produtos que mais lhes interessam. Somente na sexta-feira registramos mais de 750 milhões de interações com nossos serviços partindo de 12,5 milhões de usuários. Isso representa aproximadamente 4 vezes o valor medido na sexta da semana anterior. Apesar disso, nossos servidores responderam em 55 milissegundos durante praticamente todo o evento....…

read more...

Aguentando porrada na Black Friday 2014

As vitrines de personalização da Chaordic estão integradas nos maiores e-commerces do país, o que na prática significa que cerca de 40% dos pedidos do e-commerce nacional passam pelos nossos sistemas. Atender a essa enorme quantidade de requisições com qualidade é um grande desafio, especialmente em época de Black Friday. Em 2011 a Chaordic atingiu um pico de 70 mil requisições por minuto (RPMs) durante a Black Friday. Em 2012 foram cerca de 200 mil RPMs e em 2013 foram 313 mil RPMs (mais de 5 mil requisições por segundo). A figura abaixo mostra o pico de RPMs nas últimas...…

read more...

Chaordic @ Cassandra Summit 2014

At Chaordic we have been using Apache Cassandra to store data at scale since 2012, when we faced exponential growth and migrated from MySQL. Since then Cassandra is a key technology here, allowing us to scale from a few hundred million to tens of billions requests per month, and growing… In this post we will share some of our experiences at the Cassandra Summit, held in San Francisco from September 10th to 13th this year. well, erm... that's me at #CassandraSummit 2014 The conference had over 2000 participants from around the globe and awesome talks from leading companies in many...…

read more...