• ChatGPT 3.5

    If you lived in a cave and don’t know what is all the rave about ChatGPT, below here is a self introduction: One of OpenAI founders is Elon, hence the bias. A quick timeline of events Nov 30, 2022 – ChatGPT 3.5 is released by OpenAI to the public. Sam Altman is the CEO of…

    read more…

    categories:

  • Postgresql -> Cockroach DB

    Context: This story comes comes from Storj Labs, a storage provider that boasts resiliency of storage objects and high performance Storj Labs wants to move from Postgresql to CockroachDB. Postgresql is a production grade ACID compliant relational DB that can fit into many use cases such as OLAP, OLTP, GIS, Events Processing. Cockroach DB is…

    read more…

    categories:

  • SLATE Testing At Uber

    SLATE – Short Lived Application Testing Environments Massively successful companies Facebook(Meta) and Uber have sophisticated testing strategies that is non traditional and “works”. The Traditional Approaches are below Uber’s testing strategy is leaning forward more on “online tests” than “offline tests” A separate school of thought is that writing heavy unit tests is useless as…

    read more…

    categories:

    tags:

  • How Netflix Makes Product Decisions?

    consensus decision making is used heavily agnostic to attendees familiarity of the implications top down approach where attendees sometimes may feel unfair. Netflix runs A/B testing to build trust in the results. This is almost a requirement. Across the organisation, there is data-driven mindset. This makes Data & Insights team at the center of the…

    read more…

    categories:

    ,
  • Reduce Logging Costs at Uber with CLP

    Context: The Big Data platform at Uber where logs are generated by hundreds of thousands of Spark Jobs. On peak day 200k TB of data would be generated The cost of maintaining these logs became so high, it would cost Uber $1.3million per year to keep 1 month worth of log, or $180k to keep…

    read more…

    categories:

    tags:

  • OLTP and OLAP at Grab

    Grab processes millions of orders per day. At the rate of their scale, they encountered reliability and cost issues. High availability – even when DB has certain failure, it should always be online with some degraded experience HIgh Write and Read throughput Scalable without high cost Strong consistency for transactional query, and eventual consistencies for…

    read more…

    categories: