-
Water Fasting 7 Days and Beyond
Disclaimer – the information provided is not expert advice. As all of us have different body types, whatever worked for me may not work for everyone. If you’re fasting, be conscious of how you feel. Context Today marks the 7th day of my water fasting “experiment.” I’ve always been curious about fasting for long periods…
-
Data Mesh @ Netflix
Data Mesh platform – an in-house built solution for handling Change Data Capture design patterns for processing data from a source to a target Seems to be a real time type of processing job in comparison batch job compared to Hadoop Hive or Spark which achieves similar use cases related to ETL pipelines. However the…
-
Git Internals Part I
the blog comes from Github written by Principal Software Engineer Derrick Stolee. jfyi – git was originally created by your favourite linux guy, Linus Torvalds for him to help manage version control himself for the Linux Kernel lots of technical things here, the blog summarizing a lot of things. I am sure the technical implementations…
-
MemQ Pub/Sub + Kafka @ Pinterest
The story comes from Pinterest in Mid 2020 that used Kafka to publish Logging data to interested parties. Data Flow Systems generating data –> Producer Logging Platform –> Pub/Sub (Kafka) –> Consumer Logging platform–> Extract data from related systems Separating storage and serving components which may have different parameters and requirement Custom replications and rebalancing…
-
Staff Software Engineer Archetypes
Staff Engineers is closer to the technical career track in contrast to similar leadership position that is managerial track for engineers as they advance their career. It is typically the split road for the next level of senior software engineers should they decide to stay on the technical track. This role is one of the…
-
ChatGPT 3.5
If you lived in a cave and don’t know what is all the rave about ChatGPT, below here is a self introduction: One of OpenAI founders is Elon, hence the bias. A quick timeline of events Nov 30, 2022 – ChatGPT 3.5 is released by OpenAI to the public. Sam Altman is the CEO of…
-
Postgresql -> Cockroach DB
Context: This story comes comes from Storj Labs, a storage provider that boasts resiliency of storage objects and high performance Storj Labs wants to move from Postgresql to CockroachDB. Postgresql is a production grade ACID compliant relational DB that can fit into many use cases such as OLAP, OLTP, GIS, Events Processing. Cockroach DB is…
-
SLATE Testing At Uber
SLATE – Short Lived Application Testing Environments Massively successful companies Facebook(Meta) and Uber have sophisticated testing strategies that is non traditional and “works”. The Traditional Approaches are below Uber’s testing strategy is leaning forward more on “online tests” than “offline tests” A separate school of thought is that writing heavy unit tests is useless as…
-
How Netflix Makes Product Decisions?
consensus decision making is used heavily agnostic to attendees familiarity of the implications top down approach where attendees sometimes may feel unfair. Netflix runs A/B testing to build trust in the results. This is almost a requirement. Across the organisation, there is data-driven mindset. This makes Data & Insights team at the center of the…
-
Reduce Logging Costs at Uber with CLP
Context: The Big Data platform at Uber where logs are generated by hundreds of thousands of Spark Jobs. On peak day 200k TB of data would be generated The cost of maintaining these logs became so high, it would cost Uber $1.3million per year to keep 1 month worth of log, or $180k to keep…
-
Subscribe
Subscribed
Have a WordPress.com account? Log in now.
