Unrelated and Unusual Note: I am devastated seeing India’s helpless COVID-19 second wave situation due to severe Oxygen, medical equipment supply shortage and broken supply chain. I have been in a dilemma whether to publish any article at all since so many people are posting all around to seek help, the overall mood is a bit gloomy. However, it’s part of life and learning should never stop as long as we are healthy. So, here I am with the hope and prayer to see India coming out stronger.
Let’s pray for India together and continue our learning!
When we discuss…
The first article of this series discussed physical time, the second article discussed logical time predominantly vector clock and version vectors, the third article mostly discussed Google True Time and related systems in detail.
By now, we know the following points:
In our last article, we looked into logical clocks especially vector clocks and version vectors deeply and analyzed how they are very useful in real life for designing eventually consistent systems like Riak or Amazon Dynamo DB. But not every eventually consistent systems use vector clock.
Cassandra is a NoSQL distributed wide column based storage system which is also eventually consistent, yet it does not use any logical clock to order events. There is a reason for it:
Key-Value (KV) stores like Riak, Dynamo DB are like big hash tables — they map an immutable key to a value. Dynamo…
In the previous article, we talked about physical clocks in detail, we discussed that in most of the use cases, for a high scale system, using physical clock time is not the right choice unless the time difference across the nodes in a cluster is bounded by an extreme tight limit which is not a feasible option for many companies.
We discuss an alternative option here — logical clocks, monotonic counter-based clocks in short. This article is going to be very comprehensive and practical system design heavy.
Don’t be in hurry, take enough time to digest the content, understanding is…
As a human being, we take birth as a child ( 0th year), start going to school ( ~ 5th year ), university for under graduation ( ~ 18th year ), possibly post-graduation ( ~ 22nd year ), then start our career ( ~ 22nd to 24+ year ) and life moves on. The whole thing happens in a certain order: you probably can’t go to a university without completing school — you may if you are a prodigy, that’s a one-off case though. It takes on average around 20+ years to complete this journey. …
Note: On October 04, 2017, I published a small article on redis hash table internals. But I think, there are lot more details to cover on redis dictionary topic. Hence as an extension to the previous one, I decided to write this new article which is more comprehensive.
Redis is a very well known in-memory ( persistence can also be enabled for fault tolerance ) data structure store which serves usual key value pairs, List, Set, Sorted Set, HyperLogLog, Geo Spatial structures etc. Redis trades off memory to achieve speed, it performs all client operations in a single thread &…
We discussed Raft replication in details in part 2 of this series. That becomes prerequisite for this article. Please head over to part 2 if you haven’t already.
We’ll simulate some scenarios in this article and see how Raft behaves in such cases. It’ll build a better understanding of the algorithms in your mind.
Let’s see the happiest case first. It will help you to get a fair understanding of the life cycle of data replication process in Raft.
The leader node
S2 gets a command from the client. It adds the entry to its own log at index
In the part 1 of this series, we got the basics of Raft. Please go through that part first if you have not already.
In this part, we’ll concentrate on detailed Raft replication technique. The concepts explained here are very important & core to Raft.
The core idea behind any consensus algorithm is that for a given data ( key ), at any point in time, either all or majority of the cluster members should return the same value. Raft uses replication technique to achieve it. Replication has been used to build fault tolerance & redundancy in distributed systems since…
There are many articles on Raft however most of them are short, theoretical, and not detailed. Consensus algorithms is an area where shortcut analysis does not work as the topic is pretty hard to understand. While reading the raft paper and many articles, I found out that no code and not playing with a Raft simulator makes it even harder to understand.
So this series of articles intends to be more natural and lucid from an engineering perspective — it thoroughly explains concepts, core algorithms with code and simulates some important use cases. …
We use real time location search services all the time in our real life — whether it’s a food ordering app or on-demand cab booking service, it’s everywhere nowadays. The intention of this article is to look into how to design the back end infrastructure for a Geo-spatial index in real life.
We will explore how several companies in relevant space have solved the problem, what are the pros & cons of different approaches & how you can approach the problem. Also we will see how to approach the problem in a System & Architecture Design Technical Interview perspective.