Skip to main content

Introduction

Hi, I am kapil garg and i have created this blog to post anything i learn or find interesting in day to day life. You may find something to learn here as well. I am currently an employee at Flipkart Pvt. Ltd. working as a software engineer. Most of my work revolves around huge amount of data processing and event streaming. I take help of Apache Spark and Apache Kafka to handle the data and all the events that come my way. I don't intend to teach anything in this blog so you might find my posts a little rough but that's okay. If you wish to only learn something then i would suggest you to checkout my other blog fetch-info. I have also worked in Samsung Semiconducter as a software engineer and there i worked mostly on database designs and SSD frameworks.  

Currently i am focusing on Apache kafka and i will try to contribute to the project. Also there are some personal projects going on which are not noteworthy right now but if i learn something doing those, i will post it. I guess that's enough for the first post. Lets see what happens tomorrow.

Labels:
Location: India

Comments

Post a Comment

Popular posts from this blog

Why applications are moving to decentralized network ?

Fig 1. Decentralized Network Recently, I started studying blockchain in order to get certified from blockchain-council. I am in the middle of my training and have started looking into some real-world applications of blockchain. Some problems the applications are trying to solve are decentralized storage, decentralized voting/consensus, decentralized money and many more. The ultimate goal is to take whatever we have built so far and put it over a decentralized network. To achieve this developers are building  custom applications on top of bitcoin blockchain. When I thought about this goal, I couldn't help but think that is this not a waste of time. We spent last 30 years to build the centralized applications and now people are building something from the scratch to put the centralized applications over a decentralized network. Are people doing it just for fun or just because they can so they are. I will share what have I learnt from my 20 days of training, blogs, articles and whit...
Post a Comment
Read more

Kafka consumers

  Today i watched one video on kafka consumers presented by Igor Buzatoic at kafka summit. He demonstrated the working of kafka consumers, group re-balancing and multi-threading environment. The complete video should become available on youtube soon. So the presentation was divided basically in 2 sections - Single threaded mechanism - Multi threaded mechanism Single Thread The idea here is that a single thread is responsible to poll the records from kafka and do the processing of those records. Now here the offset commit step can either be executed manually after the processing is done or periodically if we are sure than our delay in processing is less than commit interval. Usually people go with manual commit only as this ensures that we are committing what we have already processed. Now this is simple architecture and doesn't require any synchronization. Multi Thread In multi threaded environment, there are 2 possibilities - Single thread responsible for polling the records, proc...
Post a Comment
Read more

Decentralized Applications

DApp: Decentralized Application In the previous post, I discussed why people should move to decentralized applications. In this post, it would be better to understand what a decentralized application is and what is the backbone of all major DApps in the market. To understand a decentralized application, you will have to understand the decentralized network. In a decentralized network, there is no single entity governing the operations of the network. All machines/nodes participating in the network are equally responsible for the state and security of the network. In the above diagram, as you can see, nodes A, B and C are the ones keeping the network running. Those nodes are running the application and no single node can modify the behavior of the application for the connected clients. This means that even when node A stops working and its connected clients connect to node B or node C, they don't notice a difference. When we talk about decentralized network, then we implicitly mean ...
Post a Comment
Read more