All posts by sid

Best ergonomic mouse for your workstation 2023

Most of us are working from home and I know how hard it is to work without a proper workstation setup. Also, working from home without following proper ergonomics could be quite harmful to your overall posture. Finding proper ergonomics equipment could be quite challenging sometimes and with this blog, I would like to recommend every other avid laptop/computer user to switch to an ergonomic mouse to get relief from the pain in the wrist, arms, hands, etc while working for prolonged hours. Also if you’re planning to buy a mouse go with an ergonomic mouse and thank me later!

After doing a lot of research on ergonomics here are my top two recommendations:

1. logitech MX Master 3S

No doubt this is one of the best in the market and I understand its price is a little bit on the higher side but let me tell you it’s actually worth it. After doing my research I just ordered this beast today and waiting to receive it by end of the week. I’ll update my review after using it for a month or so.

Some really cool things which I like about this mouse are listed below:

  1. It’s premium built quality.
  2. You can customize every other button to function differently.
  3. You can customize gestures.
  4. It’s extremely comfortable.
  5. Compatible with multiple devices at the same time. You can seamlessly switch between three devices at max at the same time.
  6. USB-C quick charging (after a full charge it can last up to 70 days)
  7. 12 months Warranty.

I mean what else do you want? Huh! Just go for it if you work on multiple devices for ~8-9 hours, care about your wrist and ergonomics, and also if you have the budget 🙂

Buy from amazon:

2. Anker 2.4G Wireless Vertical Ergonomic Optical Mouse

This vertical mouse from Anker is a cheaper option as compared to MX Master 3 but this is as good as MX based on ergonomics. You’ll be relieved of your wrist pain using this mouse. On a sale, in Amazon, you can even get it for as cheap as 2k which is a very reasonable amount for the functionalities you get.

  1. Affordable/Budget-friendly.
  2. Comes with automatic standby.
  3. Non-rechargeable batteries. (2 AAA batteries work for almost ~40 days)
  4. 18 months Warranty.

Buy from amazon:

Top 5 Laptops You Can Get In India For Under 55,000


If you’re looking for a powerful laptop without investing too much money, our list of the top five best laptops in India under 55000 offers great options. With a budget of 55000, you can get a laptop that is powerful enough to handle most tasks and gaming. We have compiled a list of the top 5 laptops you can get in India for under 55000. This list includes laptops from brands like HP, Acer, Lenovo, and Asus. So, if you are looking for a new laptop, make sure to check out this list!

What to look for when choosing a laptop

When looking for the best laptop under 55000, there are a few key features to consider. Firstly, check the processor – the faster the better, meaning higher-end Intel Core processors will give you the best performance. Secondly, check the RAM – having at least 8GB of RAM is recommended to ensure that you can multitask without any issues. Thirdly, look at the display – most laptops will come with Full-HD displays, though paying a bit more may get you a 4K display. Lastly, consider the battery life – a longer-lasting battery will ensure you can use your laptop for a longer period of time.

The top 5 laptops under 55,000 in India

Now that you know what to look for in a laptop, here is a list of the top 5 laptops you can get in India for under 55000. The following laptops offer great performance, plenty of storage, and comfortable ergonomics.

1. Lenovo IdeaPad Slim 3

The Lenovo Ideapad 3 is a great laptop for under 55000. It has a 15.6-inch FHD display, an Intel Core i5 processor, and 16GB of RAM. It also has 512GB of storage for all your files, and up to 5 hours of battery life. This also comes with built-in Alexa!

2. ASUS Vivobook Go 15 (2023)

The Asus Vivobook Go 15 (2023) is an excellent device for users seeking a powerful laptop. With a premium 15.6-inch FHD display, AMD Ryzen 5 7520U Mobile processor, 16GB of RAM, and 512GB of storage, this laptop offers impressive performance in a portable package. It also features up to 6 hours of battery life, making it the ideal choice for on-the-go use.

3. HP Pavilion 14

The HP Pavilion 14 laptop offers an impressive performance at an affordable price. It features a 14-inch FHD Micro-Edge display, a Ryzen 5 processor, and 8GB of RAM, allowing you to multitask with ease. Additionally, this laptop boasts 512GB of storage and up to 6.5 hours of battery life, providing plenty of power for everyday tasks.

4. Acer Aspire 5

The Acer Aspire 5 is a suitable laptop for users looking for comprehensive features within a budget of Rupees 55000. This laptop comes with a 15.6-inch FHD display, a 12th Gen Intel Core i5 processor, 16GB RAM, 512GB of storage, and up to 6.5 hours of battery life. It offers good value for money at this price point.

5. Dell Inspiron 3520

The Dell Inspiron 3520 offers incredible value for its price point. It has a 15.6-inch Full High Definition display, the latest Intel Core i5 processor, 8GB of RAM, and 512GB of storage. On top of that, it provides users with up to 5.5 hours of battery life. The laptop also has exceptional build quality and is ruggedly durable. It’s outfitted with an excellent keyboard and trackpad combination which makes it comfortable to work on and simple to use.


The best laptops in India under 55000 are a great way to get your hands on a powerful machine without spending too much money. Each of these laptops offers great performance and plenty of storage, making them perfect for everyday use. So, make sure to check out our list before you make your purchase!

The Top 5 Smartwatches You Should Be Looking Forward To In 2023

When it comes to wearable technology, smartwatches are at the top of the list for many people. While there are many different types and brands of smartwatches available on the market today, finding the best one for you can be a challenge. With so many options to choose from, it can be difficult to know where to start.

To help you narrow down your search, we’ve compiled a list of the best smartwatches for 2023. Whether you’re looking for a smartwatch with the latest features or one that’s more budget-friendly, we’ve got you covered.

What to look for in a smartwatch:

When searching for the perfect smartwatch, there are a few important factors to consider. The battery life of the watch, the operating system it runs on, the design, and the features it offers are all important factors. You should think about what type of features you need and choose a watch that meets those needs.

Battery life

The battery life of the watch is an important factor to consider. Most modern smartwatches have batteries that will last for days, but some have batteries that will only last for a few hours.

Operating System

It is also important to consider the operating system it runs on. Most of the popular smartwatches run on either Android Wear or Apple’s watchOS. These operating systems provide access to hundreds of different apps and features.


The design of the watch is another important consideration. Do you prefer a more traditional watch design or would you prefer a modern, touchscreen-based design?

Other features

The features the watch offers are equally important. Most smartwatches offer basic features like step tracking and notifications, but some offer more advanced features like GPS, heart rate monitoring, and music playback. You should think about what features you will use most and look for a watch that has those features.

The Top 5 Smartwatches of 2023:

1. Apple Watch Series 8: The Series 8 is the latest model from Apple and it’s sure to be a hit in 2023. It features a gorgeous OLED display, loads of new health and fitness features, and of course, the ever-popular Apple ecosystem.

2. Samsung Galaxy Watch 5: Samsung’s Galaxy Watch 5 is expected to be one of the most feature-packed smartwatches of 2023. It is likely to feature an even slimmer design than its predecessor, along with a larger battery and improved fitness tracking capabilities.

3. Fitbit Versa 4: The Versa 4 is the successor to the popular Versa 3 and is expected to feature an improved battery life, better fitness tracking capabilities, and even an onboard GPS.

4. Android Wear Pixel: Google’s Android Wear Pixel is expected to be released in 2023 and is likely to feature a sleek design, an updated operating system, and an improved battery life.

5. Garmin Venu 2: Garmin’s Venu 2 Watch is released in 2023 and it has a new sleek design, very good battery life, and loads of new features.


Smartwatches are becoming more popular every year and 2023 is no exception. These five smartwatches are sure to be the best of the best when it is released and any one of them would make a great addition to your collection. Be sure to consider your needs and budget when looking for a new smartwatch and you’ll be sure to find the perfect one for you. Thanks for reading and happy shopping!

Best headphones under Rs.2000 in India 2021

This post will talk about the best wireless headphones available to order in India under 2000. I have listed down 4 such headphones all with some pros and cons.

1. OnePlus Bullets Wireless Z

There are two variants to it first one is the BMZ version and the second one is the BWZ-Bass Edition. The BMZ-Bass Edition has better bass than the BMZ version but it has less playtime by 3 hours. BMZ-Bass Edition version is good for people who love extra Bass and the regular BMZ version is good for extended calls as it has 20 hours of playtime. If you have a OnePlus smartphone that supports Dolby Atmos (7/7 Pro/7T/7T Pro/8/8 Pro devices) you can even enable that feature to get better sound output from the headphones.

Battery life ~17/20 hours | Fast charging wrap | Passive noise cancellation | USB Type-C | IP55 (Sweat and water-resistant) | Magnetic Control | Bluetooth version: 5.0

BMZ Edition –

BWZ-Bass Edition –

2. Realme Buds Wireless

These in-ear bluetooth earphones from Realme are the cheaper options as compared to one plus in the list. If you like a bit of bass boost then these headphones are for you. It comes with a battery life of 12 hours which is less than OnePlus Bullets Wireless Z but its bass is a bit on the higher side as compared to OnePlus Bullets Wireless Z.

Battery life ~12 hours | Fast charging | Bass Boost | Micro USB | quick Magnetic Control | Bluetooth version: 5.0

Realme Buds Wireless


This is our very first on-the-ear headphones on the list. These headphones come with decent sound quality with Dual Equaliser Modes for Normal & Deep Bass Output and with quick charging. If you’re looking for lightweight wireless on-the-ear headphones that can be used for long hours then you can go for it.

Battery life ~20 hours | Quick charging ~2 hours | Lightweight | Dual Equaliser | Deep bass | No active noise cancellation | No 3.5 mm cable


4. BOAT Rockerz 510

These stylish-looking on-the-ear headphones by boAt come with very powerful bass and very good sound quality. Its playtime is of about 20 hours. It’s quite comfortable to wear for long hours while coding or playing games.

Battery life ~20 hours | Stylish looks | Comfortable | Thumping bass | No active noise cancellation | 3.5 mm cable

BOAT Rockerz 510:

Affordable Monitor setup for Work from Home 2021

While I was setting up my work from setup I was looking for a monitor that fits in the budget, is sturdy, and meets all the following specification:

  1. The display size should be greater than 24 inches.
  2. Full HD 1920 x 1080 Resolution or better.
  3. The refresh rate should be better than 70 Hz (I play PC games)

My search ended up with buying the Acer 27-inch KG271 Cbmidpx:

This monitor comes with a 144 Hz Refresh Rate, 27 inch Full HD 1920 x 1080 Resolution with Display Port and HDMI cables included and I got it for a very good deal from Amazon (16k). 

One cool thing about this monitor is it can be mounted on a wall or a table and for that one can buy a Height Adjustable Arm Mount which can be affixed with the table or wall so that one can move/rotate the monitor freely.

In my case I bought a Monitor arm which can be affixed with the table and my search ended up with buying the AmazonBasics Monitor Stand, Height Adjustable Arm Mount- Steel:


Monitor Stand:

Bonus Content:

This is how my current work from home setup looks like:

I did a fair amount of research on ergonomics before ordering the chair and it works in my case i.e. I did not feel any back pain while sitting on it for hours 🙂


Laptop Stand:


Extended Mouse pad:

I’m using a MacBook pro which only has USB-c ports and in order to connect my MacBook to the monitor instead of buying an expensive dock, I bought one handy USB-C adapter which has a display and HDMI port to connect to the monitor.

I hope this helps somebody looking to set their work from home setup and using a MacBook. Cheers!

Data Lakehouse: a new paradigm for Modern Data Platforms?

Modern Data platforms consider using a data lake over a data warehouse for their storage requirements. However, a data lake lacks in providing support for business intelligence use cases and often organizations end up having a separate data warehouse for their business intelligence use case and operational users. This article explains the different paradigms organizations follow and how Lakehouse solves the problems of both the worlds.

Let’s first start with the difference between a Data lake and a Data warehouse.

Data Lake v/s Data Warehouse

Let me explain the difference using a simple analogy:

Think of a data warehouse as a store of packaged drinking water and data lake as a body of natural drinking water.

The packaged drinking water is directly consumable because the water has been cleaned/filtered and packaged. But in order to consume water from a lake, you have to process/filter it before you can use it. Also, only users who know how to open a bottle will be able to consume water from the bottle. But in the case of the lake even animals can consume water from it. Water in the lake is in its rawest form whereas water in the packaged bottles is processed.

Water in the lake can be used for a variety of different purposes but nobody uses bottled water for cleaning the car and irrigation etc. Bottled water is used to drink most of the time.

The above points clarify a few things:

  • Drinking water from bottled water requires some expertise or in other words consuming data from a data warehouse requires some expertise and is meant for a few users, implying that the user base of a data warehouse is limited. In the case of a lake, anybody can consume water from it after doing their own filtration. Similarly, data lake has a vast user base and is not limited to a few users.
  • The usage of bottled water is pre-decided (most of the time it will be used for drinking) but the usage of lake water depends on its user. This implies that the purpose of lake water is decided as and when somebody wants to consume water from it whereas a lot of pre-processing has already been done to design the data warehouse. This ties to the distinction between schema on write vs schema on read approach followed by data warehouse and data lake respectively.
  • The bottled water is preprocessed but water in a lake requires some cleaning before it becomes consumable and the onus is on individual users to clean it.
  • Water in the bottle is confined to some structure whereas water in the lake is in different formats and might be coming from a variety of streams as well. This implies that data lake supports all data types whereas data warehouse only supports structured data.

Now you might be thinking with all the points mentioned above that data lake is the way to go as it supports all types of data: structured, semistructured, and unstructured along with a vast majority of users whereas Data warehouses only support structured information and have a small user base.

While data lake is good for data science and machine learning use cases as it provides all sorts of data for deep analysis, it misses out on the operational use case cases where the data should be well structured to get the key metrics and reports.

And what if you want to derive some business intelligence from the data in your data lake? You probably will load the data from the data lake into any data warehouse for the business intelligence use case. And a lot of organizations do that, but in that case, they end up managing two different siloed storage systems: one for BI use cases and another for use cases such as Data Science, Machine Learning, etc.

Data Lake and Data Warehouse

New modern organizations have data management platforms which combine both data lake and data warehouse in a single entity by performing ETL/ELT on the raw data of data lake and define some structures on it.

So far all good no need to maintain two separate storage entities and now your data warehouse sits inside your data lake which acts as a landing place to ingest raw data. A data warehouse is the portion of the data lake where we enrich and structure the data so that it can be queried directly.

This paradigm copes up with the challenges of having two separate storage systems but we still need to maintain the ETL/ELT layer.

Data Lake + Data Warehouse = Data Lakehouse

But wait there is another paradigm known as Data Lakehouse that is trending nowadays and is bringing the best of both paradigms (Data Lake and Data Warehouse). In simple words, Data Lakehouse enables you to perform queries for BI/reports use cases directly on the data stored in the data lake without having to load it in a separate data warehouse. The processing/structural layer is responsible for making that happen.

This approach saves a lot of operational costs as you don’t have to manage two storage systems simultaneously. It also eliminates the ETL/ELT layer which transforms data from the data lake and loads in the data warehouse as you can now directly have your query engine querying data directly from the lake.

This paradigm helps organizations to move faster and make better decisions.

The processing or structured layer could be AWS Athena, Delta lake, etc. I personally like Delta lake for this kind of functionality as it provides ACID transactions on top of your data lake which enables you to perform some business-critical actions like GDPR DSR requests etc.

Final thoughts: Deciding on which paradigm to use should be a conscious decision and it mostly depends on your use case and how your organization operates.

Spring Boot + Apache Kafka

This post will guide you to create a simple web application using Spring Boot and Apache Kafka. 

Apache Kafka is a distributed streaming platform which is well known for moving data between systems in a distributed way.

Spring boot provides out of the box integration with Apache Kafka so that you don’t have to write a lot of boiler plate code to integrate Kafka in your web application. 

Let’s get started!


The only prerequisite is to have Kafka up and running in your environment. And to do that follow the links based on what operating system you’re using:

  1. Install Kafka on windows
  2. Install Kafka on macOS

I generally prefer running all my system dependencies using docker which is an awesome tool for developers. Docker enables developers to focus on writing code and not worry about the system it will run on. If you want to run Kafka using docker, you can use the docker-compose.yaml file that I have added in the github repo. So before running the application just copy the file and perform a docker-compose up -d which will spin up a single Kafka broker, Zookeeper and will also create the required topic. This means you don’t need to perform step 6.

Steps to create Spring Boot + Apache Kafka web application:

Follow the below steps to create a Spring Boot application with which you can produce and consume messages from Kafka using a Rest client.

1) Creating the Web Application Template:

We’ll be using Spring Initializr to create the web application project structure and the easiest way to use Spring Initializr is to use its web interface.

1.1) Go to

1.2) Enter Group and Artifact details:

1.3) Add Spring Web & Spring for Apache Kafka dependencies:

Then, click on Generate Project

This will generate and download the file which is your maven project structure.

1.4) Unzip the file and then import it in your favourite IDE.

After importing the project in your IDE (Intellij in my case), you’ll see a project structure like this:

2) Configure Kafka Producer and Consumer:

We can configure the Kafka producer and consumer either by creating configuration classes (annotating classes with @Configuration annotation) for both producer and consumer or by using file to configure them. In this tutorial, I’ll be demonstrating both for integrity. I personally prefer the latter approach ( as it is quite handy and doesn’t require all the boilerplate code of the configuration class and that’s the beauty of spring boot one should leverage. 

2.1) Creating configuration classes for Kafka consumer and producer:

Creating the consumer config class:

Create a class in package com.technocratsid.kafkaspringapp.config with the following content:

package com.technocratsid.kafkaspringapp.config;

import java.util.HashMap;
import java.util.Map;

import org.apache.kafka.common.serialization.StringDeserializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;

public class ConsumerConfig {

    private String kafkaServers;

    private String groupId;

    public ConsumerFactory<String, String> getConsumer() {
        Map<String, Object> configProp = new HashMap<>();
        configProp.put(org.apache.kafka.clients.consumer.ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaServers);
        configProp.put(org.apache.kafka.clients.consumer.ConsumerConfig.GROUP_ID_CONFIG, groupId);
        configProp.put(org.apache.kafka.clients.consumer.ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        configProp.put(org.apache.kafka.clients.consumer.ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        return new DefaultKafkaConsumerFactory<>(configProp);

    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
        return factory;


Note: Here we are declaring ConsumerFactory and ConcurrentKafkaListenerContainerFactory as beans (using @Bean annotation). This tells the spring container to manage them for us. ConsumerFactory is used to specify the strategy to create a Consumer instance(s) and ConcurrentKafkaListenerContainerFactory is used to create and configure containers for @KafkaListener annotated methods.

@Configuration is used to tell Spring that this is a Java-based configuration file and contains the bean definitions.

@EnableKafka annotation tells Spring that we want to talk to Kafka and allows Spring to detect the methods that are annotated with @KafkaListener.

@Value annotation is used to inject value from a properties file based on the property name.

The file for properties spring.kafka.bootstrap-servers and spring.kafka.groupId is inside src/main/resources and contains the following key value pairs:


If you wish to run the application with a remote Kafka cluster then edit spring.kafka.bootstrap-servers pointing to your remote brokers.

Creating the producer config class:

Create another class in the same package com.technocratsid.kafkaspringapp.config with the following content:

package com.technocratsid.kafkaspringapp.config;

import java.util.HashMap;
import java.util.Map;

import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.core.ProducerFactory;

public class ProducerConfig {

    private String kafkaServers;

    public ProducerFactory<String, String> getProducer() {
        Map<String, Object> configProp = new HashMap<>();
        configProp.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaServers);
        configProp.put(org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        configProp.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        return new DefaultKafkaProducerFactory<>(configProp);

    public KafkaTemplate<String, String> kafkaTemplate() {
        return new KafkaTemplate<>(getProducer());


Note: Here we are declaring ProducerFactory and KafkaTemplate as beans where ProducerFactory is used to specify the strategy to create a Producer instance(s) and KafkaTemplate is a template for executing high-level operations like sending messages to a Kafka topic etc.

The value of kafkaServers is injected from the property spring.kafka.bootstrap-servers of the file same as ConsumerConfig class.

2.2) Configuring the Kafka consumer and producer using

Open the file inside src/main/resources and add the following key value pairs:



That’s it!

The above properties will configure the Kafka consumer and producer without having to write a single line of code and that’s the beauty of spring boot.

The key value pair in (allows to specify configuration) allows spring to do different things. With this file you can tell spring to configure things without writing any code.

3) Creating a consumer service:

Create a class in package com.technocratsid.kafkaspringapp.service with the following content:

package com.technocratsid.kafkaspringapp.service;

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Service;

import java.util.ArrayList;
import java.util.List;

public class KafkaConsumer {

    public static List<String> messages = new ArrayList<>();
    private final static String topic = "technocratsid-kafka-spring";
    private final static String groupId = "kafka-spring-app";

    @KafkaListener(topics = topic, groupId = groupId)
    public void listen(String message) {

@KafkaListener allows a method to listen/subscribe to specified topics.

In our case whenever a message is produced on the topic “technocratsid-kafka-spring“, we are adding that message to a List of String (which is adding stuff to memory and it is not a good practice, in real world you might consider writing the messages to some datastore) so that we can display the messages later.

@Service tells Spring that this file performs a business service.

4) Creating a producer service:

Create a class in package com.technocratsid.kafkaspringapp.service with the following content:

package com.technocratsid.kafkaspringapp.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;

public class KafkaProducer {

    private KafkaTemplate<String, String> kafkaTemplate;

    private String topic;

    public void produce(String message) {
        kafkaTemplate.send(topic, message);


Here we are using the KafkaTemplate send method to send messages to a particular topic.

@Autowired tells Spring to automatically wire or inject the value of variable from the beans which are managed by the the spring container. So in our case the value of kafkaTemplate is injected from the bean kafkaTemplate() defined in ProducerConfig class.

@Service tells Spring that this file performs a business service.

5) Creating a rest controller:

Create a class in package com.technocratsid.kafkaspringapp.controller with the following content:

package com.technocratsid.kafkaspringapp.controller;

import com.technocratsid.kafkaspringapp.service.KafkaConsumer;
import com.technocratsid.kafkaspringapp.service.KafkaProducer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

public class KafkaController {

    private KafkaConsumer consumer;

    private KafkaProducer producer;

    @RequestMapping(value="/send", method= RequestMethod.POST)
    public void send(@RequestBody String data) {

    @RequestMapping(value="/receive", method=RequestMethod.GET)
    public List<String> receive() {
        return consumer.messages;

This class registers two endpoints /send and /receive which sends and receive messages to and from Kafka respectively, where /send is a POST request with String body and /receive returns the sent messages.

6) Create the required Kafka topic:

Before running the application create the following topic:

kafka-topics --bootstrap-server localhost:9092 --topic technocratsid-kafka-spring --create --partitions 1 --replication-factor 1

7) Running the web application:

Either run the KafkaSpringAppApplication class as a Java Application from your IDE or use the following command:

mvn spring-boot:run

8) Testing the web app with a REST client:

To test the Spring Boot + Apache Kafka web application I am using Insomnia REST Client which is my favourite Rest client because of its simple interface. You can use any REST client.

Once your application is up and running, perform a POST request to the URL http://localhost:8080/send with the body {“key1”: “value1”}:

Then, perform a GET request to the URL http://localhost:8080/receive and this is the response you’ll get:

Congratulations on building a Spring Boot + Apache Kafka web application.

Hack into the github repo to see the complete code.

Install Kafka on Windows

This post is a step-by-step guide to install and run Apache Kafka on Windows.


The only prerequisite for this setup is JRE.

Install Java (Skip if you already have it)

  1. Download Java 8 from here. (Java 8 is recommended by Apache Kafka)
  2. Run the installer and follow the instructions on Installation wizard.
  3. Please note/copy the Destination Folder location.
  4. Go to Control Panel -> System -> Advanced system settings -> Environment Variables.
  5. Create a new user variable named JAVA_HOME and paste the path copied from step 3 to the variable value and click OK.
  6. Now edit PATH variable in User variables and add “%JAVA_HOME%\bin” at the end of the variable value. If it’s an older windows version, then add “;%JAVA_HOME%\bin;” at the end of the text. If PATH variable doesn’t exist create it with the value “%JAVA_HOME%\bin”.
  7. Open command prompt and type “java -version” to validate the installation.
  8. If you get the following output in your command prompt you’re good to go:
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)

Steps to install Kafka on windows:

1) Download and extract Apache Kafka from here.

Note: At the time of writing this post the current stable version is 2.4.0. If you’re using Scala for development then make sure to select the Kafka version corresponding to your Scala version.

2) Go to the folder where you’ve extracted Kafka, open the file in the config folder and then edit the line log.dirs=/tmp/kafka-logs to log.dirs=C:\path_where_kafka_is_extracted\kafka-logs.

Note: If you won’t change log.dirs value, you’ll keep getting the following error:

java.util.NoSuchElementException: key not found: /tmp/kafka-logs

3) Start Zookeeper

Kafka uses ZooKeeper so before starting Kafka you have to make sure that ZooKeeper is up and running. We’ll be running the single node ZooKeeper instance packaged with Kafka. 

Go to your Kafka installation directory and open command prompt and start Zookeeper using the following command:

> bin\windows\zookeeper-server-start.bat config\

Once zookeeper is started you should see this in the command prompt:

INFO binding to port (org.apache.zookeeper.server.NIOServerCnxnFactory)

4) Start Kafka Server

You need to open another command prompt in the Kafka installation directory and run the following command:

> bin\windows\kafka-server-start.bat \config\

Once the Kafka server is started you should see something like this in the command prompt:

INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

5) Validate the Kafka setup

Create a Kafka topic:

Open a command prompt in the Kafka installation directory and run the following command:

> bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic

The above command will create a topic “testTopic” with 1 partition and 1 replication factor.

Produce messages to the created topic:

In the same command prompt start a kafka-console-producer and produce some messages:

> bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic testTopic
>How are you?

Consume the messages produced by the kafka-console-producer:

Open another command prompt in the Kafka installation directory and start a kafka-console-consumer which subscribes to testTopic:

> bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic testTopic --from-beginning
How are you?

The messages which are produced by the kafka-console-producer will be visible on the kafka-console-consumer console.

Congratulations on completing a single broker Kafka setup on windows!

Install Kafka on macOS

I really like Homebrew to install stuff on macOS. So in this tutorial I’ll be using Hombrew to install Apache Kafka on macOS.

Install Homebrew:

Open Terminal, paste the following command and then press Return key:

$ /usr/bin/ruby -e "$(curl -fsSL"

The above command will install Homebrew. Once it is installed, follow the next steps:

1)  Install Java8:

After Homebrew is installed, you can just paste the following commands in Terminal and press Return to install java on macOS:

$ brew tap adoptopenjdk/openjdk
$ brew cask install adoptopenjdk8

The above commands will install openjdk 8. In order to validate the java installation, execute the following command in Terminal:

$ java -version

If you get the following output, it means that java is installed correctly:

openjdk version "1.8.0_212"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b04, mixed mode)

2) Install Kafka:

To install Kafka on macOS: open Terminal, paste the following command and then press Return:

$ brew install kafka

The above command will install Kafka along with Zookeeper.

Start Zookeeper:

To start ZooKeeper, paste the following command in the Terminal and press Return:

$ zookeeper-server-start /usr/local/etc/kafka/

Once ZooKeeper is started you’ll see something like this in the console:

INFO binding to port (org.apache.zookeeper.server.NIOServerCnxnFactory)

Start Kafka:

To start Kafka, paste the following command in another Terminal and press Return:

$ kafka-server-start /usr/local/etc/kafka/

Once Kafka is started you’ll see something like this in the console:

INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

Create a Kafka topic:

Open another Terminal and execute the following command to create a topic testTopic with 1 partition and 1 replication factor:

kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic

Produce messages to the created topic:

In the same Terminal prompt, start a kafka-console-producer and produce some messages:

$ kafka-console-producer --broker-list localhost:9092 --topic testTopic
>How are you?

Consume the messages produced by the producer:

Open another Terminal and start a kafka-console-consumer which subscribes to testTopic:

$ kafka-console-consumer --bootstrap-server localhost:9092 --topic testTopic --from-beginning
How are you?

The messages which are produced by the kafka-console-producer will be visible on the kafka-console-consumer console.

Congratulations on setting up a single broker Kafka setup on macOS!

Exploring Twitter Data using Elasticseach & Kibana’s Canvas

After thinking a lot about what to write next, I stumbled upon a very cool idea.

And this is what I thought I should do: Discover something interesting using code ?

Let’s find out what’s the most popular drink among tea, coffee and beer in the world?  All using code!!!

Yes, you heard it right.

How do we do this?

First and foremost, we need data!! And by data, I mean real time data because trends may change day to day. Social media is full of data, and we should thank Twitter for writing a Java HTTP client for streaming real-time Tweets using Twitter’s own Streaming API.

This client is known as Hosebird Client (hbc). Though it was written by Twitter a long time back and Twitter has deprecated some of its features but it will perfectly work for our requirement.

Also, we need to store the streaming data into some data-store and for this purpose we’ll be using Elasticsearch.

Why Elasticsearch?

The sole purpose of using Elasticsearch is to use Kibana’s Canvas to further visualise the data.

Canvas is a whole new way of making data look amazing. Canvas combines data with colours, shapes, text, and your own imagination to bring dynamic, multi-page, pixel-perfect, data displays to screens large and small.


In simple words it is an application which lets you visualise data stored in Elasticsearch in a better and customised way in real time (while data is being ingested in Elasticsearch) and is currently in beta release.

You’ll be thrilled to see the end result using Elasticsearch Canvas.

Note: For the demonstration Elasticsearch & Kibana 6.5.2 are used.


  • Make sure Elasticsearch and Kibana are installed.

Let’s get started. Cheers to the beginning ?

Follow the steps below to implement the above concept:

1) Setting up a maven project:

1.1) Create a Maven Project (for the demonstration I am using Eclipse IDE, you can use any IDE):

1.2) Skip the archetype selection:

1.3) Add the Group Id, Artifact Id and Name, then click Finish:

2) Configuring the maven project:

2.1) Open the pom.xml and add the following dependencies:


These are the Java client libraries of Twitter and Elasticsearch.

2.2) Configuring the maven-compiler-plugin to use Java 8:


2.3) After this update the maven project:

Alternately you can also press Alt+F5 after selecting the project.

3) Create an Application class:

3.1) Go to src/main/java and create a new class:

3.2) Add the Package and Name of the class then click Finish:

4) Configure the Twitter Java Client:

4.1) Create a static method createTwitterClient in Application class and add the following lines of code:

public static Client createTwitterClient(BlockingQueue<String> msgQueue, List<String> terms) {
	Hosts hosebirdHosts = new HttpHosts(Constants.STREAM_HOST);
	StatusesFilterEndpoint hosebirdEndpoint = new StatusesFilterEndpoint();
	hosebirdEndpoint.trackTerms(terms); // tweets with the specified terms
	Authentication hosebirdAuth = new OAuth1(consumerKey, consumerSecret, token, secret);
	ClientBuilder builder = new ClientBuilder().name("Twitter-Elastic-Client").hosts(hosebirdHosts)
				.processor(new StringDelimitedProcessor(msgQueue));
	Client hosebirdClient =;
	return hosebirdClient;

Notice, that this method expects two arguments: one is the BlockingQueue which is used as a message queue for the tweets and another is the List of terms we want our tweets to be filtered with (in our case “tea”, “coffee” & “beer”). So we are configuring our client to return real time filtered tweets (tweets with terms “tea”, “coffee” or “beer”).

Notice the line of code shown below:

Authentication hosebirdAuth = new OAuth1(consumerKey, consumerSecret, token, secret);

Twitter Java Client uses OAuth to provide authorised access to the Streaming API, which requires you to have the consumer/access keys and tokens.
So to stream Twitter data you must have the consumer/access keys and tokens.

4.2) Getting Twitter Consumer API/Access token keys:

Follow the link Getting Twitter Consumer API/Access token keys to obtain the keys and tokens.

After getting the Consumer API key, Consumer API secret key, Access token and Access token secret,add them as Strings in the Application class:

private final static String consumerKey = "xxxxxxxxxxxxxxxxxx";
private final static String consumerSecret = "xxxxxxxxxxxxxxxxxx";
private final static String token = "xxxxxxxxxxxxxxxxxx";
private final static String secret = "xxxxxxxxxxxxxxxxxx";

It is not advisable to put this info in the program itself and should be read from a config file but for brevity I am putting these values in Application class as static final Strings.

5) Configure the Elasticsearch Transport Client:

5.1) Create a static method createElasticTransportClient  in Application class and add the following lines of code:

public static TransportClient createElasticTransportClient() throws UnknownHostException {
	TransportClient client = new PreBuiltTransportClient(Settings.EMPTY)
			.addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300));
	return client;

The above method returns a Transport Client which talks to locally running Elasticsearch on port 9300.

If your Elasticsearch is running on some other port or host then you may need to change the values of “localhost” to your “host” and “9300” to your “port”, if your Elasticsearch cluster name is different that “elasticsearch”, then you need to create the client like this:

TransportClient client = new PreBuiltTransportClient(Settings.builder().put("", "myClusterName").build())
				.addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300));

6) Streaming data from twitter:

Once the client establishes a connection:

// establish a connection

The blocking queue will now start being filled with messages. However we would like to read only first 1000 messages from the queue:

int count = 0;
while (!client.isDone() && count != 1000) {
  String msg = msgQueue.take(); // reading a tweet
  // Segregating the tweet and writing result to elasticsearch

7) Segregating tweets based on terms and then indexing the segregated result to Elasticsearch:

For brevity I am streaming first 1000 tweets (containing terms “tea”, “coffee” & “beer”), segregating them one by one and indexing the results in Elasticsearch.

Example: Let’s say if a tweet contains the term “ tea ” then I will index the following document into Elasticsearch:

{ “tweet” : “tea” }

One thing I would like to clear: Let’s say if a tweet has tea and coffee both then I will consider only the first term. However, if you want to consider both the terms then hack into my repo stated at the end of this article.

This is how the complete Application class looks like:

package com.technocratsid.elastic;

import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;

import java.util.List;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.transport.client.PreBuiltTransportClient;

import com.twitter.hbc.ClientBuilder;
import com.twitter.hbc.core.Client;
import com.twitter.hbc.core.Constants;
import com.twitter.hbc.core.Hosts;
import com.twitter.hbc.core.HttpHosts;
import com.twitter.hbc.core.endpoint.StatusesFilterEndpoint;
import com.twitter.hbc.core.processor.StringDelimitedProcessor;
import com.twitter.hbc.httpclient.auth.Authentication;
import com.twitter.hbc.httpclient.auth.OAuth1;

public class Application {

	private final static String consumerKey = "xxxxxxxxxxxxxxxxxx";
	private final static String consumerSecret = "xxxxxxxxxxxxxxxxxx";
	private final static String token = "xxxxxxxxxxxxxxxxxx";
	private final static String secret = "xxxxxxxxxxxxxxxxxx";
	private static Logger logger = Logger.getLogger(Application.class.getName());

	public static void main(String[] args) {
		BlockingQueue<String> msgQueue = new LinkedBlockingQueue<String>(1000);
		List<String> terms = Lists.newArrayList("tea", "coffee", "beer");

		// Elasticsearch Transport Client
		TransportClient elasticClient = createElasticTransportClient();

		// Twitter HoseBird Client
		Client client = createTwitterClient(msgQueue, terms);

		String msg = null;
		int count = 0;

		// Streaming 1000 tweets
		while (!client.isDone() && count != 1000) {
			try {
				msg = msgQueue.take();
				logger.log(Level.INFO, msg);

				// Segregating the tweets
				if (msg.contains(" tea ")) {
					insertIntoElastic(elasticClient, "tea");
				} else if (msg.contains(" coffee ")) {
					insertIntoElastic(elasticClient, "coffee");
				} else {
					insertIntoElastic(elasticClient, "beer");
			} catch (InterruptedException ex) {
				logger.log(Level.SEVERE, ex.getMessage());
		// Closing the clients 

	public static Client createTwitterClient(BlockingQueue<String> msgQueue, List<String> terms) {
		Hosts hosebirdHosts = new HttpHosts(Constants.STREAM_HOST);
		StatusesFilterEndpoint hosebirdEndpoint = new StatusesFilterEndpoint();
		hosebirdEndpoint.trackTerms(terms); // tweets with the specified terms
		Authentication hosebirdAuth = new OAuth1(consumerKey, consumerSecret, token, secret);
		ClientBuilder builder = new ClientBuilder().name("Twitter-Elastic-Client").hosts(hosebirdHosts)
				.processor(new StringDelimitedProcessor(msgQueue));
		Client hosebirdClient =;
		return hosebirdClient;

	public static TransportClient createElasticTransportClient() {
		TransportClient client = null;
		try {
			client = new PreBuiltTransportClient(Settings.EMPTY)
					.addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300));
		} catch (UnknownHostException ex) {
			logger.log(Level.SEVERE, ex.getMessage());
		return client;

	public static void insertIntoElastic(TransportClient client, String tweet) {
		try {
			client.prepareIndex("drink-popularity", "_doc")
					.setSource(jsonBuilder().startObject().field("tweet", tweet).endObject()).get();
		} catch (IOException e) {


8) Configuring Canvas in Kibana:

Make sure your Kibana server is running. Mine is running locally at http://localhost:5601.

8.1) Go to http://localhost:5601.

8.2) Go to Dev Tools and perform the following requests:

PUT drink-popularity

The above PUT request creates an index drink-popularity.

PUT drink-popularity/_mapping/_doc
  "properties": {
    "tweet" : {
      "type" : "keyword"

The above request adds a new field tweet to the _doc mapping type.

8.3) Go to Canvas:

8.4) I have already created a Canvas workpad. You just need to download it from here and import it in your own Canvas by clicking on Import workpad JSON file and then selecting the downloaded JSON file.

8.5) Once you have imported the workpad, open it by clicking on Drink Popularity workpad from Canvas workpads list.

This is what you should see:

Now click on No of tweets metric:

On the side panel of Selected Layer click on Data:

Notice the Elasticsearch SQL query used to fetch total no of tweets. Looks familiar right?

The above Elasticsearch SQL counts the total number of documents in drink-popularity index.

Do the same for one of the Horizontal progress bars:

Notice the Data panel:

So the above query is counting the no of tweets where tweet = ‘tea’ and dividing it by total no of tweets i.e. 1000.

Same thing has been done for other two progress bars.

9) Run the program to see live results in Canvas:

Before running the program the initial Canvas looks like this:

Now run the Application class and enable the auto-refresh in Canvas to see live updates and notice the Canvas.

After sometime:

In the end:

Cheers !!! Beer is the winner 🙂 You can also check the results for a specific location by filtering the tweets based on location.

I hope you guys like the concept.

Feel free to hack into the github repo.