Latest Versions of TechStack

 Latest Versions of TechStack


Software Used Version(s) Current
Java 8,13 Java SE 17 (LTS)
Spring boot 2.3.12 2.6.3
Angular NA 13.1.1
Android studio NA 4.1
Lombok v1.17 v1.18.22
Log4j 2 2.17.1 2.17.1
Oracle 12 19c
Springfox swagger 2.9.2 3.0.0
Resiliance 4j 1.7 1.7.1
Jenkins 2.324 2.324

Read more


Apache Kafka - FAQs

 


Apache Kafka : an open-source message broker project developed by the Apache Software Foundation written in Scala and is a distributed publish-subscribe messaging system.


Features of kafka 

High Throughput : Support for millions of messages with modest hardware

Scalability : Highly scalable distributed systems with no downtime

Replication : Messages are replicated across the cluster to provide support for multiple subscribers and balances the consumers in case of failures

Durability Provides support for persistence of message to disk

Stream Processing Used with real-time streaming applications like Apache Spark & Storm

Data Loss Kafka with proper configurations can ensure zero data loss


Various components of Kafka:


Topic – a stream of messages belonging to the same type

Producer – that can publish messages to a topic

Brokers – a set of servers where the publishes messages are stored

Consumer – that subscribes to various topics and pulls data from the brokers.


Topic : 

Topic is like a table identified by name.

Topic is split in partitions.


Topic 1 -- Partition 0, partition 1 , partition 2.


Explain the role of the offset.

Messages contained in the partitions are assigned a unique ID number that is called the offset. The role of the offset is to uniquely identify every message within the partition.

  • kafka stores offsets at which consumer group has been reading.
  • offsets will be stored in a separate topic called "_consumer_offsets"
  • When consumer has processed data received from kafka it should be committing the offsets. 
  • offset is specific to a partition. because offset 3 in partion 1 and 2 are not same. 
  • offset is in order with in a partition.
  • data / offsets kept only for one week by default. 
  • once data written cant be changed (immutable)
  • Data is assigned randomly to a partition if we dont specify key 

What is a Consumer Group?

To enhance parallelism.

Consumer Groups is a concept exclusive to Kafka.  Every Kafka consumer group consists of one or more consumers that jointly consume a set of subscribed topics.


You cant have more consumers than partitions. 

if you have 3 partitions u should not have 4 consumers in one group. because consumers in a group shares the partitions. if we have 3 partitions for a topic and 4 consumers in a group each consumer connects to one partitions and 4th one become idle and do nothing. 

consumer has to specify broker name and topic name to read and kafka will take care of pulling data from right brokers 

Messages are read in order like 0,1,2,... but in parallel across the partitions. 

B1 - Topic 1 - partition 0 - 0,1,2,3,4

B2 - Topic 2 - partition 1 - 0,1,2,3,4,5,6,7

Each consumer within a group read from exclusive partitions.


Brokers 

  • A Kafka  cluster is composed of multiple brokers(servers)
  • Each broker contains certain topic partitions.
  • After connecting to any broker , you will be connected to entire cluster. 
  • A cluster with 3 brokers can be seen as follows , data is distributed with partitions.

Broker 1         Broker 2            Broker 3  


Topic 1          Topic 1             Topic 1 

P-0               P-2                   P-1


Topic 2          Topic 2 Topic 1

P-1               P-0                  P-0   


Replication Factor always > 1 

Partitions : 

  • Partitions are the main concurrency mechanism in kafka. 
  • A topic is divided into 1 or more partitions enabling producer and consumer loads to be scaled. 

What is the role of the ZooKeeper?

Kafka uses Zookeeper to store offsets of messages consumed for a specific topic and partition by a specific Consumer Group.

 Is it possible to use Kafka without ZooKeeper?

No, it is not possible to bypass Zookeeper and connect directly to the Kafka server. If, for some reason, ZooKeeper is down, you cannot service any client request.

Explain the concept of Leader and Follower.

Every partition in Kafka has one server which plays the role of a Leader, and none or more servers that act as Followers. The Leader performs the task of all read and write requests for the partition, while the role of the Followers is to passively replicate the leader. In the event of the Leader failing, one of the Followers will take on the role of the Leader. This ensures load balancing of the server.

Why are Replications critical in Kafka? Kafka is durable with replications.

Replication ensures that published messages are not lost and can be consumed in the event of any machine error, program error or frequent software upgrades.

How do you define a Partitioning Key?

Within the Producer, the role of a Partitioning Key is to indicate the destination partition of the message. By default, a hashing-based Partitioner is used to determine the partition ID given the key. Alternatively, users can also use customized Partitions.


Read more


New Features in Java 13

 

New Features in Java 13


  • We initially saw switch expressions in JDK 12 itself, build on the previous version by adding a new yield statement.
  • Text Blocks - To embed JSON , XML etc to avoid escaping in the strings.
  • Class data sharing (CDS) 
  • ZGC: Uncommit Unused Memory 
  • Reimplement the Legacy Socket API 

Read more


GIT Eclipse usage basic

GIT- Basics & Eclipse Usage
This post aim at very basic usage of using GIT based tools and eclipse configurations for the same. 

  • create a new repository : create a new directory, open it and perform a
    • git init
to create a new git repository.

  • checkout a repository : Create a working copy of a local repository by running the command 
    • git clone /path/to/repository 
when using a remote server, your command will be
git clone username@host:/path/to/repository

  • Goto project dir 
1. git init
2. git status
3.git add .
4. git status
5. git commit -m "first commit"
6. Create an empty repo in GITHUB
7.copy clone url
8.git remote add origin << URL >>
9.git push -u origin master
10. Check your files are there on GIT remote

To add more / update files

  • git add . 
  • git status 
  • git commit -m "second commit"
  • git push -u origin master

Read more


Static keyword in java

Static keyword in java

The static keyword in java is used for memory management in JAVA. Static keywords belong to the class rather than the instance/object of a class.
We can use static keywords with:
  1. Variable
  2. Method
  3. Block
  4. Class

1. Static variable in Java

You can declare a static variable by simply use of static keyword with a variable. 
  • Static variables are also known as a class variable because they are associated with the class and common for all the objects of a class.
  • Static variables are declared using the static keyword within a class outside any method/constructor/block.
  • When a class created then one copy of a static variable created in memory. Static variables would be common for all objects of class because a static variable is associated with a class. 
  • Static variables are created at the start of program execution and destroyed automatically when execution ends.
  • Initialization of Static Variable is not Mandatory. Its default value is depending on the data type of variable.
  • If we access the static variable through an object the compiler will show the warning message. The compiler will replace the object name to the class name automatically.
  • We can create static variables at class-level only.

2. Static method in Java

You can declare a static method by using of static keyword.
1. Static methods are belonging to a class.
2. You can access a static method by using the class name. You do not have the need to create an object.
3. A static method can access only static variables. If you write any non-static variable in a static method, then it will throw an exception at compilation time.
Why a static method is important
A static method has two main purposes: For utility or helper methods that don't require any object state. Since there is no need to access instance variables, having static methods eliminates the need for the caller to instantiate the object just to call the method.
Ex: To know KM to Miles someone doesnt to build a car object. Car.convertKMtoMiles(25).
Static method can access without any object. They can access directly by class name. The most common example of a static method is main( ) method. If it were a non-static method, JVM creates an object first then call main() method that will lead the problem of extra memory allocation.

3. Static block in Java

If you want to do some calculation and initialize the static variables you can use static block. The static block is get executed once when the class is first loaded.

4. Static class in Java

You can read the static class from my previous post
--
Cheers
Krishna babu Ghanta
Mobile : 303-886-1490 
Time zone : CST

Read more


Spring architecture and Spring boot

Spring is well-organized architecture consisting  of seven modules. Modules in the Spring framework are:


Spring AOP
One of the key components of Spring is the AOP framework. AOP is used in Spring:

To provide declarative enterprise services, especially as a replacement for EJB declarative services. The most important such service is declarative transaction management, which builds on Spring’s transaction abstraction.
To allow users to implement custom aspects, complementing their use of OOP with AOP
Spring ORM
The ORM package is related to the database access. It provides integration layers for popular object-relational mapping APIs, including JDO, Hibernate and iBatis.

Spring Web
The Spring Web module is part of Spring?s web application development stack, which includes Spring MVC.

Spring DAO
The DAO (Data Access Object) support in Spring is primarily for standardizing the data access work using the technologies like JDBC, Hibernate or JDO.

Spring Context
This package builds on the beans package to add support for message sources and for the Observer design pattern, and the ability for application objects to obtain resources using a consistent API.

Spring Web MVC
This is the Module which provides the MVC implementations for the web applications.

Spring Core
The Core package is the most import component of the Spring Framework.
This component provides the Dependency Injection features. The BeanFactory  provides a factory pattern which separates the dependencies like initialization, creation and access of the objects from your actual program logic.

Read more

Popular Posts

Enter your email address:

Buffs ...

Tags


Powered by WidgetsForFree