Tuning Handbook of Apache Kafka! We all know the power and advantages of Apache Kafka. It is publish-subscribe messaging system which basically has three major components Apache Kafka Consumer Apache Producer Apache Kafka Broker This doc is all about how we can achieve maximum throughput while planning to have Kafka in production or in POCs. […]
Understand Kappa Architecture in 2 minutes What is Kappa Architecture ? Kappa architecture makes all the data processing in Near Real Time or Streaming mode, which in simple terms removing the batch layer from Lambda Architecture makes it a Kappa Architecture, to know quickly about lambda Architecture visit Understand Lambda Architecture in 2 minutes. Evolution […]
Understand Lambda Architecture in 2 minutes What is Lambda Architecture ? Lambda architecture which provides us a combined solution of realtime data with batch data. What is the Need for Lambda Architecture ? lambda Architecture was implemented mainly due to the Latency provided by the Map reduce paradigm, where the batch views was created on […]
10 Key Features in Apache Storm 1.0.0 The Apache Storm community recently announced the release of Apache Storm 1.0.0 stable. This is a noteworthy release that delivers several powerful features that relate to enterprise readiness, operational simplicity and ease of use by dramatically enhancing areas around performance, scalability, debugging ability, maintainability, and manageability. Apache Storm […]
This blog introduces the convergence of complementary technologies – Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. And we will see how Apache Kafka can help us to get data under control and what is it role in our data pipeline, how Spark & Akka help us to process the data, and how Cassandra to […]
Today emerging big data technology firm focused on helping enterprises build breakthrough software solutions powered by disruptive enterprise software trends like Machine learning and data science, Cyber-security, Enterprise IOT, and Cloud. So Hadoop is one of the proven software in big data space, but is it only Hadoop. Nope we have many more technologies which […]
Looking For College Projects ?
Data Lake Architecture Considerations & Composition In our last blog we saw the key benefits of Data Lake, but let’s deep dive in to the internals of a Data Lake via discussing the key considerations and compositions. Architecture Considerations: Take in any solution considerations it is practical difficult to arrives with a one-size-fit-all architecture; hence […]
Team, this time i go with the title called “Top 3 methods of skipping big data’s bad data using Hadoop !“ which describes about how to get corrupt records out from the large data sets which has different format of data. While doing our analysis if the corrupt records are in small percentage we can ignore or […]
Team thanks for reading & engaging ! This time am planned to share with you the my learning on Hadoop Schedulers; titled “Simplified Hadoop Schedulers Overview !” With the help of choosing suitable scheduler, we can make the response times faster for all smaller jobs and also for all the production jobs it’s guaranteed with SLA’s (Service […]
Hadoop compression techniques bring us more benefits in the Hadoop I/O operations, such as space savings and processing speeds. We’ve lot compression formats and algorithm, with pros and cons. Here nothing new is added, just consolidated to have it handy to use it in production implementations. All techniques exhibit a space/time trade-off. We’ve options from […]
Tons of thanks for your valuable time, this time we like to share with you the details on how data movement is happening in the big data ecosystem. It’s named as “The Data Movement in Big Data Ecosystem”. Ingesting data in to Hadoop is so vital from systems like RDBMS, Mainframes, logs, machine-generated data, event data […]
Big Data Meets Microsoft Azure ! For Big Data & Cloud...
How to Ingest HDFS in JSON format using Apache Sqoop ?...
The 4 Key Concepts in the Anatomy of an Apache Spark Job!...
The 1-2-3-4-5-6-7-8-9 of Cognitive Computing ! Dear Data...