The 4 Key Concepts in the Anatomy of an Apache Spark Job! For Big Data & Cloud Community members Apache Spark is Awesome to handle any workloads such as Batch, Streaming, Real-Time, and Ad-hoc. However, to fine tune and optimize of our Apache Spark Applications we need to have a grip on the Apache Spark […]
What is Beyond Classic Hadoop? Is it Spark and Flink? In this blog, we will explore the two new big data friends to Hadoop, and they are Apache Spark and Apache Flink. And if we take the Hadoop improvements with the parallel processing MapReduce; speed is very first focus. However, MapReduce is designed and developed for […]
Self-Learn Yourself Scala in 21 Blogs – #6 Blog 6 – Recursion and Tail Recursion in Functional Programming. Missed the previous blogs have a quick look with Self-Learn Yourself Scala in 21 Blogs (#1, #2, #3, #4, #5). In this blog let’s understand the recursion and tail recursion in functional programming. Recursion is frequently used […]
Scalable Apache Spark Solution to Big Data Secondary Sort Problem! – Part 1 In Big Data era the secondary sort problem is relates to sorting values associated with a key in the reduce phase. It can be called as value-to-key conversion. The secondary sorting technique will help us to sort the values in ascending or […]
Relationship between MapReduce, Spark, YARN, and HDFS ! In Big Data era Hadoop is the de facto standard for developing of big data applications by using MapReduce framework. And Hadoop is composed of one or more master nodes and any number of slave nodes depends up on the data needed. Hadoop simplifies distributed applications by […]
Self-Learn Yourself Apache Spark in 21 Blogs – #8 In this blog let us discuss on How to loading data, what is Lambdas, How to do Transforming Data and more on Transformations. And want to have quick read on the other blogs in this learning series. Apache Spark can load from any input sources like […]
Self-Learn Yourself Scala in 21 Blogs – #5 Blog 5 – Does functional programming matters and what are monads? Missed the previous blogs have a quick look with Self-Learn Yourself Scala in 21 Blogs (#1, #2, #3, #4). In this blog let’s understand for Scala developers does the functional programming matters and also what is […]
This blog introduces the convergence of complementary technologies – Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. And we will see how Apache Kafka can help us to get data under control and what is it role in our data pipeline, how Spark & Akka help us to process the data, and how Cassandra to […]
Blog 4 – OOP to Functional Programming We are already using functional programming using Scala with the previous blog series of Self-Learn Yourself Scala in 21 Blogs (#1, #2, #3). Let’s start with defining functional programming which is a programming paradigm that models computation as the evaluation of expressions and expressions are built using functions […]
Blog 3 – Functional Programming & Data Structures In Scala Functional programming and functional data structures are very interesting and powerful. Actually it supports both types data structures called immutable & mutable. And let us discuss on two more vital concepts called type parameterization and higher-order functions. The type is similar to Java generics; it […]
Blog 2 – Lets’ get started with Scala Just type Scala in your environment to get the Scala interpreter and if everything is fine we will prompt with scala>. If you have problem with installation please follow the link, which has step by step explanations. So we are good to explore the Scala commands. Now […]
Self-Biearn Yourself Scala in 21 Blogs – #1 Blog 1 – Scala the basics Thanks to the communities like LinkedIn, hadoop, Spark, Apache Software, Yahoo and more…from dataottam. As a new learning and sharing initiative we the dataottam team launched “Self-Learn Yourself Scala in 21 Blogs”. Scala is something Object-Oriented meets functional to have best […]
Big Data Meets Microsoft Azure ! For Big Data & Cloud...
How to Ingest HDFS in JSON format using Apache Sqoop ?...
The 4 Key Concepts in the Anatomy of an Apache Spark Job!...
The 1-2-3-4-5-6-7-8-9 of Cognitive Computing ! Dear Data...