Category: Scala

Redshift Database connection in spark 0

Redshift Database connection in spark

This blog primarily focus on how to connect to redshift from Spark. Redshift: Amazon Redshift is a fully managed petabyte-scale data warehouse service. Redshift is designed for analytic...

scala-logo 0

File Operation in scala

File operation is important operation in an application. We might have to provide some configuration information or some input for application in such scenario we have to perform...

Missing Imputation in scala 0

Missing Imputation in scala

Imputation: In statistics, imputation is the process of replacing missing data with substituted values. When substituting for a data point, it is known as “unit imputation”; when substituting...

Spark SQL Using Parquet 0

Spark SQL Using Parquet

Today, I’m focusing on how to use parquet format in spark.  Please get the more insight about parquet format If you are new to this format. Parquet: Apache Parquet is a...

Spark SQL Using Hive 0

Spark SQL Using Hive

In this blog I’m going to describe how to integrate hive with spark. You may find this code on spark’s official github page. My effort is to describe...