Spark 2.0

Learnomate Technologies > Spark 2.0
Difference between var and val in spark- Interview question
11
Jun

Var keyword is just similar to variable declaration in Java whereas Val is little different. Once a variable is declared using Val the reference cannot be changed to point to another reference. This functionality of Val keyword in Scala can be related to the functionality of java final keyword. Val refers to immutable declaration of […]

Posted in: Hadoop ,
Tags: Difference between var and val in spark , hadoop , hadoop Interview Question , Hadoop Tutorial , Interview Question , spark , Spark 2.0 ,
Read CSV and JSON file format in spark 2.0
06
Jun

Read CSV with spark 2.0 STEP 1. Open the spark-shell and fire the following command. scala> spark.read.format(“csv”).option(“header”,”true”).load(“F:/Hadoop Youtube/customer.csv”) STEP 2. Display the result with show command scala> .show +—–+——+———–+——-+———-+—-+——+ |empno| ename|designation|manager| hire_date| sal|deptno| +—–+——+———–+——-+———-+—-+——+ | 7369| SMITH| CLERK| 7902|12/17/1980| 800| 20| | 7499| ALLEN| SALESMAN| 7698| 2/20/1981|1600| 30| | 7521| WARD| SALESMAN| 7698| 2/22/1981|1250| 30| […]

Posted in: Hadoop ,
Tags: CSV , csv file , json , ReadCSVandJSON , Spark 2.0 , spark-shell ,