csv file

Learnomate Technologies > csv file
Read CSV and JSON file format in spark 2.0
06
Jun

Read CSV with spark 2.0 STEP 1. Open the spark-shell and fire the following command. scala> spark.read.format(“csv”).option(“header”,”true”).load(“F:/Hadoop Youtube/customer.csv”) STEP 2. Display the result with show command scala> .show +—–+——+———–+——-+———-+—-+——+ |empno| ename|designation|manager| hire_date| sal|deptno| +—–+——+———–+——-+———-+—-+——+ | 7369| SMITH| CLERK| 7902|12/17/1980| 800| 20| | 7499| ALLEN| SALESMAN| 7698| 2/20/1981|1600| 30| | 7521| WARD| SALESMAN| 7698| 2/22/1981|1250| 30| […]

Posted in: Hadoop ,
Tags: CSV , csv file , json , ReadCSVandJSON , Spark 2.0 , spark-shell ,
Querying on CSV file using Spark SQL
06
Jun

Apache Spark SQL is a Spark module to simplify working with structured data using DataFrame and DataSet abstractions in Python, Java, and Scala. These abstractions are the distributed collection of data organized into named columns. It provides a good optimization technique. Using Spark SQL we can query data, both from inside a Spark program and […]

Posted in: Uncategorized ,
Tags: csv file , hadoop , spark , sql ,