Exclude parameter in SQOOP to exclude import tables
Exclude parameter is useful to exclude the tables from sqoop export operation. exclude parameter work with import-all-tables parameter. here import-all-tables will import all tables but exclude the tables in exclude list.
STEP 1. Check the no of tables in mysql database.
mysql> show tables;
+—————-+
| Tables_in_test |
+—————-+
| company |
| student |
| transactions |
+—————-+
3 rows in set (0.00 sec)
STEP 2. Use import-all-tables command to import all tables from test database excluding company table.
sqoop import-all-tables –connect jdbc:mysql://127.0.0.1:3306/test –username root –password cloudera –m 1 -warehouse-dir /user/cloudera/excludetables –exclude-tables company
above command will import all the tables from test database but not company. use , if you are looking for multiple tables to exclude.
STEP 3. Crosscheck the data on hdfs level.
[cloudera@quickstart ~]$ hadoop fs -ls /user/cloudera/excludetables
Found 2 items
drwxr-xr-x – cloudera cloudera 0 2019-01-22 23:00 /user/cloudera/excl udetables/student
drwxr-xr-x – cloudera cloudera 0 2019-01-22 23:01 /user/cloudera/excl udetables/transactions
[cloudera@quickstart ~]$ hadoop fs -ls /user/cloudera/excludetables/student
Found 2 items
-rw-r–r– 1 cloudera cloudera 0 2019-01-22 23:00 /user/cloudera/excl udetables/student/_SUCCESS
-rw-r–r– 1 cloudera cloudera 17 2019-01-22 23:00 /user/cloudera/excl udetables/student/part-m-00000
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$
[cloudera@quickstart ~]$ hadoop fs -cat /user/cloudera/excludetables/student/par t-m-00000
1,2018-04-26,200
[cloudera@quickstart ~]$
