sqoop incremantal import error

Question

i want to import last updates on my as400 table with sqoop import incremantal, this is my sqoop command: i'm sure about allvariables, to_porcess_ts it's a string timestamp (yyyymmddhhmmss)

sqoop import --verbose --driver $SRC_DRIVER_CLASS --connect $SRC_URL --username $SRC_LOGIN --password $SRC_PASSWORD \
    --table $SRC_TABLE --hive-import --hive-table $SRC_TABLE_HIVE  --target-dir $DST_HDFS \
    --hive-partition-key "to_porcess_ts" --hive-partition-value $current_date --split-by $DST_SPLIT_COLUMN --num-mappers 1 \
    --boundary-query "$DST_QUERY_BOUNDARY" \
    --incremental-append --check-column "to_porcess_ts" --last-value $(hive -e "select max(unix_timestamp(to_porcess_ts, 'ddmmyyyyhhmmss')) from $SRC_TABLE_HIVE"); \
    --compress --compression-codec org.apache.hadoop.io.compress.SnappyCodec

I got this error:

   18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Error parsing arguments for import:
    18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Unrecognized argument: --incremental-append
    18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Unrecognized argument: --check-column
    18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Unrecognized argument: to_porcess_ts
    18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Unrecognized argument: --last-value
    18/05/14 16:14:18 ERROR tool.BaseSqoopTool: Unrecognized argument: -46039745595

score 1 · Accepted Answer · answered May 14 '18 at 21:22

1

Remove ; or replace it on ) from the end of this line:

--last-value $(hive -e "select max(unix_timestamp(to_porcess_ts, 'ddmmyyyyhhmmss')) from $SRC_TABLE_HIVE";) \

answered May 14 '18 at 21:22

Iskuskov Alexander

4,077
3
23
38

thank you @Iskuskov; i still have the error `18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Error parsing arguments for import: 18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Unrecognized argument: --incremental-append 18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Unrecognized argument: --check-column 18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Unrecognized argument: to_porcess_ts 18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Unrecognized argument: --last-value 18/05/15 09:38:43 ERROR tool.BaseSqoopTool: Unrecognized argument: -45945053285` – Zied Hermi May 15 '18 at 07:41
@ZiedHermi, I think you are wrong about converting month number: you need to use `MM` instead of `mm` (See [format examples](https://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html)). Please show output of `select to_porcess_ts from $SRC_TABLE_HIVE limit 1` – Iskuskov Alexander May 15 '18 at 11:54
the error is to delete the `--incremental-append` but after fixing the syntax, i got this error : `ERROR tool.ImportTool: Import failed: java.io.IOException: java.sql.SQLException: [SQL0206] Column TO_PORCESS_TS not in specified tables` – Zied Hermi May 15 '18 at 16:02

sqoop incremantal import error

1 Answers1