1

Is there a way through the AWS console to understand the number of records that got loaded into a redshift table using the AWS data pipeline?

Keerthika Koka
  • 33
  • 6

1 Answers1

3

You can get that information from STL_LOAD_COMMITS table.

Especially look at lines_scanned column:

Number of lines scanned from the load file. This number may not match the number of rows that are actually loaded. For example, the load may scan but tolerate a number of bad records, based on the MAXERROR option in the COPY command.

Like

SELECT * FROM STL_LOAD_COMMITS
ORDER BY curtime DESC
LIMIT 10
botchniaque
  • 4,698
  • 3
  • 35
  • 63
  • 1
    MAXERROR should always be set to zero, as a matter of good practise. Incoming files should either fully load, or not load at all. The complexity which comes from *partial* loads is practically impossible to justify. –  Jun 15 '21 at 18:25