6

What is exactly "$path" used for? I just ran "select "$path" from table limit 10", in athena it's showing the file path of S3 where data is pointed. But when i gave limit 10, it's showing same path 10 times, if i don't limit the statement it's scanning entire data. Can please someone expalin.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Roy
  • 109
  • 1
  • 8

1 Answers1

8

"$path" is a pseudo-column which evaluates to the path of the source file given row comes from. This is provided by Presto's Hive connector. If you have a file with 100 rows, you will get same path 100 times.

If you want to get first ten distinct paths, you should try

select DISTINCT "$path" from table limit 10
Piotr Findeisen
  • 19,480
  • 2
  • 52
  • 82