Is there an equivalent to `pwd` in hdfs?

Question

I tried to do hdfs dfs -pwd, but that command does not exist. So currently I am resorting to doing hdfs dfs -ls .. followed by hdfs dfs -ls ../...

I also looked at the command listing for hdfs dfs but did not see anything that looked promising.

Is there a more direct way to find the absolute path?

Iirc it's in the api but not exposed in the command line runner. — Dave Newton, Feb 03 '14 at 23:20

score 27 · Accepted Answer · edited Oct 20 '19 at 10:58

27

hdfs dfs -pwd does not exist because there is no "working directory" concept in HDFS when you run commands from command line.

You cannot execute hdfs dfs -cd in HDFS shell, and then run commands from there, since both HDFS shell and hdfs dfs -cd commands do not exist too, thus making the idea of working directory redundant.

Your home dir is always the prefix of the path, unless it starts from /.

edited Oct 20 '19 at 10:58

cristid9

1,070
1
17
37

answered Feb 04 '14 at 21:47

Evgeny Benediktov

1,389
1
10
13

2

you know I do feel there is a need of pwd, that is because sometimes when we create the active directories in hdfs only accessible to the current users, at that time hdfs have the default directory as /user_name. – Devendra Bhat Mar 16 '17 at 18:59
That's unfortunate - I want to see what the path is to my home directory. – Hein du Plessis Mar 31 '20 at 09:11

score 27 · Answer 2 · answered Feb 14 '17 at 02:12

If you want to see the absolute path of home directory,you can make a empty directory and delete it,the information can show you the absolute path.

[test@test ~]$ hadoop fs -mkdir t1                                            
[test@test ~]$ hadoop fs -rm -r t1    
17/02/14 10:00:46 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 1 minutes, Emptier interval = 0 minutes.
Moved: 'hdfs://nameservice1/user/test/t1' to trash at: hdfs://nameservice1/user/test/.Trash/Current

'hdfs://nameservice1/user/test/t1' is the absolute path of the t1 directory

score 15 · Answer 3 · edited May 23 '17 at 12:18

15

There is no -cd command, so I guess you expect -pwd to return the home directory.

According to this related question: HDFS Home Directory, the home directory of hdfs is always /user/<ShortUserName> (it is hardcoded in the source).

edited May 23 '17 at 12:18

Community

1
1

answered Dec 07 '15 at 13:48

jrouquie

4,315
4
27
43

score 7 · Answer 4 · edited Jun 20 '20 at 09:12

There is no such PWD, CD suportability in hadoop like unix/linux.

The possible Hadoop commands are

hadoop fs

    -appendToFile
    -cat
    -checksum
    -chgrp
    -chmod
    -chown
    -copyFromLocal
    -copyToLocal
    -count
    -cp
    -createSnapshot
    -deleteSnapshot
    -df
    -du
    -expunge
    -get
    -getfacl
    -getfattr
    -getmerge
    -help
    -ls
    -mkdir
    -moveFromLocal
    -moveToLocal
    -mv
    -put
    -renameSnapshot
    -rm
    -rmdir
    -setfacl
    -setfattr
    -setrep
    -stat
    -tail
    -test
    -text
    -touchz
    -usage

score 6 · Answer 5 · answered Jan 31 '18 at 17:48

There is no equivalent of the pwd command in the file system shell utility hdfs dfs, but here's a workaround using mkdir and find:

$ hdfs dfs -mkdir xYz
$ hdfs dfs -find / -name "xYz"

(where "xYz" is an unused file name)

In one command:

$ echo $(dirname $(hdfs dfs -rm -R xYz>/dev/null 2>&1;hdfs dfs -mkdir xYz&&hdfs dfs -find / -name "xYz"))

score 2 · Answer 6 · answered Mar 12 '20 at 13:44

There is no concept of a present working directory in Hadoop. That being said it would be worth noting that if you would like to understand the directory structure of the Hadoop system, you can start your way from the root and keep iteratively moving to the desired directory.

So begin with typing, observe the output and keep traversing: hadoop fs -ls /

score 1 · Answer 7 · answered Feb 03 '14 at 23:27

1

There's the hdfs.pwd() function, but you unfortunately cannot use it from the command line. HDFS User's Guide

answered Feb 03 '14 at 23:27

LeonardBlunderbuss

1,264
1
11
22

In the example in your link, I see a shell that looks like `R>`. Do you know how I can access that shell? – merlin2011 Feb 03 '14 at 23:36
It appears to be the Oracle R Connector, although I've never used it: [link](http://docs.oracle.com/cd/E37231_01/doc.20/e36961/orch.htm#CFHFIGAI) – LeonardBlunderbuss Feb 03 '14 at 23:46

Is there an equivalent to `pwd` in hdfs?

7 Answers7

hadoop fs

Linked