How to check if the file or directory exists in HDFS?
Contents
Hadoop command to check whether the directory exists or not:
Syntax: hdfs dfs -test -d hdfs_path
Example: hdfs dfs -test -d /revisit/content/files
echo $? —> to validate the previous command return code.
Explanation
If you are using hadoop version 1, please use hadoop fs instead of hdfs dfs and rest of the command is same for it. If the given hdfs directory present in the system,it will return the code as 0.Otherwise it will return the code as 1.We can validate the return code by using echo $? command after the execution of hadoop command.
Hadoop command to check whether the file exists or not
Syntax : hdfs dfs -test -e hdfs_path/filename
Example: hdfs dfs -test -e /revisit/content/files/schema.xml
echo $? —> To validate the previous command return code.
Explanation
If the file schema.xml present in the given hdfs path,it will return the code as 0.Otherwise it will return the code as 1.If you are using the shell script to execute the hadoop command,you can validate the return code using the if condition in the program.
Sample shell script code to validate the hadoop command return codes
|
1 2 3 4 5 6 7 8 |
hdfs dfs -test -e /revisit/content/files/schema.xml if [ $? -eq 0 ] then echo "schema.xml file exist in the hdfs direcotry" else echo "schema.xml file doesn't exist in the hdfs directory" fi |
Recommended Articles
- How to get the HDFS file size using WebHDFS?
- Hadoop command to check the size of the HDFS file
- How to count the number of lines in a HDFS file?
HI , i am trying to get the below thing work but i am unable too can you please let me know wats wrong here
if $(hadoop fs -test -e $File_name) && ! $(hadoop fs -test -e $File_name);
Hi ,
try the below code and let us know
if $(hadoop fs -test -e $File_name) && $(!(hadoop fs -test -e $File_name));