Change HDFS storage location

[Joseph Lin]

I'm new to the Hadoop ecosystem, so apologies in advance for using incorrect terminology.

I'm trying to change my HDFS storage location. I recently installed Syncfusion Big Data Platform version v3.1.0.3 to try out on my computer. It's installed on my C: drive which is an SSD drive.

Am I correct in assuming that the HDFS storage is located in the installation directory. I'd prefer to move it to a regular drive.

Is there a way to do this without having to uninstall?

[Churchil Mariyasagayaraj]

Hi Joseph Lin,   

  

Thank you for using Syncfusion BigData Platform.   

  

You don’t have to uninstall. Please do the following changes in your Hadoop package for it to work.   

  

·       Stop all the running Hadoop services by using service manager.   

   

·       Open hdfs-site.xml file from the following location.   

   

“<installDrive>:\Syncfusion\BigData\<Build version>\BigDataSDK\SDK\Hadoop\etc\hadoop\hdfs-site.xml”.   

   

·       Change the NameNode and DataNode directory according to your SSD location.   

   

Please refer to the below screenshot (here we have used SSD which has the drive letter D:/).   

   

     

·       Copy the Metadata folders namenode and datanode from the below location.   

 

“<installDrive>:\Syncfusion\BigData\<Build version>\BigDataSDK\Metadata\data\dfs”   

 

·       Paste the metadata folders to SSD directory as mentioned in the hdfs-site.xml file.   

(Ex: D:\Syncfusion\BigData\3.1.0.25\BigDataSDK\Metadata\data\dfs)   

 

·       Restart the Hadoop services, hdfs location changed from default C:/ to other directories of your SSD.   

  

Note: The folder structure must be the same as hdfs-site.xml. If not, it may cause the problem while starting the Hadoop services.    

  

Please let us know if you have any other concern.   

  

Regards,   

Churchil M.