Change HDFS storage location

Question

I'm new to the Hadoop ecosystem, so apologies in advance for using incorrect terminology.

I'm trying to change my HDFS storage location. I recently installed Syncfusion Big Data Platform version v3.1.0.3 to try out on my computer. It's installed on my C: drive which is an SSD drive.

Am I correct in assuming that the HDFS storage is located in the installation directory. I'd prefer to move it to a regular drive.

Is there a way to do this without having to uninstall?

Churchil Mariyasagayaraj · Answer

Hi Joseph Lin,

Thank you for using Syncfusion BigData Platform.

You don’t have to uninstall. Please do the following changes in your Hadoop package for it to work.

· Stop all the running Hadoop services by using service manager.

· Open hdfs-site.xml file from the following location.

“<installDrive>:\Syncfusion\BigData\<Build version>\BigDataSDK\SDK\Hadoop\etc\hadoop\hdfs-site.xml”.

· Change the NameNode and DataNode directory according to your SSD location.

Please refer to the below screenshot (here we have used SSD which has the drive letter D:/).

· Copy the Metadata folders namenode and datanode from the below location.

“<installDrive>:\Syncfusion\BigData\<Build version>\BigDataSDK\Metadata\data\dfs”

· Paste the metadata folders to SSD directory as mentioned in the hdfs-site.xml file.

(Ex: D:\Syncfusion\BigData\3.1.0.25\BigDataSDK\Metadata\data\dfs)

· Restart the Hadoop services, hdfs location changed from default C:/ to other directories of your SSD.

Note: The folder structure must be the same as hdfs-site.xml. If not, it may cause the problem while starting the Hadoop services.

Please let us know if you have any other concern.

Regards,

Churchil M.