We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy. (Last updated on: November 16, 2018).
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

Relocate HDFS datanode directory for Syncfusion Big Data Platform

Thread ID:





123757 Apr 19,2016 12:44 AM UTC Apr 25,2016 08:49 AM UTC Big Data Platform 3
Tags: General
Asked On April 19, 2016 12:44 AM UTC


I've installed Syncfusion Big Data Platform to root drive [D:\] which has limited available space.  Can you please advise how can I relocate/change the HDFS datanode directory to another directory [E:\]?

Current datanode directory:  D:\Syncfusion\BigData\\BigDataSDK\Metadata\data\dfs\datanode
Destination datanode directory:  E:\Syncfusion\BigData\\BigDataSDK\Metadata\data\dfs\datanode

Thank you.


Mahendran Maragatha Patchai [Syncfusion]
Replied On April 19, 2016 01:52 PM UTC

Hi Kelly,                                                                                
Thank you for using Syncfusion Big Data Platform.
You can able to Relocate the HDFS data node directory to another directory without any data loss. Please find the steps to “Relocate HDFS Datanode directory for Syncfusion Big Data Platform” below.

1.      Stop the Hadoop services, if running.


2.      Copy the Data node directory from "D:\Syncfusion\BigData\\BigDataSDK\Metadata\data\dfs\datanode" to the Destination Directory “E:\Syncfusion\BigData\\BigDataSDK\Metadata\data\dfs\datanode


3.      Go to “D:\Syncfusion\BigData\\BigDataSDK\SDK\Hadoop\etc\hadoop” Directory and locate the hdfs-site.xml.               

4.      Find out the “dfs.datanode.data.dir” property which is located in hdfs-site.xml and change the value of that property to your NEW_LOCATION (E:\Syncfusion\BigData\\BigDataSDK\Metadata\data\dfs\datanode)

5.      Restart the Hadoop Services. 

Please let us know if you have any other queries.                                        
Mahendran Maragatha Patchai                                                                                                        

Replied On April 25, 2016 03:02 AM UTC

Hi Mahendran,

I've successfully relocate the HDFS datanode directory.  However, I'm getting error while trying sample scripts for scala, python, and iPython.  These sample scripts were able to execute without error before relocating the datanode directory.  Appreciate your advise on the resolution.

Here's the changes done in hdfs-site.xml file.  All services started successfully after this change.
From old value

To new value:

Here's sample script:
// sc is an existing SparkContext.
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)

sqlContext.sql("create External table if not exists recommend_Ratings1(criticid string,movieid string,rating double) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/Data/Ratings/'")

// Queries are expressed in HiveQL
sqlContext.sql("FROM recommend_Ratings1 SELECT criticid,movieid,rating").collect().foreach(println)

Thank you for your help!


Attachment: errormessage_41400dd4.zip

Mahendran Maragatha Patchai [Syncfusion]
Replied On April 25, 2016 08:49 AM UTC

 Hi Kelly,

A support incident has been created under your account, for follow-up of this requirement. Please log on to our support website to check for further updates


Please let me know if you have any questions.

Mahendran Maragatha Patchai


This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Please sign in to access our forum

This page will automatically be redirected to the sign-in page in 10 seconds.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon