We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy.
Unfortunately, activation email could not send to your email. Please try again.

Local Cluster Development

Thread ID:

Created:

Updated:

Platform:

Replies:

127128 Oct 27,2016 10:23 AM Jul 20,2017 12:44 AM Big Data Platform 7
loading
Tags: General
Oula Alshiekh
Asked On October 27, 2016 10:23 AM

After Local cluster Installation
if we access the cluster through cluster manager we will be able to access cluster  monitoring  functions as described here https://help.syncfusion.com/bigdata/cluster-manager/cluster-monitoring-and-alert

but what if we want to handle hdfs and hive operations ?
i think if iam right we should connect to this cluster through syncfusion big data studio  Add Cluster window

but iam getting the error
Primary Name node Wasn't accessible
even though all services are working

is this way right ,and what is the problem please


Baskaran [Syncfusion]
Replied On October 28, 2016 05:13 AM

Hi Oula,

 

Could you please share us the following details to assist you further?

 

1.       1. Are studio and single node cluster running on different machines or in same machine?

2.       2. How did you try to add the cluster in studio, using IP or hostname?

3.       3. Please share the following logs from single node cluster machine.

           Agent logs:  <drive>:\Syncfusion\BigData\<version>\Agent\logs\

          Configuration file: <drive>:\Syncfusion\BigData\<version>\Agent\Configuration.xml

4.       4. Please share the following logs from studio machine.

          <drive>:\Syncfusion\BigData\<version>\BigDataSDK\Logs\Studio

 

Regards,

Baskaran V


Oula Alshiekh
Replied On October 29, 2016 08:20 AM

thanks for your help
1 Syncfusion big data studio is working on my local pc while cluster is working on a vm that is hosted on my pc  through vmware
2- i added  the cluster by ip     (of course iam able to ping this ip )
3 - the strange thing is that:
when i tried to connect to the local cluster which is hosted on a vm on my pc  by adding host name every thing was ok and i was able to handle hive instructions
but after a restart of my pc when i entered cluster manager on hosted vm  i find it as dead and  all services are  not working and iwas  not able to connect it through big data studio by entering host name now

i wish if there was a task bar telling you that start all services or any service operation is happening now (what time should it take does that depends on ram capacity)
here are the logs
vm  which hosts local cluster and contains cluster manager is called  WIN-GD3ERV5R8PT

i couldn't find this path in  single node cluster machine instead i find this log under the path
C:\Syncfusion\BigData\3.1.0.3\BigDataAgent\Manager Service\logs\InstallerAgent

and configuration.xml under the path
C:\Syncfusion\BigData\3.1.0.3\BigDataAgent\Manager Service

Note: my local cluster is working on a vm which is 2 Gb Ram is there any hard ware requirements for local cluster ?


Attachment: Local_cluster_Log_b1382cf5.rar

Oula Alshiekh
Replied On October 29, 2016 10:05 AM

this is the message i get when connecting to local cluster

Attachment: Local_Cluster_Photo_2facbf69.rar

Oula Alshiekh
Replied On October 30, 2016 03:56 AM

when i hit the button collect logs from local cluster these are the logs i got

Attachment: OULA_bb6214d7.zip

Baskaran [Syncfusion]
Replied On October 31, 2016 03:20 AM

Hi Oula, 
  
Thank you for the update.  
  
Please find the response as follows. 
  
Query 
Response 
Regarding restarting PC that hosts local cluster on a VM and 
Message to start all services or any service operation is happening now 
Studio services will be automatically started when system is restarted whereas multi node cluster or local cluster are having different workflow. You can start local cluster service as shown in below screenshot using cluster manager and then try to add this cluster in Studio. 
  
 
  
My local cluster is working on a vm which is 2 Gb Ram is there any hard ware requirements for local cluster? 
Yes. As we recommended earlier threads Studio, local cluster and cluster nodes all require specific hardware requirement to run the services properly. Once again I am listing out the requirement as follows. 
  
  
RAM 
Local Disk 
Number of machines or VMs needed 
Studio 
8 GB or higher 
100 GB or higher 
1 
Local cluster 
8 GB or higher 
100 GB or higher 
1 
Multi node cluster 
Name node - 8 GB or higher
Data Node - 4 GB or higher
 
100 GB or higher 
3 or higher 
  
As we are following you in multiple threads, we understand that you couldn't evaluate the Syncfusion Big Data Platform especially cluster manager due to hardware shortage. We recommend you to avail the above mentioned hardware requirements to configure production environment properly.  
  
Or if you couldn't avail required hardware as of now, please let us know your current available machines, hardware details and the specific thing you are trying to work with (e.g. polybase, hive, etc.) so that we can check and let you know some workaround and tweaks to meet your requirement with available hardware resources. 
  
Regards, 
Baskaran V 


Tim Maher
Replied On July 19, 2017 05:44 PM

Hi, How do I find out what Hostname/IP my local pc is?




Aravindraja Thinakaran [Syncfusion]
Replied On July 20, 2017 12:44 AM

Hi Tim, 

Thanks for contacting Syncfusion support. 

You can get the IP address and Hostname of your local PC as follows. 

Get the IP address of PC 
Open Command prompt, run the ipconfig command and you can get the IP address as like below 
 
Get the host name of PC 
Open Command prompt, run the hostname command and you can get the hostname of your local pc as like follows. 
 


Thanks, 
Aravindraja T 


CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.

;