About Hive Input

[Klueze]

Hello,everyone.I'm a new user of big data and this big data platform.
Today I loaded some tables from SQL Server 2008 into HDFS.Now I want to load those files in HDFS into Hive.It's a pity that errors occured.

This is my file's folder:
HDFS Root/user/Raww/VFPrice/vfPriceInfo

This is my statement:
LOAD DATA INPATH 'VFPrice/vfPriceInfo' INTO TABLE stockpricedatabase.vf_price

And the error says "Wrong file format".
I just want to know how I can fix this problem.
Thanks a lot in advance!

[Klueze]

After trying,The problem above has been solved.But new problems come out.

My table in SQL Server 2008 contains some Chinese columns and their data type is 'nvarchar'.
After loaded them into HIVE,I use the load function to check the result.Then I find that the Chinese words become messy code.
How can I fix this problem on Syncfusion Big Data Platform?
Thanks in advance!

[Praveena P]

Hi Klueze,

Thanks For using  our Syncfusion Products.

 

Query:   In hive Chinese words  becomes messy code.

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.   Please refer the attached screenshot which shows result of hive when loaded in command shell,       The following Jira issue link of Apache hive is still open:   https://issues.apache.org/jira/browse/HIVE-3914

Please let me know if you have any queries.

Regards,

Praveena.

[Praveena P]

Hi Klueze,,

Please ignore the previous response.

 

Query:   In hive Chinese words  becomes messy code.

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.   Please refer the attached screenshot which shows result of hive when loaded in command shell.   The following Jira issue link of Apache hive is still open:   https://issues.apache.org/jira/browse/HIVE-3914

Please let me know if you have any queries.

 

Regards,

Praveena.

Attachment: image_7826dedd.zip

[Klueze]

Hi Klueze,,

Please ignore the previous response.

 

Query:   In hive Chinese words  becomes messy code.

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.   Please refer the attached screenshot which shows result of hive when loaded in command shell.   The following Jira issue link of Apache hive is still open:   https://issues.apache.org/jira/browse/HIVE-3914

Please let me know if you have any queries.

 

Regards,

Praveena.

Attachment: image_7826dedd.zip

Thank you very much for your suggestion,Praveena.
I'm sorry that I'm just at the beginning of learning big data,so I still do not use command shell but the BDMS.The attach file below shows the situation.
And I find that If I use Sqoop in BDMS to export those data loaded into Hive back to SQL Server 2008,the Chinese contents won't be messy code in SQL Server.So can I say that the messy code problem only appear in display?If the real contents stored in Hive are correct,I think it would be acceptable for me.
Later I will learn how to use Eclipse to operate the platform.I think I could explore more things.
Thanks in advance!

Attachment: image_141e89de.zip

[Athiram S]

Hi Klueze,

 

Thank you for your update.

 

Yes, as you have suspected, the Chinese text content issue (messy code problem) occurs only on displaying the text in big data management studio, but the real contents are actually stored in HDFS storage without any data loss. The original processed contents (file) can be directly found in the default browser for HDFS or via HDFS browser in Big data management studio. Please check the below link for the default HDFS file browser:

http://localhost:50070/explorer.html#/

 

To know more about HDFS browser in Big data management studio, refer below link:

http://help.syncfusion.com/ug/bigdata/default.htm#!documents/workingwithhdfsexplo.htm

 

Regarding the displaying of text issue in big data management studio, It would be easier if we could use the sample data used on your end. Could you please provide the text file (result file) containing the sample data? So that it would be easier to analyze and resolve the issue reproduced on your end at the earliest.

 

Please let me know if you have any concerns.

 

Regards,

Athiram S​

[Klueze]

Thanks a lot for your explaination,Athiram,and here is my sample data.

Attachment: Samples_d3bfd76.zip

[Klueze]

What's more,when I enter this site,an error comes out.
http://localhost:50070/explorer.html#/
The attach file will clearly show the problem.While I click the folders' name,it occurs,with some of other information(e.g. size) being incorrect.My platform is not installed in disk C but D,and all services has been started.
Thanks in advance.

Attachment: Error_Message_83f01e6e.zip

[Praveena P]

Hi Klueze,

 

Thanks for your update.

 

Error Message in HDFS	The error message displayed in HDFS browser will occur when the webhdfs.enabled property in Hadoop configuration file has false value.   So please change the value of dfs.webhdfs.enabled property to true in  hdfs-site.xml file in the following directory <Installed_Drive_of_BigData_Setup>:\Syncfusion\BigDataSDK\1.1.0.8\SDK\Hadoop\etc\Hadoop\ as shown below               <property>                         <name>dfs.webhdfs.enabled</name>                         <value>true</value>                         <final>true</final>             </property>
Folder  size being incorrect	By default in HDFS Browser, the size of folders will be displayed as 0B , only the sizes of file inside the folder will be displayed adjacent to the files.   Please refer the attached screenshots for  more clearance.
Displaying of Chinese text issue in BigDataManagementStudio	This issue in Bigdata Management Studio is due to lack of Unicode support. This issue will be fixed and will be included in our next release which will be announced later in our website.

 

Please let me know if you have any concerns.

 Regards,

Praveena.

Attachment: Filesize_e202ed57.zip

[Klueze]

OK,roger that.Thanks a lot.

[Praveena P]

 

Hi Klueze,

The issue with "Displaying Chinese Text in BigDataManagement Studio”  has been resolved. Please download the exe and dll  for the fix in Beta Release of Big Data Management Studio (1.1.0.8) from the following location:

http://www.syncfusion.com/downloads/support/directtrac/118360/IssueFix-2068925090.zip

Steps to apply fix:

1.       Close the BigdataManagement Studio .

2.       Replace the corresponding exe and dll in the following directory:

             <Installed_Drive_of_BigData_Setup>:\Syncfusion\BigDataSDK\1.1.0.8\ Utilities\BigDataManagementStudio\4.0\bin

3.       Open the Big Data Management Studio and load the results in Hive.

Please refer the attached screenshot of loading result of your  provided sample data with hive in BigDataManagementStudio.

chineseword.zip

This fix will be included in our upcoming release of Big Data.

Please let me know if you need any further assistance on this.

Regards,

Praveena

[Klueze]

Great,it works well.Thank you very much!

[Praveena P]

Hi Klueze,

We are glad that your problem has resolved. Please let us know if you have any queries.

Regards,

Praveena.