We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy.
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

About Hive Input

Thread ID:

Created:

Updated:

Platform:

Replies:

118360 Mar 2,2015 03:36 PM UTC Mar 17,2015 02:25 AM UTC Big Data Platform 12
loading
Tags: General
Klueze
Asked On March 2, 2015 03:36 PM UTC

Hello,everyone.I'm a new user of big data and this big data platform.
Today I loaded some tables from SQL Server 2008 into HDFS.Now I want to load those files in HDFS into Hive.It's a pity that errors occured.

This is my file's folder:
HDFS Root/user/Raww/VFPrice/vfPriceInfo

This is my statement:
LOAD DATA INPATH 'VFPrice/vfPriceInfo' INTO TABLE stockpricedatabase.vf_price

And the error says "Wrong file format".
I just want to know how I can fix this problem.
Thanks a lot in advance!

Klueze
Replied On March 3, 2015 03:01 AM UTC

After trying,The problem above has been solved.But new problems come out.

My table in SQL Server 2008 contains some Chinese columns and their data type is 'nvarchar'.
After loaded them into HIVE,I use the load function to check the result.Then I find that the Chinese words become messy code.
How can I fix this problem on Syncfusion Big Data Platform?
Thanks in advance!

Praveena P [Syncfusion]
Replied On March 4, 2015 05:46 AM UTC

Hi Klueze,

Thanks For using  our Syncfusion Products.

 

 

 

 

 

 

 

 

 

 

Query:

 

In hive Chinese words  becomes messy code.

 

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.

 

Please refer the attached screenshot which shows result of hive when loaded in command shell,

 

cid:image001.png@01D05666.D9A4E620

 

 

The following Jira issue link of Apache hive is still open:

 

https://issues.apache.org/jira/browse/HIVE-3914



Please let me know if you have any queries.

Regards,
Praveena.

Praveena P [Syncfusion]
Replied On March 4, 2015 06:27 AM UTC


Hi Klueze,,

Please ignore the previous response.

 

 

 

 

Query:

 

In hive Chinese words  becomes messy code.

 

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.

 

Please refer the attached screenshot which shows result of hive when loaded in command shell.

 

The following Jira issue link of Apache hive is still open:

 

https://issues.apache.org/jira/browse/HIVE-3914

 

 

Please let me know if you have any queries.

 

Regards,

Praveena.


Attachment: image_7826dedd.zip

Klueze
Replied On March 5, 2015 09:06 AM UTC


Hi Klueze,,

Please ignore the previous response.

 

 

 

 

Query:

 

In hive Chinese words  becomes messy code.

 

As Apache Hive currently doesn’t have option to display or handle Chinese words , the result of hive which is in Chinese words will be displayed wrongly. This is an issue of apache hive.

 

Please refer the attached screenshot which shows result of hive when loaded in command shell.

 

The following Jira issue link of Apache hive is still open:

 

https://issues.apache.org/jira/browse/HIVE-3914

 

 

Please let me know if you have any queries.

 

Regards,

Praveena.


Attachment: image_7826dedd.zip

Thank you very much for your suggestion,Praveena.
I'm sorry that I'm just at the beginning of learning big data,so I still do not use command shell but the BDMS.The attach file below shows the situation.
And I find that If I use Sqoop in BDMS to export those data loaded into Hive back to SQL Server 2008,the Chinese contents won't be messy code in SQL Server.So can I say that the messy code problem only appear in display?If the real contents stored in Hive are correct,I think it would be acceptable for me.
Later I will learn how to use Eclipse to operate the platform.I think I could explore more things.
Thanks in advance!

Attachment: image_141e89de.zip

Athiram S [Syncfusion]
Replied On March 5, 2015 01:14 PM UTC

Hi Klueze,
 
Thank you for your update.
 
Yes, as you have suspected, the Chinese text content issue (messy code problem) occurs only on displaying the text in big data management studio, but the real contents are actually stored in HDFS storage without any data loss. The original processed contents (file) can be directly found in the default browser for HDFS or via HDFS browser in Big data management studio. Please check the below link for the default HDFS file browser:
 
To know more about HDFS browser in Big data management studio, refer below link:
 
Regarding the displaying of text issue in big data management studio, It would be easier if we could use the sample data used on your end. Could you please provide the text file (result file) containing the sample data? So that it would be easier to analyze and resolve the issue reproduced on your end at the earliest.
 
Please let me know if you have any concerns.
 
Regards,
Athiram S​

Klueze
Replied On March 6, 2015 03:42 PM UTC

Thanks a lot for your explaination,Athiram,and here is my sample data.

Attachment: Samples_d3bfd76.zip

Klueze
Replied On March 6, 2015 03:59 PM UTC

What's more,when I enter this site,an error comes out.
http://localhost:50070/explorer.html#/
The attach file will clearly show the problem.While I click the folders' name,it occurs,with some of other information(e.g. size) being incorrect.My platform is not installed in disk C but D,and all services has been started.
Thanks in advance.

Attachment: Error_Message_83f01e6e.zip

Praveena P [Syncfusion]
Replied On March 9, 2015 01:24 PM UTC

Hi Klueze,

 

Thanks for your update.

 

Error Message in HDFS

The error message displayed in HDFS browser will occur when the webhdfs.enabled property in Hadoop configuration file has false value.

 

So please change the value of dfs.webhdfs.enabled property to true in  hdfs-site.xml file in the following directory <Installed_Drive_of_BigData_Setup>:\Syncfusion\BigDataSDK\1.1.0.8\SDK\Hadoop\etc\Hadoop\

as shown below

 

            <property>

                        <name>dfs.webhdfs.enabled</name>

                        <value>true</value>

                        <final>true</final>

            </property>

 

Folder  size being incorrect

By default in HDFS Browser, the size of folders will be displayed as 0B , only the sizes of file inside the folder will be displayed adjacent to the files.

 

Please refer the attached screenshots for  more clearance.

 

Displaying of Chinese text issue in BigDataManagementStudio

This issue in Bigdata Management Studio is due to lack of Unicode support. This issue will be fixed and will be included in our next release which will be announced later in our website.

 

 

Please let me know if you have any concerns.

 Regards,

Praveena.


Attachment: Filesize_e202ed57.zip

Klueze
Replied On March 11, 2015 03:46 AM UTC

OK,roger that.Thanks a lot.

Praveena P [Syncfusion]
Replied On March 13, 2015 12:41 PM UTC

 

Hi Klueze,

The issue with "Displaying Chinese Text in BigDataManagement Studio”  has been resolved. Please download the exe and dll  for the fix in Beta Release of Big Data Management Studio (1.1.0.8) from the following location:

http://www.syncfusion.com/downloads/support/directtrac/118360/IssueFix-2068925090.zip

Steps to apply fix:

1.       Close the BigdataManagement Studio .

2.       Replace the corresponding exe and dll in the following directory:

             <Installed_Drive_of_BigData_Setup>:\Syncfusion\BigDataSDK\1.1.0.8\ Utilities\BigDataManagementStudio\4.0\bin

3.       Open the Big Data Management Studio and load the results in Hive.

Please refer the attached screenshot of loading result of your  provided sample data with hive in BigDataManagementStudio.

chineseword.zip

This fix will be included in our upcoming release of Big Data.

Please let me know if you need any further assistance on this.

Regards,

Praveena



Klueze
Replied On March 16, 2015 05:49 AM UTC

Great,it works well.Thank you very much!

Praveena P [Syncfusion]
Replied On March 17, 2015 02:25 AM UTC

Hi Klueze,

We are glad that your problem has resolved. Please let us know if you have any queries.

Regards,
Praveena.

CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon

;