We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy.
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

piggybank.jar compatibility with syncfusion bigdata management studio

Thread ID:

Created:

Updated:

Platform:

Replies:

118556 Mar 18,2015 11:58 AM Mar 24,2015 12:12 AM Big Data Platform 4
loading
Tags: General
carl
Asked On March 18, 2015 11:58 AM

i have created a basic pig script to load an XML file using the piggybank.jar included with the syncfusion install:

REGISTER file:///c:/Syncfusion/BigDataSDK/1.1.0.8/SDK/Pig/contrib/piggybank/java/piggybank.jar;
DEFINE XMLLoader org.apache.pig.piggybank.storage.XMLLoader();
mydata = LOAD '/myxml.xml' USING XMLLoader();
DUMP mydata;

The contents of the "myxml.xml" file are:

<?xml version="1.0" encoding="UTF-8" ?>
<rootnode>
<elementnode attributenode="value">element content</elementnode>
</rootnode>

The XML import does not work and fails with the following error:

2015-03-18 10:52:57,112 [JobControl] INFO  org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /0.0.0.0:8032
2015-03-18 10:52:57,175 [JobControl] INFO  org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2015-03-18 10:52:58,238 [JobControl] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2015-03-18 10:52:58,253 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter - Cleaning up the staging area /tmp/hadoop-yarn/staging/csavell/.staging/job_1426522352277_0018
2015-03-18 10:52:58,269 [JobControl] ERROR org.apache.pig.backend.hadoop23.PigJobControl - Error while trying to run jobs.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:130)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
at java.lang.Thread.run(Thread.java:744)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
... 3 more
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at org.apache.pig.piggybank.storage.XMLLoader$XMLFileInputFormat.isSplitable(XMLLoader.java:615)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:352)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
... 8 more

in attempting to correct the issue, i have downloaded and tried the following piggybank.jar versions from apache:

REGISTER file:///c:/pig-0.14.0/pig-0.14.0/contrib/piggybank/java/piggybank.jar;
REGISTER file:///c:/pig-0.13.0/pig-0.13.0/contrib/piggybank/java/piggybank.jar;
REGISTER file:///c:/pig-0.12.1/pig-0.12.1/contrib/piggybank/java/piggybank.jar;

None of these versions of piggybank.jar will work properly.
Can you help me understand how to solve this problem?

thanks


carl
Replied On March 18, 2015 12:01 PM

as a follow-up, please note that the mydata.xml file WILL load successfully as text without using the XMLLoader()

Praveena P [Syncfusion]
Replied On March 20, 2015 09:12 AM

Hi Carl, 

Thank you for using Syncfusion products.

 

Loading Xml file using piggy bank.jar failed.

The issue occurred due to version incompatibility between the piggybank build and the hadoop version in our Syncfusion platform.

 

To apply fix:

 

  •          Close the BigdataManagement Studio .

 

  •         Replace the attached piggybank.jar in the respective location.

 

  •         Open the Big Data Management Studio and execute the pig script.

 

Pig script for loading xml file

By applying the fix and executing the provided pig script will submit the job and run successfully but no data will be dumped as it stores 0 records.

 

So please modify the script as shown below by passing the element of the xml file while defining XMLLoader().

 

REGISTER file:///c:/Syncfusion/BigDataSDK/1.1.0.8/SDK/Pig/contrib/piggybank/java/piggybank.jar;

 

DEFINE XMLLoader org.apache.pig.piggybank.storage.XMLLoader('rootnode');

 

mydata = LOAD '/myxml.xml' USING XMLLoader();

 

DUMP mydata;

 

 

 

Please let me know if you need any further assistance on this.

 

Regards

Praveena


Attachment: piggybankjar_2bfb59fa.zip

carl
Replied On March 20, 2015 12:27 PM

thank you Praveena, the updated piggybank.jar corrected my issue.

Praveena P [Syncfusion]
Replied On March 24, 2015 12:12 AM

Hi Carl,

We are glad that your problem has resolved. Please let us know if you have any queries.

Regards,

Praveena.


CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon

;