1. Home
  2. Forum
  3. Big Data Platform
  4. Own wordcount program

Own wordcount program

Hi,

I'd want to develop my own file (java, hive, pig, scala or python) in Sycfusion BD platform. How can I do this?

Regards,
Florin

6 Replies
AT Aravindraja Thinakaran Syncfusion Team November 29, 2017 07:18 AM UTC

Hi Miron, 
 
Thank you for contacting Syncfusion support. 
 
You can create your own word count program, Syncfusion Big Data Platform shipped with lot of getting started samples. 
You can explore and run in corresponding tab in Big Data Studio. Please find some of word count samples’ location within our platform. 
 
  1. Pig –  <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Scripts\Pig\ShowcaseSamples\KeywordDensity.pig
  2. Hive –  <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Scripts\Hive\ShowcaseSamples\KeywordDensity.hql
  3. Scala –  <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Scripts\Spark\Scala\GettingStarted\WordCount.scala
  4. Python – <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Scripts\Spark\Python\GettingStarted\WordCount.py
  5. Hadoop -  <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Scripts\Hadoop\Mapreduce\WordCount
  6. Java – <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Java\MapReduce\WordCount
  7. C# Map Reduce –  <Installed Directory>:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\CSharp\C# MapReduce Samples\Samples\WordCount Demo
 
You can run the C# word count sample from Big Data Dashboard’s sample browser itself. 
                    
 
Please let us know if you need further assistance. 
 
Thanks, 
Aravindraja T 


MG MARY GRIFFITHE J May 3, 2018 05:57 AM UTC

How to create jar file?
I know only python. How to work with python on hadoop??

AT Aravindraja Thinakaran Syncfusion Team May 3, 2018 11:17 AM UTC

Hi Mary Griffithe, 
 
Thanks for contacting Syncfusion support. 

Please find response for your queries as follows. 
How to create jar file? 
You can create Jar files in following ways using Syncfusion Big Data Platform support. 

  1. To create Jar in Eclipse\Net-Beans using Java gradle sample, please follow the guidelines in ReadMe.txt file.
For Example: 
<Installed Drive>C:\Syncfusion\BigData\<Installed Version>\BigDataSDK\Samples\Java\MapReduce\WordCount\ReadMe.txt 
 
  1. To create Jar without IDE using Hadoop and Java, follow the steps explained in below document.
I know only python. How to work with python on hadoop? 
We have shipped set of Python Samples with Syncfusion Big Data Platform. You can use the samples for your reference to work with Python on Hadoop. 
 
Also, we have provided Python getting started samples in Big Data Studio. You can run the python scripts in Python console and IPython Notebooks. 
 
 
 
Please let us know if you need further assistance. 

Thanks, 
Aravindraja T 


MG MARY GRIFFITHE J May 7, 2018 02:48 PM UTC

Is there a source for codes for hadoop in syncfusion?
What is the syntax for hadoop mapreduce?


MG MARY GRIFFITHE J May 7, 2018 02:51 PM UTC

how to create jar file using python? Is there a similar stepwise guidance?


DK Dinesh Kumar P Syncfusion Team May 8, 2018 06:40 PM UTC

Hi Mary Griffithe, 
  
Please find the response for your queries, 
  
What is the syntax for hadoop mapreduce? 
1.       Create 2 classes for Map and Reduce extending base class like for e.g. 'MapReduceBase' in Java.  
2.       Locate the names of Map and Reduce class names and other parameters like Job Name. 
3.       In a main class, provide arguments for input file, output file and execute the logic. Please check the MapReduce programs present in the installed location for more information, 
Java - C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\Samples\Java\MapReduce 
C# - C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\Samples\CSharp\C# MapReduce Samples 
4.       If the language is C# you can run the application directly to check the output. In case of Java, you have another option along with running directly by preparing the Java code as Jar file and execute like below using 'hadoop jar' util. Please check the mapreduce samples that is shipped in Big Data Studio, 
 
Is there a source for codes for hadoop in syncfusion? 
Can you please explain your requirement about which source code you would like to check? So that we can help you better. 
how to create jar file using python? Is there a similar stepwise guidance? 
We cannot create a jar with Python script files. A '.jar' file is a compiled collection of Java classes.  
  
For your query, if you would like to implement MapReduce application using Python scripts we can execute the same via Hadoop Streaming as specified in the following reference. We will check and update you a sample for the same by May 9, 2018. 
  
  
Please let us know in case of any other queries, 
  
Regards, 
Dinesh Kumar P

Loader.
Up arrow icon