We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy.
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

Transform and store data

Thread ID:

Created:

Updated:

Platform:

Replies:

131661 Jul 21,2017 04:11 PM UTC Jul 24,2017 06:12 PM UTC Big Data Platform 1
loading
Tags: General
Tim Maher
Asked On July 21, 2017 04:11 PM UTC

HI, 

I have a csv file within HDFS in my BD Platform.  I need to create a set of ETL queries which transforms the data on a weekly basis and stores it in a HBase 'table'.  What would the most ideal way to achieve this?

Thanks you

Aravindraja Thinakaran [Syncfusion]
Replied On July 24, 2017 06:12 PM UTC

Hi Tim, 
 
You can achieve this requirement using Syncfusion Data Integration Platform 
 
Overview:  
   
  
  
Please follow the below steps to achieve your requirement. 
 
CSV file in HDFS  
Fetch CSV file from HDFS into Data Integration Platform using below processors.  
  • GetHDFS
  • FetchHDFS
Perform ETL operations  
We have 200+ processors in Data Integration Platform to perform 95% of the ETL operations.  
Move final output into HBase table  
We have following processors to move processed data into HBase table.  
  • PutHBaseCell
  • PutHBaseJSON
Scheduling on weekly basics  
We can schedule the workflow on weekly basics using timer driven or cron driven mode.  
 
   
 
Please check with sample for your reference. 
 
Let us know, if you need any further assistance.  
 
Thanks, 
Aravindraja T. 


CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon

;