Transform and store data

[Tim Maher]

HI, 

I have a csv file within HDFS in my BD Platform.  I need to create a set of ETL queries which transforms the data on a weekly basis and stores it in a HBase 'table'.  What would the most ideal way to achieve this?

Thanks you

[Aravindraja Thinakaran]

Hi Tim, 

 

You can achieve this requirement using Syncfusion Data Integration Platform.  

 

Overview:  

   

  

  

Please follow the below steps to achieve your requirement. 

 

CSV file in HDFS	Fetch CSV file from HDFS into Data Integration Platform using below processors.   GetHDFS FetchHDFS
Perform ETL operations	We have 200+ processors in Data Integration Platform to perform 95% of the ETL operations.   Help documentations: https://help.syncfusion.com/data-integration/overview
Move final output into HBase table	We have following processors to move processed data into HBase table.   PutHBaseCell PutHBaseJSON
Scheduling on weekly basics	We can schedule the workflow on weekly basics using timer driven or cron driven mode.

 

Please check with sample for your reference. 

http://www.syncfusion.com/downloads/support/forum/131661/ze/Sample-1145302425.zip  

 

Let us know, if you need any further assistance.  

 

Thanks, 

Aravindraja T.