We use cookies to give you the best experience on our website. If you continue to browse, then you agree to our privacy policy and cookie policy. (Last updated on: November 16, 2018).
Unfortunately, activation email could not send to your email. Please try again.
Syncfusion Feedback

Some Questions

Thread ID:

Created:

Updated:

Platform:

Replies:

127159 Oct 30,2016 11:58 PM UTC Nov 14,2016 12:27 PM UTC Data Integration Platform 7
loading
Tags: Data Integration
Jono Walker
Asked On October 30, 2016 11:58 PM UTC

Firstly, this looks like an excellent solution that covers lots of integration use cases and I am keen to download and give it a try. I was unable to attend the webinar due to my timezone, however I watched it via Youtube and have a few follow up questions that weren't covered above:

1. How do you source control projects/solutions/templates etc?

2. Is there support for environment variables e.g. Dev/Test/Prod?

3. Do you only need do deploy the Template.xml file to a new environment?

4. How do you monitor production projects/solutions for errors, or failures to run?

5. Can connections be saved for re-use?

6. How can you incrementally load data from source to destination?

7. Is there OData support? (via HTTP/JSON possibly?) 

8. Can you connect to MySQL over SSH?

Thanks!


Suresh Ram C [Syncfusion]
Replied On October 31, 2016 12:16 PM UTC

Hi Jono Walker, 
 
Thank you for contacting Syncfusion Support. 
 
I have given below the response to your queries: 
 
1.   
How do you source control projects/solutions/templates etc?   
By default, there is no support to maintain the old version of templates. We need to maintain it in SVN, Bitbucket, GitHub, or any other source repository.   
2.   
Is there support for environment variables e.g. Dev/Test/Prod?   
No support for environment variable. 
We need to maintain separate environments for Development/Testing and Production. 
Once developed/tested the workflow in Development/Testing environment, you need to move the templates into Production environment.   
3.   
Do you only need do deploy the Template.xml file to a new environment?   
Yes.   
4.   
How do you monitor production projects/solutions for errors, or failures to run?   
In case of failure/error, we have an option to send mail.   
   
   
We can manage error, warning or exception in the bulletin board.   
   
5.   
Can connections be saved for re-use?   
Yes.   
6.   
How can you incrementally load data from source to destination?   
We can achieve it using the “QueryDataBaseTable processor” for RDBMS source. This processor has maximum-value columns property (usually an auto-increment id, timestamp). For other data source we can create our own custom processor.   
7.   
Is there OData support? (via HTTP/JSON possibly?)   
Yes, OData has Rest API support. So, by using “GetHTTP” and “InvokeHTTP” processor we can achieve it.   
8.   
Can you connect to MySQL over SSH?   
To connect with MySQL server, you need to download mysql-connector-java-5.1.39.jar file and move it to the below location. Then restart the service.   
Location: {Drive}:\Syncfusion\Data Integration\1.1.0.60\SDK\NIFI\lib   
But this jar file doesn’t support SSH connection. We can achieve it, by creating our own custom processor.   
 
Please let us know, if you need further assistance. 
 
Regards,   
Sureshram 


Jono Walker
Replied On November 7, 2016 10:23 PM UTC

Thanks for the reply and further info.

A feature request (for your backlog) would be the ability to store environment variables so that when deploying the template.xml file between environments, we don't need to manually reconfigure things like paths, connections strings etc.

Suresh Ram C [Syncfusion]
Replied On November 8, 2016 11:14 AM UTC

Hi Jono Walker, 
 
Thanks for the update. 
 
The requested feature is already available in Data Integration Platform. While deploying the template.xml file between environments, no manual reconfiguration is needed for paths, connection string, etc. Manual changes is needed only for sensitive values likes passwords, secret keys etc. 
 
 
Please let us know, If you need further assistance. 
 
Sureshram 


Jono Walker
Replied On November 8, 2016 10:05 PM UTC

Thanks for the reply, but how do you handle:
  • Different DB connection string for DEV/TEST/PROD
  • Different FTP folder paths for DEV/TEST/PROD
  • Different HTTP endpoints for DEV/TEST/PROD
I think these, along with credentials would need to be manually configured, unless I am missing something ?

Suresh Ram C [Syncfusion]
Replied On November 9, 2016 12:33 PM UTC

Hi Jono Walker, 
 
Thanks for the update. 
 
Manual configuration is needed, Only if we use different DB connection string/folder paths/endpoints for Development/Testing/Production environments. 
 
Please let us know, If you need further assistance. 
 
Sureshram 


Jono Walker
Replied On November 12, 2016 12:40 AM UTC

Thanks again. So my request still stands for your backlog then :)

A feature request (for your backlog) would be the ability to store environment variables so that when deploying the template.xml file between environments, we don't need to manually reconfigure things like paths, connections strings etc.

Suresh Ram C [Syncfusion]
Replied On November 14, 2016 12:27 PM UTC

Hi Jono Walker, 
  
Thanks for the update. 
  
We are going to log a feature request, to store environment variables to avoid manual reconfiguration.  For security reasons, we have not stored passwords and secret keys. So, please provide your complete requirement in detail. This would be helpful. 
  
Sureshram 


CONFIRMATION

This post will be permanently deleted. Are you sure you want to continue?

Sorry, An error occured while processing your request. Please try again later.

Warning Icon You are using an outdated version of Internet Explorer that may not display all features of this and other websites. Upgrade to Internet Explorer 8 or newer for a better experience.Close Icon

;