Some Questions

[Jono Walker]

Firstly, this looks like an excellent solution that covers lots of integration use cases and I am keen to download and give it a try. I was unable to attend the webinar due to my timezone, however I watched it via Youtube and have a few follow up questions that weren't covered above:

1. How do you source control projects/solutions/templates etc?

2. Is there support for environment variables e.g. Dev/Test/Prod?

3. Do you only need do deploy the Template.xml file to a new environment?

4. How do you monitor production projects/solutions for errors, or failures to run?

5. Can connections be saved for re-use?

6. How can you incrementally load data from source to destination?

7. Is there OData support? (via HTTP/JSON possibly?) 

8. Can you connect to MySQL over SSH?

Thanks!

[Suresh Ram C]

Hi Jono Walker, 

 

Thank you for contacting Syncfusion Support. 

 

I have given below the response to your queries: 

 

1.	How do you source control projects/solutions/templates etc?	By default, there is no support to maintain the old version of templates. We need to maintain it in SVN, Bitbucket, GitHub, or any other source repository.
2.	Is there support for environment variables e.g. Dev/Test/Prod?	No support for environment variable.  We need to maintain separate environments for Development/Testing and Production.  Once developed/tested the workflow in Development/Testing environment, you need to move the templates into Production environment.
3.	Do you only need do deploy the Template.xml file to a new environment?	Yes.
4.	How do you monitor production projects/solutions for errors, or failures to run?	In case of failure/error, we have an option to send mail.            We can manage error, warning or exception in the bulletin board.
5.	Can connections be saved for re-use?	Yes.
6.	How can you incrementally load data from source to destination?	We can achieve it using the “QueryDataBaseTable processor” for RDBMS source. This processor has maximum-value columns property (usually an auto-increment id, timestamp). For other data source we can create our own custom processor.
7.	Is there OData support? (via HTTP/JSON possibly?)	Yes, OData has Rest API support. So, by using “GetHTTP” and “InvokeHTTP” processor we can achieve it.
8.	Can you connect to MySQL over SSH?	To connect with MySQL server, you need to download mysql-connector-java-5.1.39.jar file and move it to the below location. Then restart the service.    Location: {Drive}:\Syncfusion\Data Integration\1.1.0.60\SDK\NIFI\lib    But this jar file doesn’t support SSH connection. We can achieve it, by creating our own custom processor.

 

Please let us know, if you need further assistance. 

 

Regards,   

Sureshram 

[Jono Walker]

Thanks for the reply and further info.

A feature request (for your backlog) would be the ability to store environment variables so that when deploying the template.xml file between environments, we don't need to manually reconfigure things like paths, connections strings etc.

[Suresh Ram C]

Hi Jono Walker, 

 

Thanks for the update. 

 

The requested feature is already available in Data Integration Platform. While deploying the template.xml file between environments, no manual reconfiguration is needed for paths, connection string, etc. Manual changes is needed only for sensitive values likes passwords, secret keys etc. 

 

 

Please let us know, If you need further assistance. 

 

Regards, 

Sureshram 

[Jono Walker]

Thanks for the reply, but how do you handle:

• Different DB connection string for DEV/TEST/PROD
• Different FTP folder paths for DEV/TEST/PROD
• Different HTTP endpoints for DEV/TEST/PROD

I think these, along with credentials would need to be manually configured, unless I am missing something ?

[Suresh Ram C]

Hi Jono Walker, 

 

Thanks for the update. 

 

Manual configuration is needed, Only if we use different DB connection string/folder paths/endpoints for Development/Testing/Production environments. 

 

Please let us know, If you need further assistance. 

 

Regards, 

Sureshram 

[Jono Walker]

Thanks again. So my request still stands for your backlog then :)

A feature request (for your backlog) would be the ability to store environment variables so that when deploying the template.xml file between environments, we don't need to manually reconfigure things like paths, connections strings etc.

[Suresh Ram C]

Hi Jono Walker, 

  

Thanks for the update. 

  

We are going to log a feature request, to store environment variables to avoid manual reconfiguration.  For security reasons, we have not stored passwords and secret keys. So, please provide your complete requirement in detail. This would be helpful. 

  

Regards, 

Sureshram