1. Home
  2. Forum
  3. Blazor
  4. Query Builder over Elastic Search

Query Builder over Elastic Search

We have created a custom data adapter that binds to a ViewModel.  

Data is added, updated and deleted to a write store that is separate from the read store following the CQRS Event Sourcing pattern.   

The write store is EventStore while the read store is Elastic Search.  

As such, data that is added to EventStore is projected into Elastic read store on our back end.

We would like to flow queries from data manager over to Elastic Search via the custom data adapter via the Net NEST client for Elastic Search, specifically using the NEST SearchDescriptor.  

I am looking for guidance on how to apply sorting and filtering from the DataManager to the Elastic Search .Net Client (Nest)

The following depicts the data flow...







What I need to do is create/overide custom DataOperations methods and apply the same to the .Net Elastic Search Nest client's SearchDescriptor instance on my ViewModel following the same pattern as per example below...    

Is there any source code examples that depicts each of the DataOperations class methods such as PerformSearching, PerformFiltering etc. that I can derive a custom data operations from.   My thought is that I would create an IDataOperations interface on my ViewModels to implement these operations per each view model against the local instance of the SearchDescriptor.  


  if (dm.Search != null && dm.Search.Count > 0)
            {
                // Searching
                DataSource = DataOperations.PerformSearching(DataSource, dm.Search);
            }
            if (dm.Sorted != null && dm.Sorted.Count > 0)
            {
                // Sorting
                DataSource = DataOperations.PerformSorting(DataSource, dm.Sorted);
            }
            if (dm.Where != null && dm.Where.Count > 0)
            {
                // Filtering
                DataSource = DataOperations.PerformFiltering(DataSource, dm.Where, dm.Where[0].Operator);
            }


9 Replies 1 reply marked as answer
JO John September 1, 2020 04:41 PM UTC

I would like to use Query Builder to query a DataManager that uses a custom data adaptor that is bound to a data grid.



I have followed this example and can see how Query Builder is binding to the same data source as the data grid but in my case I am using the DataManager with a custom data adaptor. 

How can I set the Query Builder's Filtered Query such that it is binding to the DataManager's custom data adaptor's filter? 

See this image of current code  

Tying this back to what I am attempting to do.  I would like to see the Where Filters, derived from the Query Builder passed through and into the custom Data Adaptor where I can then compose the Elastic Search Binary Query similar to the following example... 

To support this on our ViewModels I created an IDataAdaptor Interface that mirrors Sf DataAdaptor and included an ExecuteQueryAsync method to be executed on our ViewModel...




The ExecuteQueryAsync method mirrors the same functionality normally found in the ReadAsync method of the Custom DataAdaptor.   

Within the ExecuteQueryAsync method we are initializing the ElasticSearch SearchDescriptorRequest which will be serialized to JSON and then passed through to our back-end microservices then used to invoke the query on the Elastic Index associated to the ViewModel.   

Currently, Sorting, Paging and Searching work really well, however we required the Query Builder integration with the custom Data Adaptor via Data Manager in order to build up the Elastic DSL query on the SearchDescriptorRequest.     





JP Jeevakanth Palaniappan Syncfusion Team September 2, 2020 10:42 AM UTC

Hi John, 

Greetings from Syncfusion support. 

We have analyzed your query and we suspect that you need to integrate the query builder with the custom adaptor and trying to perform filtering action in the grid using the query builder. We have prepared a sample in which we set the grid query based on the filtering in the query builder. So that in the ReadAsync method, the filtering action will be performed and the corresponding data will be bounded to the grid. Please find the below code snippet and the sample for your reference. 

<div class="col-lg-12 control-section"> 
    <SfQueryBuilder @ref="Querybuilder"> 
        <SfDataManager AdaptorInstance="@typeof(CustomAdaptor)" Adaptor="Adaptors.CustomAdaptor"></SfDataManager> 
        <QueryBuilderRule Condition="or" Rules="@ImportRules"></QueryBuilderRule> 
        <QueryBuilderEvents RuleChanged="UpdateRule"></QueryBuilderEvents> 
        <QueryBuilderColumns> 
            <QueryBuilderColumn Field="OrderID" Label="OrderID" Type="number"></QueryBuilderColumn> 
            <QueryBuilderColumn Field="CustomerID" Label="Name" Type="string"></QueryBuilderColumn> 
        </QueryBuilderColumns> 
    </SfQueryBuilder> 
</div> 
 
<SfGrid TValue="Order" ID="Grid" Query="GridQueryData" AllowSorting="true" AllowFiltering="true" AllowPaging="true" Toolbar="@(new List<string>() { "Add", "Delete", "Update", "Cancel", "Search" })"> 
    <SfDataManager AdaptorInstance="@typeof(CustomAdaptor)" Adaptor="Adaptors.CustomAdaptor"></SfDataManager> 
    <GridPageSettings PageSize="8"></GridPageSettings> 
    <GridEditSettings AllowEditing="true" AllowDeleting="true" AllowAdding="true" Mode="@EditMode.Normal"></GridEditSettings> 
    <GridColumns> 
        <GridColumn Field=@nameof(Order.OrderID) HeaderText="Order ID" IsPrimaryKey="true" TextAlign="@TextAlign.Center" Width="140"></GridColumn> 
        <GridColumn Field=@nameof(Order.CustomerID) HeaderText="Customer Name" Width="150"></GridColumn> 
    </GridColumns> 
</SfGrid> 


@code{ 
 
    private async void UpdateRule(RuleChangeEventArgs args) 
    { 
        this.QueryData = await Querybuilder.GetFilteredRecords(); 
     if (this.QueryData != null) 
        { 
            this.FilterQuery = ((JObject)this.QueryData).ToObject<WhereData>(); 
            if (this.FilterQuery.Where != null && this.FilterQuery.Where.Count > 0) 
            { 
                GridQueryData = new Query().Where(this.FilterQuery.Where); 
            } 
            this.StateHasChanged(); 
        } 
        else 
        { 
            GridQueryData = new Query(); 
            this.StateHasChanged(); 
        } 
    } 
 
    public List<RuleModel> ImportRules = new List<RuleModel> 
    { 
        new RuleModel { Field = "CustomerID", Value = "BOLID", Operator = "equal" } 
    }; 
 
    public class WhereData 
    { 
        public List<WhereFilter> Where { get; set; } 
    } 
 
 
    public class CustomAdaptor : DataAdaptor 
    { 
        // Performs data Read operation 
        public override object Read(DataManagerRequest dm, string key = null) 
        { 
            IEnumerable<Order> DataSource = Orders; 
            if (dm.Search != null && dm.Search.Count > 0) 
            { 
                // Searching 
                DataSource = DataOperations.PerformSearching(DataSource, dm.Search); 
            } 
            if (dm.Sorted != null && dm.Sorted.Count > 0) 
            { 
                // Sorting 
                DataSource = DataOperations.PerformSorting(DataSource, dm.Sorted); 
            } 
            if (dm.Where != null && dm.Where.Count > 0) 
            { 
                // Filtering 
                DataSource = DataOperations.PerformFiltering(DataSource, dm.Where, dm.Where[0].Operator); 
            } 
            int count = DataSource.Cast<Order>().Count(); 
            if (dm.Skip != 0) 
            { 
                //Paging 
                DataSource = DataOperations.PerformSkip(DataSource, dm.Skip); 
            } 
            if (dm.Take != 0) 
            { 
                DataSource = DataOperations.PerformTake(DataSource, dm.Take); 
            } 
            return dm.RequiresCounts ? new DataResult() { Result = DataSource, Count = count } : (object)DataSource; 
        } 
 
    } 
} 



Please get back to us if we misunderstood your query. 

Regards, 
Jeevakanth SP. 


Marked as answer
JO John September 2, 2020 11:03 AM UTC

Hi Jeevakanth, 

Thank you for this solution, although I have yet to try it, this looks on the surface to be exactly what I am looking for.  

SyncFusion is awesome, excellent support, thanks so much!

Cheers
John

JO John September 2, 2020 07:08 PM UTC

I tried this and it works perfectly!

Thank you once again, so very much appreciated!

JP Jeevakanth Palaniappan Syncfusion Team September 3, 2020 04:25 AM UTC

Hi John, 

We are glad to know that your issue has been resolved. Please get back to us if you need further assistance. 

Regards, 
Jeevakanth SP. 


JO John September 10, 2020 09:32 AM UTC

Hi Jeevakanth, 

We are making excellent progress thanks to your earlier support however I have 4 areas that I need your support with. 


1. How to disable UI while processing changes in Data Adaptor/ViewModel?

As we are using CQRS with Event Sourcing pattern within our microservices, we have implemented logic that deals with Eventual Consistency.  

As such when we insert, update or delete via the custom adaptor, the work is handled within or ViewModel and we set a flag in the backing ViewModel which indicates we have one of those command in progress. 

While that is happening I would like to disable the UI until this flag is cleared.  How can I disable the Query Builder UI along with the the Data Grid.  

The backing ViewModel is inherited into the View (line 3 below) and as such available to bind to UI properties...



2. The backing ViewModel has a collection of Validation Errors that can occur from backend business rules, how do I surface these in the view?
I would like to include a list of client side and server side validations. The server side validations will be returned from the microservices and are stored as a collection of strings within the ViewModel.  How can I surface these within the View?

3. How do I setup Bulk Updating in the above pattern?
I need some example that shows how to apply the Bulk Update pattern with the Custom Data Adaptor.  I am easily able to follow the logic in the custom data adaptor but how do I invoke this from the View?

4. What event to hook to know when the Query Builder has no conditions?

The Query Builder works perfect however when I apply conditions and then remove these conditions I need to have the Read fire in the Data Adaptor but instead I am left with the last query prior to removal of the last condition.  

In the following example, I apply the filter on step 1 and I filter all Organizations that start with "S".. On step 2 I remove that condition, and step 3 shows that I am still filtered because the data adaptor was not called when no further conditions were present...




JP Jeevakanth Palaniappan Syncfusion Team September 15, 2020 01:51 PM UTC

Hi John, 

Sorry for the delay in getting back to you. 

Query 1: How to disable UI 
 
We have validated your query and you can disable the UI by using the pointer-events css property as none. Add this css to any class and then based on your requirement add/remove the class from the DOM. For your reference we have added the css for a div with e-disable-ui class. Please find the below code snippet and the sample for your reference [Index.razor]. 

<style> 
. e-disable-ui{ 
    pointer-events: none; 
    opacity: 0.5; 
} 
</style> 

Query 2: The backing ViewModel has a collection of Validation Errors that can occur from backend business rules, how do I surface these in the view? 
 
We suspect that you need to show the custom validations in the grid. If so, we suggest you to achieve this by using Custom validation support in Data Annotation. We have provided this support in the latest Syncfusion NuGet version(18.2.0.58). Please refer the sample[TestValidation.razor] and the screenshot for your reference. 

 

In case if we misunderstood your query then please provide us more information about your requirement. 

Query3: How to bulk update in grid by using Custom Adaptor 
 
We have Batch editing feature to do bulk update in the grid. On editing the changes only gets reflected in the UI and only on clicking the update button in the toolbar, the changed values gets updated in the datasourc. In Custom Adaptor, the updation has to be handled in the BatchUpdate/BatchUpdateAsync method of the DataAdaptor. Please refer the below documentation link for further reference. 


Query: 4. What event to hook to know when the Query Builder has no conditions?   
  
We have checked your reported query, we are not able to reproduce it in our end. RuleChanged event triggered when click the remove condition. Data also loaded properly. For your convenience we have prepared a video. Please find the below link  
  
Please check the above video for your reference. 


Please get back to us if you need further assistance. 

Regards, 
Jeevakanth SP. 


JO John September 16, 2020 02:50 PM UTC

Where is the TestValidation.razor sample?  

JP Jeevakanth Palaniappan Syncfusion Team September 16, 2020 03:20 PM UTC

Hi John, 

We have attached the sample at the end of the previous response. We are attaching it below also for your reference. 


Regards, 
Jeevakanth SP. 


Loader.
Up arrow icon