Page count, size, goToPage issues when using (remote) local data

[Jake Bleazby]

Hello,

I am having a few issues issues when using the grid component with local data binding. Strictly speaking the data is not local at all; I fetch 6 pages at a time and then fetch more when the last page is viewed, and I fetch new data whenever the user sorts or filters. The issues are:

The page size is frequently ignored. The easiest to reproduce instance of this is when I skip to end once (causing data to be appended to the data source) and then hover over filter in the column menu. But it does occur under other circumstances if the data source is updated many times.

The number of pages as displayed at the bottom of the table does not update when additional data has been added. I've managed to work around this by calling goToPage twice, but it'd be good if I didn't have to.

When goToPage(1) is called after columnClick sorting or filtering event, instead of going to the first page the table goes to the first page viewable on screen (so If I have 1-5 on screen and skip to end so that 6-10 appears on screen, goToPage(1) sends me to page 6).

I've noticed in other threads you frequently suggest using your Data Manager. I'd strongly prefer not to as my back end services are complex and already working, and the data model is determined at run time.

I'll attach my code in a follow up comment.

Thanks in advance,

Jake.

[Jake Bleazby]

import React from 'react';

import "./material.min.css"


import {
   DetailRow,
   Inject,
   Filter,
   Page,
   Sort,
   Reorder,
   Resize,
   ColumnMenu,
   ExcelExport,
   PdfExport,
   Toolbar
} from "@syncfusion/ej2-react-grids";


import { GridComponent } from "@syncfusion/ej2-react-grids/src/grid/grid.component";
import { ColumnDirective, ColumnsDirective } from "@syncfusion/ej2-react-grids/src/grid/columns-directive";
import { arraysEqual, camelToTitle } from "../../../utils";


class GeneratedReport extends React.Component{


   constructor(props) {
      super(props);
      this.state = {
         sortBy: undefined,
         sortAscending: true,
         filterBy: undefined,
         filterOperator: undefined,
         filterValue: undefined,
      };
      this.grid = null;
   }


   componentDidUpdate(prevProps, prevState, snapshot) {
      if(prevProps.data && this.props.data && !arraysEqual(prevProps.data[0], this.props.data[0])){
         this.grid.dataSource = {
            result: this.props.data ? this.props.data[0] : [],
            count: this.props.count
         };
      }
   }


   renderComplete = () => {
      if(!this.grid || this.grid.dataSource.length <= 0) {
         this.dataStateChange();
      }
   }


   dataStateChange = (args = {skip: 0, take: 14}) => {
      this.grid.dataSource = {
         result: this.props.data ? this.props.data[0].slice(args.skip, args.take) : [],
         count: this.props.count
      };
   }


   toolbarClick = (args) => {
      let exportFn = null;


      if(!this.grid || !this.props.data || !this.props.data[0]) return;
      else if(this.props.data[0].length > 2000 &&
         ((args.item.id.includes('pdfexport') || args.item.id.includes('excelexport') || args.item.id.includes('csvexport') || args.item.id.includes('Tab Delimited Export')))){
         this.props.appFuncs.openAlert(
            'Export Error',
            ['Report contains over 2,000 entries. Add additional filters in order to proceed.'],
            [{ text: 'OK' }]
         );
         return;
      }
      else if (this.grid && args.item.id.includes('pdfexport')) {
         exportFn = this.grid.pdfExport;
      }
      else if (this.grid && args.item.id.includes('excelexport')) {
         exportFn = this.grid.excelExport;
      }
      else if (this.grid && args.item.id.includes('csvexport')) {
         exportFn = this.grid.csvExport;
      }
      else if (this.grid && args.item.id.includes('Tab Delimited Export')) {
         exportFn = () => this.grid.csvExport({separator: '\t'});
      }


      if(exportFn != null) this.props.generate(exportFn, this.state.sortBy, this.state.sortAscending, this.state.filterBy, this.state.filterOperator, this.state.filterValue, 0, 2001);
   }


   columnClick = (args) => {
      if(args && args.currentPage && args.currentPage + 1 >= (this.props.count / 14) ){
         let callBack = () => {
            this.grid.goToPage(args.currentPage - 1);
            this.grid.goToPage(args.currentPage);
         };
         this.props.generate(callBack, this.state.sortBy, this.state.sortAscending, this.state.filterBy, this.state.filterOperator, this.state.filterValue, this.props.count / (6*14));
      }
      if(args && args.requestType && args.requestType === "sorting"){
         this.setState(
            {sortAscending: args.direction === "Ascending", sortBy: args.columnName},
            () => this.props.generate(() => this.grid.goToPage(1), this.state.sortBy, this.state.sortAscending,
               this.state.filterBy, this.state.filterOperator, this.state.filterValue)
         );
      }
      else if(args && args.requestType && args.requestType === "filtering"){
         this.setState(
            {filterBy: args.currentFilterObject.field, filterOperator: args.currentFilterObject.operator, filterValue: args.currentFilterObject.value},
            () => this.props.generate(() => this.grid.goToPage(1), this.state.sortBy, this.state.sortAscending,
               this.state.filterBy, this.state.filterOperator, this.state.filterValue)
         );
      }
   }


   dataBound() {
      if(!this.props.data) {
         this.grid.columns = [];
      }
   }


   render() {
      let columns = [];
      if(this.props.data && this.props.columnOrder) {
         this.props.columnOrder.forEach(x => {
         if(!x.endsWith('Id'))
            columns.push(<ColumnDirective
                  field={x}
                  headerText={camelToTitle(x)}
                  textAlign="Left"
               />)
         });
      }


      this.renderComplete = this.renderComplete.bind(this);
      this.dataStateChange = this.dataStateChange.bind(this);
      this.toolbarClick = this.toolbarClick.bind(this);
      this.columnClick = this.columnClick.bind(this);


      return (
         <div className="control-pane">
            <div className="control-section">
               <GridComponent
                  allowPaging={true}
                  dataSource={this.props.data ? this.props.data[0] : []}
                  pageSettings={{ pageSize: 14, pageCount: 5 }}
                  dataBound={this.renderComplete}
                  dataStateChange={this.dataStateChange}
                  ref={g => this.grid = g}
                  toolbarClick={this.toolbarClick}
                  allowFiltering={true}
                  allowReordering={true}
                  allowTextWrap={true}
                  allowEditing={true}
                  allowSorting={true}
                  allowMultiSorting={false}
                  allowResizing={true}
                  showColumnMenu={true}
                  allowExcelExport={true}
                  allowPdfExport={true}
                  allowGrouping={true}
                  toolbar={['ExcelExport', 'PdfExport', 'CsvExport', 'Tab Delimited Export']}
                  filterSettings={{ type: 'Menu' }}
                  hierarchyExportMode='All'
                  actionBegin={this.columnClick}
               >
                  <ColumnsDirective>{columns}</ColumnsDirective>
                  <Inject services={[DetailRow, Page, Sort, Reorder, Filter, Resize, ColumnMenu, ExcelExport, PdfExport, Toolbar]} />
               </GridComponent>
            </div>
         </div>
      );
   }
}


export default GeneratedReport

[Joseph Christ Nithin Issack]

Hi Jake,

  Greetings from Syncfusion support.

  Based on the provided information, we could understand that you are trying to fetch some amount of records at a time and when you reach the last page, fetching another set of data. This is not a valid case in EJ2 Grid binding the data.

  By default the EJ2 Grid accepts either remote data or the local data. But in your case you are trying to bind data using both local and remote concepts. And also the pager will be updated based on the total record count returned from the server. If you fetch the records part by part the pager will not know the total records in the database and cannot render the pager properly and select the correct page.

    However in EJ2 Grid, we can be able to fetch the data from the server for each page using the `URL Adaptor` which handles the grid actions on-demand concept. Using URL Adaptor you can perform paging from the server side so that the data will be fetched from the datasource for each page. Please refer the below documentation for details on URL Adaptor.

Documentation: https://ej2.syncfusion.com/react/documentation/data/adaptors/#url-adaptor

Please get back to us for further details.

Regards,

Joseph I.

[Jake Bleazby]

If the data being non-static and local is not a valid use case, how about destroying the grid and creating it again whenever data changes?

As for the pager, calculating the maximum number of pages is prohibitively expensive (could take 10 minutes). Also outside of the grid I can change datasources (part of why I want to handle it myself), resulting in a different maximum number of pages. I'm okay with the number presented being wrong and just assuming there is always one more page, or just not displaying the total number of pages. 

I understand that there is some confusion between local and remote concepts. Basically I want to handle all remote concepts myself, without Syncfusion calling any external API or Database. Perhaps there is a way I could mock up an api call so that it just ignores the response and returns data from a custom function (which may or may not access the internet; none of Syncfusion's business).

I've played around with a custom Ajax binding that does just that, and it works a bit better but it has different paging issues. When I get to the end of my data and start feeding my DataService empty arrays the spinner appears and never goes away. Which I suspect will continue to be an issue when I suddenly change data sources and start pulling data with a different (unknown) maximum number of results. 

I'll include my second attempt below. 

[Jake Bleazby]

import * as React from 'react';
import "./material.min.css"
import {
   GridComponent,
   ColumnsDirective,
   ColumnDirective,
   Page,
   Sort,
   Inject,
   DetailRow,
   Filter,
   Reorder,
   Resize,
   ColumnMenu,
   ExcelExport,
   PdfExport,
   Toolbar
} from "@syncfusion/ej2-react-grids";
import { camelToTitle } from "../../../utils";


export class DataService {
   execute(state, func, count) {
      return this.getData(state, func, count);
   }


   getData(state, func, count) {
      console.log("state", state);


      return func(
         () => {},
         state.sorted ? state.sorted[0].name : undefined,
         state.sorted ? state.sorted[0].direction === 'ascending': true,
         state.where ? state.where[0].predicates[0].field : undefined,
         state.where ? state.where[0].predicates[0].operator : undefined,
         state.where ? state.where[0].predicates[0].value : undefined,
         state.skip / 14,
         14
      ).then(data => {
         console.log(data);
         return {
            result: data[0],
                count: (state.skip + 2) * state.take
         };
      });
   }
}


export default class GeneratedReport extends React.PureComponent {
   constructor(props) {
      super(props);
      this.dataService = new DataService();
      this.grid = undefined;
      this.data = this.props.data[0];
   }


   rendereComplete() {
      let state = { skip: 0, take: 14 };
      this.dataStateChange(state);
   }


   dataStateChange(state) {
      this.dataService.execute(state, this.props.generate, this.props.count).then((gridData) => { this.grid.dataSource = gridData; });
   }


   componentDidMount() {
      setTimeout(() => {
         this.rendereComplete();
      });
   }


   render() {
      let columns = [];
      if(this.props.data && this.props.columnOrder) {
         this.props.columnOrder.forEach(x => {
            if(!x.endsWith('Id'))
               columns.push(<ColumnDirective
                  field={x}
                  headerText={camelToTitle(x)}
                  textAlign="Left"
               />)
         });
      }


      return (<div className='control-pane'>
         <div className='control-section'>
            <GridComponent
               allowPaging={true}
               dataSource={this.data}
               pageSettings={{ pageSize: 14, pageCount: 5 }}
               dataBound={this.renderComplete}
               dataStateChange={this.dataStateChange.bind(this)}
               ref={g => this.grid = g}
               toolbarClick={this.toolbarClick}
               allowFiltering={true}
               allowReordering={true}
               allowTextWrap={true}
               allowEditing={true}
               allowSorting={true}
               allowMultiSorting={false}
               allowResizing={true}
               showColumnMenu={true}
               allowExcelExport={true}
               allowPdfExport={true}
               allowGrouping={true}
               toolbar={['ExcelExport', 'PdfExport', 'CsvExport', 'Tab Delimited Export']}
               filterSettings={{ type: 'Menu' }}
               hierarchyExportMode='All'
               actionBegin={this.columnClick}
            >
               <ColumnsDirective>{columns}</ColumnsDirective>
               <Inject services={[DetailRow, Page, Sort, Reorder, Filter, Resize, ColumnMenu, ExcelExport, PdfExport, Toolbar]} />
            </GridComponent>
         </div>
      </div>);
   }
}

[Joseph Christ Nithin Issack]

Hi Jake,

  Thanks for your update.

  Currently we are validating your query, we will provide further details on or before May 10^th 2022. We appreciate your patience until then.

Regards,

Joseph I

[Joseph Christ Nithin Issack]

Hi Jake,

  Sorry for the inconvenience caused.

  We are still validating your query, we will provide further details on or before May 12^th 2022, we appreciate your patience until then.

Regards,

Joseph I

[Nithya Sivaprakasam]

Hi Jake,

Sorry for the inconvenience caused.

Currently, we are checking your query and we will update further details on May 18th, 2022. Until then, we appreciate your patience.

Regards,

Nithya S

[Joseph Christ Nithin Issack]

Hi Jake,

  Sorry for the late reply.

  Query 1: If the data being non-static and local is not a valid use case, how about destroying the grid and creating it again whenever data changes?

   Before proceeding to the solution please confirm if you want to load the new data to the grid or you want to destroy the entire control and create it again? Also confirm if there will be changes in the columns when you change the datasource of the grid.

Query 2: As for the pager, calculating the maximum number of pages is prohibitively expensive (could take 10 minutes). Also outside of the grid I can change datasources (part of why I want to handle it myself), resulting in a different maximum number of pages. I'm okay with the number presented being wrong and just assuming there is always one more page, or just not displaying the total number of pages.

  By default when you use the remote data we return the current view data only from the database as a JSON object with properties result and count which contains the collection of entities and the total number of records respectively.  You can refer the below documentation for more details on remote and local data binding.

Documentation: https://ej2.syncfusion.com/react/documentation/data/data-binding/

Query 3: Basically I want to handle all remote concepts myself, without Syncfusion calling any external API or Database. Perhaps there is a way I could mock up an api call so that it just ignores the response and returns data from a custom function (which may or may not access the internet; none of Syncfusion's business).

  Based on your requirement, you want to perform all the actions using your own api calls and not any other default adaptors. We suggest you to use the custom binding feature which helps you to achieve your requirement. We have explained these approaches in detail below for your reference,

Custom binding:

If you are using the API calls to fetch and perform Grid action from the server, then we suggest you to custom binding approach to bind data in the Grid. With this you can bind data from an API call by providing your own custom queries(as required by your API) and handle all the Grid actions(Search, Sort, Page, Filter, etc. ) with it from your end. The Grid’s custom binding approach is explained below,

For using custom binding, you need to bind the response data(Grid data) returned from your API as an object of result(JSON data source) and count(Total data count) properties and set it to the Grid’s dataSource property. When the custom binding is implemented in the Grid the dataStateChange(Grid action)/dataSourceChanged(CRUD action) event will be triggered for the corresponding action along with the query details returned in the event arguments. These event arguments will return the corresponding action details from the source with which you can handle the action in your API and return back the response to the Grid.

So when the filter/sort/page actions are performed in the Grid, the action details will be returned in the dataStateChange event as shown in the below image(shows filter action details),

From this event you can form the filter/page/sort queries in the format as required by your API, process the action in your API service, return back the response and bind it to the Grid dataSource as an object of ‘result’ and ‘count’.

Similarly when the CRUD action is performed in the Grid, the action details will be returned in the dataSourceChanged event as shown in the below image,

And in this event you can update the details in your API and on success of this action you need to call the state’s(The dataSourceChanged event argument) endEdit method in order to update the changes in the Grid. This will automatically trigger the dataStateChange event from where you need to call method for fetching the updated data from the API and assigning it to the Grid data as an object of ‘result’ and ‘count’.

Please refer the below documentation for more details on custom binding.

Documentation: https://ej2.syncfusion.com/react/documentation/grid/data-binding/data-binding/#custom-binding

Api: https://ej2.syncfusion.com/react/documentation/api/grid/#datastatechange

https://ej2.syncfusion.com/react/documentation/api/grid/#datasourcechanged

https://ej2.syncfusion.com/react/documentation/api/grid/#datasource

Query 4: I've played around with a custom Ajax binding that does just that, and it works a bit better but it has different paging issues. When I get to the end of my data and start feeding my DataService empty arrays the spinner appears and never goes away. Which I suspect will continue to be an issue when I suddenly change data sources and start pulling data with a different (unknown) maximum number of results.

   By default when you change the data, the spinner will be shown before the data is loaded to the grid. Please share the details how you are changing the datasource and returning the changed data to the grid?

Please get back to us for further details.

Regards,

Joseph I.