Hi Nitin,
Thank you for your interest in Syncfusion product.
If you need the custom custom dialog in GridGroupingControl filter, then please try using CustomFilterBar in GridGroupingControl to acheive this behavior.
Please refer teh sample in below location:
http://files.syncfusion.com/support/samples/Grouping.Windows/Forum/GGCCustomFilter.zipLet me know if you have any queries.
Regards,
Lingaraj S.