Script initialization infinite loop causes out of memory problems

[Bart Gommers]

Hi,

We're running into a problem that every so often our application runs out of memory. After creating memory dumps it seems that "InvokeDeviceMode" (a method in Syncfusion.Blazor.dll in SfBaseUtils class) is stuck in an infinite loop, causing the process to run out of memory. We're not sure what triggers this loop, that seems to be random, but after inspecting the InvokeDeviceMode method with DotPeek it seems that this implementation is very naive and can easily get stuck in a recursive loop when an exception is thrown. This piece of code is very low quality and is causing our process to crash. Please address this issue as soon as possible. 

I've already tried disabling the script manager (by calling services.AddSyncfusionBlazor(true)), this seems to lessen the occurances but does not completely solve it. 

Memory dump:

Offending code:

[Muthukumar Kannan]

Hi Bart, 

Thanks for contacting Syncfusion Support.

We are validating your reported query for Memory leaks in Syncfusion initialization. However, we will update the further details within 2 business days (3rd December 2020).

Until then we appreciate your patience.

Kindly please let us know if you have any concerns.

 

Regards,

Muthukumar K

[Muthukumar Kannan]

Hi Bart,

 

Thanks for contacting Syncfusion.

 

 

We have checked your query with Memory leaks in Syncfusion initialization. Unfortunately, we cannot able to reproduce the reported issues from our end. So, we need some more information about your reported issues. Could you please share the below details to proceed further on this?

1. Is reported issue in Server or Wasm?
2. What is the target framework? .NET 5.0 or netcore3.1/netstandard2.1?
3. What are Syncfusion components do you used in that issue replication sample?
4. If possible, could please issue replicating snippets?
   

The above details will help us to provide better support from our side.

Please let us know if you have any concerns.

Regards,

Muthukumar K

[Bart Gommers]

Hi Muthukumar,

To answer your questions:

1. Server

2. .NET Core 3.1

3. We're using SfGrid, SfDialog, SfToast, SfProgressButon, SfComboBox, SfRadioButton, SfMaskedTextBox, SfButton, SfNumericTextBox, SfDataManager

4. I am not able to reproduce this locally. This happens a couple of times per day on production. The service will slowly but steadily climb to gigabytes of memory usage until it runs out. The load is about a few hundred users per day.

But to put it bluntly, to me the problem from the memory dump and the (decompiled) code dump make the problem obvious. This InvokeDeviceMode method (from SfBaseUtils in Syncfusion.Blazor.dll) is stuck in a recursive loop, probably because an exception is thrown, and in the catch there is a recursive call to itself. This is obviously a very bad design pattern and should be addressed regardless of being able to reproduce the issue. 

As this is causing serious issues on our production environment I would like to know when I can expect that your development team will address this bug. 

We are already considering dumping Syncfusion in favor of a better designed component library.

[Muthukumar Kannan]

Hi Bart,

Thanks for your detailed information.

Currently, we are validating your reported issue with high priority since it's related to memory utilization. However, we will update the status and further details within 2 business days. (9th December 2020).

Until then we appreciate your patience.

Please let us know if you have any concerns about this.

Regards,
Muthukumar K

[Muthukumar Kannan]

Hi Bart, 

 

Thanks for your patience.

During validation, we are facing difficulties to replicate this in our environment.  So, we need some time to validate and minimal replication of the issue.  We are working on this with prioritized and we will update the details within 3 business days (14th December 2020) without any further delay.

Until then we appreciate your patience.

Please get back to us if you have any concerns about this.

 

Regards,

Muthukumar K

[Bart Gommers]

Hi  Muthukumar,

We didn't hear from you by the indicated date. Can you please give an update asap. This issue is causing serious issues on our production environment.

Thanks

[Muthukumar Kannan]

Hi Bart,

 

Thanks for your patience.

As per our validation, we can able to reproduce the Memory loop issue from our end. Currently, we are working on this with high priority to resolve it ASAP since its related for memory utilization in production. However, we will provide the details for this within 17th December 2020.

Until then we appreciate your patience.

Please let us know if you have any concerns

Regards,

Muthukumar K

[Bart Gommers]

Hi Muthukumar,

Do you have an update for me?

Thanks,

Bart

[Muthukumar Kannan]

Hi Bart,

Sorry for the delayed update and thanks for your patience.

We are glad to inform you that we have minimized the reported 'infinite looping during Syncfusion initiation' production issue in our latest v18.4.0.30 Volume 4 2020 package. Could you please check this issue with the latest package?

https://www.syncfusion.com/forums/160733/essential-studio-2020-volume-4-release-v18-4-0-30-is-available-for-download

  

https://www.nuget.org/packages/Syncfusion.Blazor/18.4.0.30
 

  

Also, we are working on clearing this looping in initiation. However, we will provide the details of this issue details within 2 business days. 

Please let us know if you have any concerns about this.

Regards,

Muthukumar K 

  

  

[Muthukumar Kannan]

Hi Bart,

Thanks for your patience.

Is that the looping issue minimized as we mentioned in your environment with our latest package?. Currently, we are working on trying to clear the overall memory looping from our side with high priority. If you ensured with our Volume 4 2020 (18.4.0.30) package, please share the concerns/details about the looping issue which was minimized or not.

Please let us know if you have any questions about this.

Regards,
Muthukumar K

[Bart Gommers]

Hi Muthukumar,

Thanks for following up.

It still happens, but less. It used to be 3 or 4 times a day, now it's down to maybe once per day or less. The behaviour seems to be different too, last Saturday I saw memory "jump" between 1GB and 4GB. So it looks like it's freeing up memory where it wasn't before. Still it shouldn't be up to 4GB of memory, so not sure what is happening now.

I'll try to make another memory dump when it's at 4Gb again, to see what is different now.

Regards,

Bart

[Muthukumar Kannan]

Hi Bart,

Thanks for your update.

Please let us know if you face again these memory looping issue. Also, we have working on clearing this overall looping issue from our end as we stated earlier, and share the details of this on 24th December 2020.

Please get back to us if you have any concerns.

Regards,
Muthukumar K

[Bart Gommers]

Hi Bart,

Thanks for your update.

Please let us know if you face again these memory looping issue. Also, we have working on clearing this overall looping issue from our end as we stated earlier, and share the details of this on 24th December 2020.

Please get back to us if you have any concerns.

Regards,
Muthukumar K

Hi Muthukumar,

Do you have an update for me?

Thanks,

Bart

[Muthukumar Kannan]

Hi Bart,

Thanks for your patience.

Currently, we have working on clearing this overall looping issue from our end as we stated earlier. And now we have fixed this by changing the source level script rendering method and need to test our components with these changes. We have planned to include this fix for clearing the memory leak issue on the upcoming patch release scheduled for 30th December 2020. We will update you once it released.

Until then we appreciate your patience.

Please get back to us if you have any concerns.

Regards,
Muthukumar K

[Muthukumar Kannan]

Hi Bart,

Thanks for your patience.

We are glad to inform you that we have reduced the maximum memory heap issues from our latest weekly patch release package as we promised. So, could you please check your memory heap issue with our latest v18.4.0.32 'Syncfusion.Blazor' package?

https://www.nuget.org/packages/Syncfusion.Blazor/ 

Please let us know whether your reported issue is resolved or not.

Also, we have provided the individual package support for our syncfusion blazor components. So, we can use the specific package alone for the respective components we used in the application. Please refer to the below documentation for further details regarding Individual packages.

https://blazor.syncfusion.com/documentation/nuget-packages/ 

This individual package provides better memory consumption when compared to the overall package

Please let us know if you have any concerns about this.

Regards,
Muthukumar K
 

[Bart Gommers]

Hi Muthukumar,

Thank you for the update. Unfortunately this update complete breaks the grid for us. All I've done is update the Syncfusion.Blazor nuget and after trying the page I'm greeted with the following exception, and a process that crashes completely(!).

The service threw an unhandled exception, System.InvalidOperationException: The current thread is not associated with the Dispatcher. Use InvokeAsync() to switch execution to the Dispatcher when triggering rendering or component state.

   at Microsoft.AspNetCore.Components.Dispatcher.AssertAccess()

   at Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToRenderQueue(Int32 componentId, RenderFragment renderFragment)

   at Microsoft.AspNetCore.Components.ComponentBase.StateHasChanged()

   at Syncfusion.Blazor.Grids.SfGrid`1.DataProcess(ActionArgs action, ActionEventArgs`1 actionArgs)

   at Syncfusion.Blazor.Grids.SfGrid`1.OnAfterScriptRendered()

   at Syncfusion.Blazor.SfBaseComponent.InvokeScriptRendered()

   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__139_1(Object state)

   at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi)

   at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[TState](ExecutionContext executionContext, Action`1 callback, TState& state)

   at System.Threading.QueueUserWorkItemCallback.Execute()

   at System.Threading.ThreadPoolWorkQueue.Dispatch()

   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

[Muthukumar Kannan]

Hi Bart,

Thanks for your update.

We have checked your reported issue from our end, and we have already fixed this issue in our development. However, we will include this fix in the upcoming weekly patch release on 5th January 2021.

Sorry for the inconvenience.

Please let us know if you have any concerns about this.

Regards,

Muthukumar K

[Bart Gommers]

Hi Bart,

Thanks for your update.

We have checked your reported issue from our end, and we have already fixed this issue in our development. However, we will include this fix in the upcoming weekly patch release on 5th January 2021.

Sorry for the inconvenience.

Please let us know if you have any concerns about this.

Regards,

Muthukumar K

Hi Muthukumar,

We've updated to the latest version and have not seen the issue for two days now. So looks like it's sorted. 

Thanks,

Bart

[Muthukumar Kannan]

Hi Bart,

Thanks for your update.

We are glad to hear that your reported issue has been resolved.

Please get back to us if you need any further assistance. We would be happy to assist you.

Regards,
Muthukumar K