OutOfMemoryError and server down when loading a report with matrix grouped scales
GENERAL
TESTING
GENERAL
TESTING
Description
depends on
is related to
Miguel Pellicer
Activity
Earle Nietzel September 3, 2019 at 4:16 PMEdited
This bug is a regression from https://sakaiproject.atlassian.net/browse/EVALSYS-1558#icft=EVALSYS-1558
Sam Ottenhoff September 3, 2019 at 3:47 PM
I don't think this affects 12.x. Does it?
Daniel Merino August 21, 2019 at 7:52 AM
PR submitted at https://github.com/sakaicontrib/evaluation/pull/135
Just replacing add() by set() fixes the issue.
It seems that this bug has been latent for ages.
Daniel Merino August 21, 2019 at 6:04 AM
The offending code:
List<String> matrixLabels = RenderingUtils.getMatrixLabels(scaleOptions);
for (int i = 0; i < scaleLabels.size(); i++) {
String label = scaleLabels.get(i);
if (matrixLabels.contains(label)) {
scaleLabels.add(i, (i+1) + " - " + scaleLabels.get(i));
} else {
scaleLabels.add(i, String.valueOf(i+1));
}
}As you can see, scaleLabels.size() is increased in the block, so an infinite loop is done.
I suspect that the clone() function added an empty array to scaleLabels so this condition was never reached.
Daniel Merino August 21, 2019 at 5:31 AMEdited
The stack trace in my test server, just in case it helps:
Fatal internal error handling request:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.sakaiproject.evaluation.tool.utils.RenderingUtils.makeReportingScaleLabels(RenderingUtils.java:281)
at org.sakaiproject.evaluation.tool.producers.ReportsViewingProducer.renderTemplateItemResults(ReportsViewingProducer.java:359)
at org.sakaiproject.evaluation.tool.producers.ReportsViewingProducer.fill(ReportsViewingProducer.java:299)
at org.sakaiproject.evaluation.tool.producers.EvalCommonProducer.fillComponents(EvalCommonProducer.java:53)
at uk.org.ponder.rsf.view.support.ViewCollector.fillComponents(ViewCollector.java:56)
at uk.org.ponder.rsf.view.support.LayoutCollector.fillComponents(LayoutCollector.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy358.fillComponents(Unknown Source)
at uk.org.ponder.rsf.view.support.ViewGenerator.generateView(ViewGenerator.java:53)
at uk.org.ponder.rsf.processor.support.RSFRenderHandler$1.run(RSFRenderHandler.java:79)
at uk.org.ponder.rsf.processor.support.RequestInvoker$1.run(RequestInvoker.java:46)
at uk.org.ponder.util.CollectingRunnableInvoker$1.run(CollectingRunnableInvoker.java:25)
at uk.org.ponder.rsf.flow.support.BasicScopedAlterationWrapper.invokeRunnable(BasicScopedAlterationWrapper.java:59)
at uk.org.ponder.rsf.flow.support.BasicScopedAlterationWrapper$$FastClassBySpringCGLIB$$84f89202.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669)
at uk.org.ponder.rsf.flow.support.BasicScopedAlterationWrapper$$EnhancerBySpringCGLIB$$497bbcf7.invokeRunnable(<generated>)
at uk.org.ponder.rsf.flow.support.BasicScopedAlterationWrapper$$FastClassBySpringCGLIB$$84f89202.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:669)
at uk.org.ponder.rsf.flow.support.BasicScopedAlterationWrapper$$EnhancerBySpringCGLIB$$497bbcf7.invokeRunnable(<generated>)
at uk.org.ponder.util.CollectingRunnableInvoker$1.run(CollectingRunnableInvoker.java:29)
at uk.org.ponder.util.CollectingRunnableInvoker.invokeWrappers(CollectingRunnableInvoker.java:32)
at uk.org.ponder.util.CollectingRunnableInvoker.invokeRunnable(CollectingRunnableInvoker.java:14)
at uk.org.ponder.rsf.processor.support.RequestInvoker.invokeRunnable(RequestInvoker.java:39)
at uk.org.ponder.rsf.processor.support.RSFRenderHandler.handle(RSFRenderHandler.java:75)
at uk.org.ponder.rsf.processor.support.RSFRenderHandler$$FastClassBySpringCGLIB$$5bda3a3d.invoke(<generated>)
We have experienced server crashes after aplying https://sakaiproject.atlassian.net/browse/EVALSYS-1567#icft=EVALSYS-1567 patch.
In some specific reports, trying to generate the report causes a high CPU and a java.lang.OutOfMemoryError: GC overhead limit exceeded. In one of our test servers the full server needs a reboot.
After a deep research of our IT service, we have determined that the issue happens only in reports that contain matrix grouped scales.
I haven't been able to test this in a nightly server as Evaluations is not available in them (neither in the experimental server). Because this, I set the affected versions to 12.x but I think that the issue will also happen in 19x and 20.x as the code is the same.