excessive memory usage inside realmRoleGroupCache values

Description

RealmRoleGroupCache was introduced via KNL-600 for Kernel 1.3 tag for Sakai 2.9.

We have observed increasing memory usage problems with Sakai 2.9 code. In Yourkit, the RealmRoleGroupCache objects are on the top list of "Strong reachable" objects sorted by retained size.

Please see the attached image for the object sizes used in the realmRoleGroup cache. Basically, for each cache key (realm id), the cache contains all the roles definition, and all the realm member user id, WITH its ROLE definition again.

In the screenshot, one can see the ROLE object occupies 7K for each realm user been cached, compared to a String value of hundred bytes if the role id is used instead. This maybe a waste of large memory if dealing with 100x or 1000x realm members.

Once the realmRoleGroup cache object is allocated, it would remain in the memory unless there is an eviction due to space limit. However, the default setting of maxEntriesLocalHeap is 125000, which is a pretty big value. In CTools, the count of this cache is below 25K after one week's usage.

Though the cached value would get refreshed if it is hit after expiration time (3600s according to the default setting), it would never give up the space used. We think this is the major cause of the memory usage problem for our prod server nowadays.

Attachments

3
  • 19 Mar 2013, 02:32 PM
  • 19 Mar 2013, 01:26 PM
  • 18 Mar 2013, 01:46 PM

Activity

Show:

Matthew Buckett April 19, 2013 at 8:57 AM

Anthony Whyte April 1, 2013 at 8:05 PM

Merged kernel 1.3.x r122045.

Good faith, I'll prove him,
Speed how it will. I shall ere long have knowledge
Of my success.
Coriolanus, V, 1

Sam Ottenhoff April 1, 2013 at 9:06 AM

Tested on my local trunk. Confirmed high HIT cache ratio and basic regression testing still good.

Hudson CI Server March 19, 2013 at 3:17 PM

Integrated in kernel-trunk #615 (See http://builds.sakaiproject.org:8080/job/kernel-trunk/615/)
https://sakaiproject.atlassian.net/browse/KNL-1037#icft=KNL-1037 Fix for excessive memory use inside RealmRoleGroupCache
Patch from Zhen (UM) (Revision 121463)

Result = SUCCESS

Aaron Zeckoski March 19, 2013 at 2:32 PM

Attaching patch as applied: KNL-1037.patch.txt

Fixed

Details

Priority

Affects versions

Fix versions

Components

Assignee

Reporter

Created March 18, 2013 at 1:45 PM
Updated April 25, 2018 at 3:20 PM
Resolved March 19, 2013 at 2:19 PM

Flag notifications