Need more efficient caching of notifyingMemoryStore object

Description

The change in did not completely work as expected. Upon analyzing heap dumps, we found that NotifyingMemoryStore is still the biggest object by retained size in the cache. This is due to the use of inner class. We should make the new MemberWithRoleId class a top level class, instead of inner class of DbAuthzGroupService.

Attachments

1

Activity

Show:

Matthew Jones March 31, 2014 at 10:11 AM

Suggested as a way to reproduce larger sizes in the cache is a site that has more roles that normal. Would need to test that.

Matthew Buckett March 25, 2014 at 9:25 AM

In relation to the changes made in SVN against this ticket:

I don't follow how using an inner class would use more memory compared to a normal class.

The only difference between a normal class and an inner class is that the inner class holds a reference to the containing class. In this case DbAuthzGroupService should be a singleton as it's a spring managed bean so there should only be one copy in the JVM shared by all the inner classes.

Matthew Jones March 10, 2014 at 7:58 PM

This was open so I'm going to resolve this as incomplete based on the comments.

I think there might be some more improvements to be made around this cache still but it doesn't appear to be that bad in local tests.

The 125000 is probably too high though for a typical server (what the default is set at for this right now where most other defaults are 10000)

So 10K for 1 user, 100K for 10 users, 1MB for 100 users, 10MB for 1000 users so 1GB (at least) for 100000 users. It looks pretty predicatable to me? Even if the 80K is true that's only
really bad at 100K entries, and I'm not seeing that locally unless this is caching some much larger set?

org.sakaiproject.authz.impl.DbAuthzGroupService.realmRoleGroupCache: count:7 hits:15 misses:19 hit%:44 heap:124856
org.sakaiproject.authz.impl.DbAuthzGroupService.realmRoleGroupCache: count:108 hits:1103 misses:120 hit%:90 heap:1321272
org.sakaiproject.authz.impl.DbAuthzGroupService.realmRoleGroupCache: count:510 hits:10694 misses:522 hit%:95 heap:5653696

Matthew Jones March 10, 2014 at 6:34 PM

I can't seem to reproduce this any more on trunk. I get over 1000 items in the cache by using a script to login with 500 users and the heap memory graphs don't seem to look any different than with this cache turned off. I know ehcache was upgraded, but even after 1000 items in the cache, it hasn't increased by that much. Unless there's some edge cases that significantly increase this?

Sam Ottenhoff February 13, 2014 at 9:43 AM

Is UMich still working on this one?

Incomplete

Details

Priority

Affects versions

Fix versions

Components

Assignee

Reporter

Labels

Created April 10, 2013 at 7:07 AM
Updated April 25, 2018 at 3:20 PM
Resolved March 10, 2014 at 7:58 PM