Purpose

Provide a scalable messages and forums synoptic view tool where a user will have a quick overview of their new messages and forum threads count.

About

To achieve this goal, I have created a tool that utilizes a database table that keeps track of all users new message counts. The synoptic table is updated when a user does an action that will change either their own message count or another user's message count. This new synoptic tool consults this table and displays the information provided for the user. This exponentially increases performance since there is only one row of data that needs to be queried to provide synoptic information for the messages and the forums tools. The trade off in performance comes in the form of updating the table information when an action that changes any users new message count occurs.

To present users with a seamless upgrade, the new synoptic tool mimics the original messages and forums synoptic tool. Also, since all synoptic information is stored in a table, I have provided a quartz job that can be ran before deployment to populate this table. Since this tool has been designed to dynamically update and insert new data whenever a synoptic update action is triggered, this tool will work without populating the synoptic information through the quartz job. The difference between these options would be a lag of information while the data is populated reactively through the tool compared to instantly populating data through a batch job.

This tool will replace both the synoptic view in a user's "MyWorkspace" site as well as the synoptic view in a site's "Home" page. The look and functionality of the synoptic tool in a site's "Home" page will not be changed. The synoptic tool in a user's "MyWorkspace" site will have all the same functionality of the original synoptic view plus table ordering features and an option for hiding a site in the synoptic view. Each header of the table may be sorted by ascending or descending order, and, by holding down the "shift" key and clicking multiple headers, a user may order by multiple rows at a time. The "options" link in the new synoptic tool allows a user to decide if they would like to hide or show each site in their list. When a user clicks this link, a new "Hide Site" column and a checkbox for each row will appear. By checking a row in the "Hide Site" column and clicking update, a user will no longer see that site in their list. Also, if a user wants to show a site that they have hidden, they can uncheck that box and click update. The "Hide Site" column can also be sorted.

When the synoptic table first populates on the "MyWorkspace" site, it gets ordered by consulting a user's site order preference. When a user changes the order of their sites in the "Preferences" tool under "Customize Tabs" option, the synoptic tool's site order will change as well. I have created the sorting algorithm as a pluggable feature and can be changed easily. There is an example of a sorting algorithm that sorts by course site first, with the order based on semester and year, then by all other site types alphabetically. If a user has hidden a site in their preferences, that site will not appear in the list. A site will only appear in the list if it has not been hidden in either the user's preferences or in the synoptic options.

Profiling Data

In order to better understand the profiling data, you need to know what is happening behind the scenes. The original synoptic tool would data-mine the synoptic information for each user in each site every time the tool is loaded by using a series of queries. This means that the only performance affected for the old tool would have been in loading the synoptic tool itself. Alternatively, the new synoptic tool is a reactive tool that updates and stores synoptic information in a table for each user in each site. An update to the new synoptic table is triggered by an action by a user that changes any user's synoptic information. These actions can be broken into two types of categories: simple and complex changes.

A simple change can be easily updated since the action has a limited effect range and can easily be determined how to increment or decrement a user's new messages count. The update to the synoptic table for a simple action is fast and has little affect on that action's performance. Simple type actions include: sending a message, deleting a message, marking a message as read/unread, reading a message, and posting a thread to a forum.

A complex action is more complicated and can have a variety of affects that are unknown to the synoptic tool without using performance-slowing queries. Since the consequences of this type of action are unknown, the quickest and most accurate way of updating the synoptic tool is by refreshing each effected user's synoptic information for that tool. For the messages tool, this refreshing is done by replacing the current new messages count with a quick recount of unread messages for that user. For the forums tool, the refresh will look at the specific forum and, if possible, the specific topic and compare the new messages count before and the count after the action and increment the synoptic count based on the difference between the before and after counts. Complex actions include: deleting a forum or topic, updating forum or topic settings, deleting a messages folder, approving or denying a message (this is because of a cases like when a user could have read an approved message, then it gets denied and then reapproved), and deleting a forum thread.

I have attached a spreadsheet that breaks down the performance of the old and new synoptic tool. The synoptic profiling was tested against two sites for a quick comparison of the two different tools. The profiling was run against my local machine using a local build of Oncourse (IU's Sakai implementation) running on IU's practice database. Since this data was not tested on production, the load times will be slower than that of a production server, but the important data are the performance percentage increase and decrease values. Also, the spreadsheet doesn't take into consideration any weighted variables like the number of times on average an action takes place (ie. The MyWorkspace page is loaded more times than any other action, and hence, according to Amdahl's law, is more important to speed up than updating forum settings action).

Attached File: Synoptic Profiling Data Spreadsheet

Questions

What happens if a school deploys Forums but not Messages? Or vice-versa?
&
What happens if a site uses Forums, but not Messages? Or vice-versa?

The tool checks to see if each site in the Synoptic Table (where it saves the counts for quick retrieval) has either the messages, forums or the message & forums tool. A site gets added to the table only if there has been some activity in the messages or forums tool or during the Quartz load job (which also checks each site for these tools). So if messages or forums or messages&forums tools are not registered for that site, the Site will not show up in the list. If only the messages or the forums tool has been added to the site, then the one that is missing will have a non-linked "none" in the count column. This is being done every time the tool is loaded, so if you remove one of these tools, then it will update. It will also check that the site exists and that it is published.

Update (11/02/09)
With the addition of msgcntr.synoptic.disable.forums and msgcntr.synoptic.disable.messages sakai.properties, a school that has either Messages or Forums only, will have the choice to remove the disabled tool from the MyWorkspace synoptic view.  Read below under Implemenntation -> Settings -> Sakai.properties for more details.

Can this synoptic tool be included with others on a page, or does it need its own?

Yes. We (IU) have our version running on the MyWorkspace Home page.

Implementation

Check out https://source.sakaiproject.org/svn/msgcntr/branches/SAK-15655 This branch was branched from Sakai trunk at revision 67689.  Also, here is a patch of the diff between the SAK-15655 branch and trunk at revision 67689: Synoptic Patch

Update (11/04/09)
This new synoptic tool is now in Sakai msgctnr trunk.

Current sakai jira is located at http://jira.sakaiproject.org/browse/SAK-15655

Settings:

• If you are planning on running the Quartz load job and you are running on an Oracle DB, for better performance, set runOracleSQL flag to true in UpdateSynopticMessageCounts.java.
• Sakai.properties:
  • msgcntr.synoptic.myworkspace.performance
    • =1 #1: Show direct links to tools
    • =2 #2: Hide direct links to tools (10x faster)
  • msgcntr.synoptic.myworkspace.userRequestSynoptic
    • =true #If true, then a link will show up to ask the user to click to see the synoptic info in their MyWorkspace
    • =false #Synoptic info will load automatically in user's MyWorkspace 
  • msgcntr.synoptic.disable.forums
    • =true #If True, the column for forums on MyWorkspace will be removed
    • =false #If False, the column for forums on MyWorkspace will be visible
  • msgcntr.synoptic.disable.messages
    • =true #If True, the column for messages on MyWorkspace will be removed
    • =false #If False, the column for messages on MyWorkspace will be visible
  • msgcntr.synoptic.myworkspace.disable
    • =true #If True, synoptic MyWorkspace will be disabled
    • =false #If False, synoptic MyWorkspace will be visible
  • msgcntr.synoptic.myworkspace.disabledMessage
    • =Text for users to read #This is used when msgcntr.synoptic.myworkspace.disable=true and the institution wants to override the disabled message w/o needing a code change.
• Sakai.properties defaults (If no sakai.properties are set for the synoptic tool, the defaults would look like this)
  • msgcntr.synoptic.myworkspace.performance=1
  • msgcntr.synoptic.myworkspace.userRequestSynoptic=false
  • msgcntr.synoptic.disable.forums=false
  • msgcntr.synoptic.disable.messages=false
  • msgcntr.synoptic.myworkspace.disable=false
  • msgcntr.synoptic.myworkspace.disabledMessage=The Check for new messages link has been temporarily disabled in My Workspace. Messages and Forums Notifications are still available on the Home page of any site.
    • *This default is from Messages.properties

DDL:

• Synoptic DDL (MySql)
• Synoptic DDL (Oracle)