Lessons loses text added to page

Description

We are having an issue that is only affecting our live instance where
when you are in a Lessons page and select to add some new text to the
page you enter some words and click "Save", the item always gets added
to the page, but sometimes the text contents you entered are lost, so
you end up with an empty item in the page. We can't reproduce this on
dev or test machines although did see it on a staging machine when it
was running slowly. You can then edit the item and add the text and
this always correctly updates the item (can't get this to fail).

Looking at the code it seems the transaction boundary is on all the
calls between SimplePageBean and SimplePageToolDao so any call across
this boundary will get a new transaction and possibly a new hibernate
sessions. When a new item is added to the page there appear to be 3
calls across this boundary:

  • The first one is a call to a saveItem() which adds the new row, but
    it doesn't yet have all it's properties set. We know this must be
    completing as we see this much in the DB.

  • The second one is a call to update() which is done when setting the
    groups on the item.

  • The third one is another call to update() which is called at the end
    which is redundant if there are groups.

In between the first and second calls the HTML description is set on
the object so that is hinting that everything after the first one
isn't working when we see a failure. My current guess is that on a
heavily loaded server we end up getting back a different
session/transaction after the first call and this is what causes it to
fail. Although from looking at the code any errors that result of
hibernate not managing to persist something should result in an error
showing up in the UI and we're not getting any in the UI or in the
logs.

Our hibernate sessions should be bound to threads but once the
transactions has been closed, I'm not sure if you will get the same
session back the next time.

Activity

Show:

Adam Marshall February 14, 2017 at 8:18 AM

sean Horner says

For the 11.x branch on http://nightly2.sakaiproject.org/, I’m observing a bug in the Lessons tool where editing an existing text block will instead copy that block and add the edited version to the bottom of the page, leaving the original intact.

Here are some steps to reproduce this bug:

1. On a blank Lessons page, click Add Content and Add Text.
2. In the CKEditor add some text and click Save.
3. Click the “Edit text box 1” button.
4. Append some more text like “This is some additional text” and click Save.
5. Note that unexpectedly there are now two text blocks on the page: the original text block on top and beneath it a text block with the edited content.

This bug is not evident in 11.2, but seems to be a regression introduced into 11.x since that time. This bug seems like it might be related to the fix that was applied for LSNBLDR-722, though that’s just a guess. Let me know if you think I should open a new JIRA for this.

Anirudh Nagulapalli January 10, 2017 at 12:04 PM

Tested in Trunk. Works Fine.

Charles Hedrick September 21, 2016 at 8:49 AM

more information: I tried doing things without an explicit lock. I was unable to come up with an approach that was free of deadlocks. The only safe approach I can think of would be to do LOCK IN SHARE MODE for all items on the page when doing inserts and deletes. I'm still not sure whether it's worth doing.

Charles Hedrick September 16, 2016 at 9:15 AM

OK, after further investigation, I believe there is a race-free way to insert items. both mysql (withe isolation level we use) and oracle say that they get locks on all items scanned in where clauses. So we can create a gap using

update lesson_builder_items set seq = seq + 1 where seq >= (select seq from lesson_builder_items where itemid = ?)

That will create a gap before the specified item. Even if other updates happen between now and the save, the gap will remain, though it's seuence number may change.

The final save can now be

insert into lesson_builder_items where ... seq = (select seq from lesson_builder_items where itemid = ?) - 1

But since doing that through hibernate may be hard. the transaction can be followed by

update lesson_builder_items set seq = (select seq from lesson_buildr_items where itemid = ?) where itemid = ?

The first ? is the item we're inserting before. The second is the itemid of the new item.

This will actually be more efficient than the current code, becaue it won't have to do a full hibernate retreive of al items on the page. The success of this approach depends upon the fact that I set dynamic-update, so hibernate will never touch the sequence number on its own.

I'm reasonably sure that the syntax of the necessary update statement is the same for mysql and oracle, so I can issue it using dbWrite. (I could also try using HQL, but this is a case where I have to be really sure what SQL is being issued.)

This only works before item insertion inserts before an existing item. I don't think I could preserve consistency in the face other other processes making changes if I had do insert after. If the item we're inserting after dispappears during the transaction (presumably because another process deleted it), I believe I can recover. In that case the update will fail, in the sense that it will return a row count of 0. I can remove the new item and generate an error for the user "item which you were inserting after has disappeared." One hopes this will never happen.

Charles Hedrick September 15, 2016 at 9:42 AM

New guidelines for developers:

  • When creating or editing an item, will need to do just one save, update, or saveOrUpdate

  • Any code that plays with sequence numbers will need to call setRefreshMode at the beginning. It must be called before any items on the page have been loaded into the Session.

Fixed

Details

Priority

Affects versions

Fix versions

Components

Assignee

Reporter

Labels

Created September 9, 2016 at 8:24 AM
Updated April 18, 2018 at 7:30 AM
Resolved September 15, 2016 at 9:28 AM