Content Hosting Project

Content Hosting (CH) is currently implemented via a two-tier set of objects: one to store content and properties in a datbase, the other to store content in files. If content is saved in files and properties are kept in the database, this presents a serious problem with resource integrity. For example, if a file is deleted in Sakai, and the user asks their IT people to restore it, the metadata is lost, potentially corrupting the Content Hosting system. Also, the implementation uses an internally defined method of naming files and directories that makes restoring files nearly impossible.

Given a desire to support restoring lost files and external file system management (via AFS, etc.), I am exploring a new implementation of Content Hosting based solely on a standard file system. Naturally, there are trade-offse with this approach. It may not be as fast as the existing implementation, and some way needs to be found to associate base files with their metadata files. Coupled with this is a desire (expressed by some) to implement some form of JSR-170.

File Structure

The plan is to map resources to files in a file system and collections to directories. Simple enough until we try to associated properties and attachements.

Properties

Save properties in a file called file-name.properties, where file name corresponds to the base file name. Filter out all files that end in .properties from the list of resources contained in a collection.

Attachments

Save attachement references in a file called file-name.attachments. Filter out all files that end in .attachments from the list of resoources in a collection. Note that attachments will appear as resources in the collection. This may or may not be a good thing. If undesired, resources can be moved to a shadow context with collection structures replicated from the resources. This isn't a very good solution either, since the whole point of this exercise is to make it easier to retrieve files from back up. Moving them to a different point in the directory structure doesn't make this any easier for an administrator. An alternative approach is to put attachments into an archive (zip, for example).

Identifiers

Content resources are identified by a file name. In most cases, this name will have a file extention separated by a period, from which the MIME type may be deduced (via an extention map). All files also have a metadata file associated with it named the same, with a file extension of .properties. These properties may optionally be cached in a database to allow faster access to them.

Resources may have attachemnts - how will these be identified? How will they be kept together with their resource?

All resources (and attachments) live in a collection. Resource identifiers must be unique within the collection they live in. Collections may contain sub-collections. Collections are simply named and correspond to directories in a file system. Collection names must be unique within the collection they live in.

Collection paths are defined against a local root. This root is associated with a context (usually a site). Local roots branch from a Sakai global root, which in turn is mapped to a file system location. Context specific roots cause problems, however. Unless the context is included in a resource or collection id, it is no longer globally unique. This might not be a problem as long as methods that resolve against the id are done in the current context.

Regardless of how resources and collections are identified, let's declare that directories immediately below the root are associated with a context, such as a site. This allows for hidden context use as well.

Things To Do

Item	Status	Notes
Move the work to contrib
Resource metadata
Collection metadata
Attachments		Need to figure out how to include these.
Attachment metadata
File types		Need a mapping function.
Drop Box		Has this been changed for 2.2?

Project Status

Based implementation of resources and collections have begun. Interfaces have been copied locally to the ContentTool to simplify things for the interrim. As the implementations approach the full API specifications, the real APIs will be referenced. This is especially true of the ContentHostingService, which has many, many methods.

The APIs are sorta mid-way between Sakai 2.1 and 2.2. I need to create a sandbox in the 2.2 world and start referring to the new package names for ContentHosting. I'm holding off until I do the initial check in to contrib before making these changes.

Note: Sakai 2.2 defines a ContentEntity.

ContentCollection Methods

Method Name	Status	Notes
getUrl();
getReference();
getUrl(root-prop);		2.2 method
getReference(root-prop);		2.2 method
getId();
getProperties();
toXml();

getMembers();
getMemberResources();
getBodySizeK();
getReleaseDate();		2.2 method
getRetractDate();		2.2 method

ContentCollectionEdit Methods

Method Name	Status	Notes
isActiveEdit();
getPropertiesEdit();

setReleaseDate();		2.2 method
setRetractDate();		2.2 method

ContentResource Methods

Method Name	Status	Notes
getUrl();
getReference();
getUrl(root-prop);		2.2 method
getReference(root-prop);		2.2 method
getId();
getProperties();
toXml();

getContentLength();
String getContentType();
getContent();
getReleaseDate();		2.2 method
getRetractDate();		2.2 method
streamContent();

ContentResourceEdit Methods

Method Name	Status	Notes
isActiveEdit();
getPropertiesEdit();

setContentLength();
setContentType();
setContent();
setReleaseDate();		2.2 method
setRetractDate();		2.2 method

ContentHostingService Methods

Method Name	Status	Notes
Entity methods
getLabel();
willArchiveMerge();
willImport();
archive();
merge();
importEntities();
parseEntityReference();
getEntityDescription();
getEntityResourceProperties();
getEntity();
getEntityUrl();
getEntityAuthzGroups();
Collection methods
allowAddCollection();
addCollection(id + props);
addCollection(id);
allowGetCollection();
checkCollection();
getCollection();
getCollectionSize();
getAllResources();
allowUpdateCollection();
editCollection();
allowRemoveCollection();
removeCollection(id);
removeCollection(obj);
commitCollection();
cancelCollection();
getContainingCollectionId();
getDepth();
isRootCollection();
getCollectionMap();
eliminateDuplicates();
Resource methods
allowAddResource(String id);
addResource(id + stuff);
addResource(id + collection + stuff);
addResource(id);
allowUpdateResource();
updateResource();
editResource();
allowGetResource();
checkResource();
getResource();
allowRemoveResource();
removeResource(id);
removeResource(obj);
allowRename();
rename();
allowCopy();
copy();
copyIntoFolder();
moveIntoFolder();
commitResource(id);
commitResource(obj);
cancelResource();
findResources();
Attachment methods
allowAddAttachmentResource();
isAttachmentResource();
addAttachmentResource(name + stuff);
addAttachmentResource(name + tool + stuff);
addAttachmentResource(name);
Property methods
allowGetProperties();
getProperties();
allowAddProperty();
addProperty();
allowRemoveProperty();
removeProperty();
newResourceProperties();
Locking methods
getLocks();
lockObject();
removeLock();
isLocked();
containsLockedNode();
removeAllLcks();
Dropbox methods
createDropboxCollection();
createDropboxCollection();
getDropboxCollection();
getDropboxCollection();
isDropboxMaintainer();
isDropboxMaintainer();
getDropboxDisplayName();
getDropboxDisplayName();
Misc. methods
getUuid();
resolveUuid();
getUrl();
getReference();
getSiteCollection();
archiveResources();
isPubView();
isInheritingPubView();
setPubView();

You can track progress via my CH Project Notes.

Joining

Send me email if you are interested in participating in this project.

Seth Theriot (sp?) of Columbia indicated interest in participating.