Content Hosting Project

Owned by Former user (Deleted)
Last updated: Jan 08, 2009 by Former user (Deleted)Version comment
4 min read

With the implementation of JSR-170 in the Sakai Content Hosting Service, the need for yet another implementation is unneeded.

Content Hosting (CH) is currently implemented via a two-tier set of objects: one to store content and properties in a datbase, the other to store content in files. If content is saved in files and properties are kept in the database, this presents a serious problem with resource integrity. For example, if a file is deleted in Sakai, and the user asks their IT people to restore it, the metadata is lost, potentially corrupting the Content Hosting system. Also, the implementation uses an internally defined method of naming files and directories that makes restoring files nearly impossible.

Given a desire to support restoring lost files and external file system management (via AFS, etc.), I am exploring a new implementation of Content Hosting based solely on a standard file system. Naturally, there are trade-offse with this approach. It may not be as fast as the existing implementation, and some way needs to be found to associate base files with their metadata files. Coupled with this is a desire (expressed by some) to implement some form of JSR-170.

See also, CH Project Notes

File Structure

The plan is to map resources to files in a file system and collections to directories. Simple enough until we try to associated properties and attachements.

Properties

Save properties in a file called file-name.properties, where file name corresponds to the base file name. Filter out all files that end in .properties from the list of resources contained in a collection.

It's not clear that collection properties are used in Sakai at this time, but clearly they could be useful. Since there is only one collection properties file in a given directory, it can be given a unique name, "collection.properties". As long as this file is filtered from the list of files in a directory, all will be fine.

A base implementation of ResourceProperties exists in the entity model. An FSProperties will be created to extend this implementation primarily to handle storage into and out of a .properties file.

Attachments

Sakai content hosting supports the concept of resources having attachments. These attachments (like MIME attachments) are separate objects (files) that are appended (or attached) to the main resource. Three approaches to file-based attachments are being considered:

Locally Referenced Files

Attachment objects exist as local resources in the same collection. Attaching is accomplished either via a file-name.attachments file, or via multi-valued property in .properties (favored). While this is a simple approach, it does mean that attachment objects will be visible in tools that list files in a collection. Does the resource tool show attachments as files?

Files in Collections

Attachment objects could be contained in a special collection as a child of the current collection. This is a bit ugly, since these sub-collections either needed to be treated specially or filtered out. However, this roughly corresponds to the way JSR-170 handles attached objects.

Files in an Archive

Attachment objects could be contained in an archive format (like zip). Some overhead is added getting them in and out of the archive format.

Identifiers

Content resources are identified by a file name. In most cases, this name will have a file extention separated by a period, from which the MIME type may be deduced (via an extention map). All files also have a metadata file associated with it named the same, with a file extension of .properties. These properties may optionally be cached in a database to allow faster access to them.

Resources may have attachemnts - how will these be identified? How will they be kept together with their resource?

All resources (and attachments) live in a collection. Resource identifiers must be unique within the collection they live in. Collections may contain sub-collections. Collections are simply named and correspond to directories in a file system. Collection names must be unique within the collection they live in.

Collection paths are defined against a local root. This root is associated with a context (usually a site). Local roots branch from a Sakai global root, which in turn is mapped to a file system location. Context specific roots cause problems, however. Unless the context is included in a resource or collection id, it is no longer globally unique. This might not be a problem as long as methods that resolve against the id are done in the current context.

Regardless of how resources and collections are identified, let's declare that directories immediately below the root are associated with a context, such as a site. This allows for hidden context use as well.

Things To Do

Item

Status

Notes

Move the work to contrib

 

 

Resource metadata

(error)

 

Collection metadata

(error)

 

Attachments

(error)

Need to figure out how to include these.

Attachment metadata

(error)

 

File types

(error)

Need a mapping function.

Drop Box

(error)

Has this been changed for 2.2?

Project Status

Based implementation of resources and collections have begun. Interfaces have been copied locally to the ContentTool to simplify things for the interrim. As the implementations approach the full API specifications, the real APIs will be referenced. This is especially true of the ContentHostingService, which has many, many methods.

The APIs are sorta mid-way between Sakai 2.1 and 2.2. I need to create a sandbox in the 2.2 world and start referring to the new package names for ContentHosting. I'm holding off until I do the initial check in to contrib before making these changes.

Note: Sakai 2.2 defines a ContentEntity.

ContentCollection Methods

Method Name

Status

Notes

getUrl();

(tick)

 

getReference();

(tick)

 

getUrl(root-prop);

 

2.2 method

getReference(root-prop);

 

2.2 method

getId();

(tick)

 

getProperties();

 

 

toXml();

 

 

 

 

 

getMembers();

 

 

getMemberResources();

 

 

getBodySizeK();

 

 

getReleaseDate();

 

2.2 method

getRetractDate();

 

2.2 method

ContentCollectionEdit Methods

Method Name

Status

Notes

isActiveEdit();

 

 

getPropertiesEdit();

 

 

 

 

 

setReleaseDate();

 

2.2 method

setRetractDate();

 

2.2 method

ContentResource Methods

Method Name

Status

Notes

getUrl();

(tick)

 

getReference();

(tick)

 

getUrl(root-prop);

 

2.2 method

getReference(root-prop);

 

2.2 method

getId();

(tick)

 

getProperties();

 

 

toXml();

 

 

 

 

 

getContentLength();

(tick)

 

String getContentType();

 

 

getContent();

(tick)

 

getReleaseDate();

 

2.2 method

getRetractDate();

 

2.2 method

streamContent();

(tick)

 

ContentResourceEdit Methods

Method Name

Status

Notes

isActiveEdit();

 

 

getPropertiesEdit();

 

 

 

 

 

setContentLength();

 

 

setContentType();

 

 

setContent();

 

 

setReleaseDate();

 

2.2 method

setRetractDate();

 

2.2 method

ContentHostingService Methods

Method Name

Status

Notes

Entity methods

 

 

getLabel();

 

 

willArchiveMerge();

 

 

willImport();

 

 

archive();

 

 

merge();

 

 

importEntities();

 

 

parseEntityReference();

 

 

getEntityDescription();

 

 

getEntityResourceProperties();

 

 

getEntity();

 

 

getEntityUrl();

 

 

getEntityAuthzGroups();

 

 

Collection methods

 

 

allowAddCollection();

 

 

addCollection(id + props);

 

 

addCollection(id);

 

 

allowGetCollection();

 

 

checkCollection();

 

 

getCollection();

(tick)

 

getCollectionSize();

 

 

getAllResources();

 

 

allowUpdateCollection();

 

 

editCollection();

 

 

allowRemoveCollection();

 

 

removeCollection(id);

 

 

removeCollection(obj);

 

 

commitCollection();

 

 

cancelCollection();

 

 

getContainingCollectionId();

 

 

getDepth();

 

 

isRootCollection();

 

 

getCollectionMap();

 

 

eliminateDuplicates();

 

 

Resource methods

 

 

allowAddResource(String id);

 

 

addResource(id + stuff);

 

 

addResource(id + collection + stuff);

 

 

addResource(id);

 

 

allowUpdateResource();

 

 

updateResource();

 

 

editResource();

 

 

allowGetResource();

 

 

checkResource();

 

 

getResource();

(tick)

 

allowRemoveResource();

 

 

removeResource(id);

 

 

removeResource(obj);

 

 

allowRename();

 

 

rename();

 

 

allowCopy();

 

 

copy();

 

 

copyIntoFolder();

 

 

moveIntoFolder();

 

 

commitResource(id);

 

 

commitResource(obj);

 

 

cancelResource();

 

 

findResources();

 

 

Attachment methods

 

 

allowAddAttachmentResource();

 

 

isAttachmentResource();

 

 

addAttachmentResource(name + stuff);

 

 

addAttachmentResource(name + tool + stuff);

 

 

addAttachmentResource(name);

 

 

Property methods

 

 

allowGetProperties();

 

 

getProperties();

 

 

allowAddProperty();

 

 

addProperty();

 

 

allowRemoveProperty();

 

 

removeProperty();

 

 

newResourceProperties();

 

 

Locking methods

 

 

getLocks();

 

 

lockObject();

 

 

removeLock();

 

 

isLocked();

 

 

containsLockedNode();

 

 

removeAllLcks();

 

 

Dropbox methods

 

 

createDropboxCollection();

 

 

createDropboxCollection();

 

 

getDropboxCollection();

 

 

getDropboxCollection();

 

 

isDropboxMaintainer();

 

 

isDropboxMaintainer();

 

 

getDropboxDisplayName();

 

 

getDropboxDisplayName();

 

 

Misc. methods

 

 

getUuid();

 

 

resolveUuid();

 

 

getUrl();

 

 

getReference();

 

 

getSiteCollection();

 

 

archiveResources();

 

 

isPubView();

 

 

isInheritingPubView();

 

 

setPubView();

 

 

You can track progress via my CH Project Notes.

Joining

Send me email if you are interested in participating in this project.

Seth Theriot (sp?) of Columbia indicated interest in participating.