Filing V3 Design

Owned by Former user (Deleted)
Last updated: Mar 09, 2009Version comment
2 min read

The OKI V3 OSIDs present a number of design challenges in part due to the introduction of pervasive object design patterns developed by Tom Coppeto. These patterns introduce some significant changes and improvements over the V2 OSIDs. This document is a place to record design decisions and ideas, primarily with respect to the Filing OSID implementation.

Basic Design

Handling Archive Files

Metadata Considerations

The concept of properties has evolved into a more general pattern with the V3 OSIDs. They are now fully mutable whereas the V2 properties a immutable, something that was felt to be a serious shortcoming of the previous OSID design. Properties are added by extending OsidRecord and OsidForm objects based on a previously defined Type. Unfortunate, there is no abstract mechanism to access properties as strings or as atomic data types (string, boolean, int, float, etc.).

Two approaches to support for metadata present themselves: file-based and data-based. To some extent, the file based approach is more "pure" since it doesn't require any additional software for it's support. Furthermore, a database approach introduces the problem of file to metadata synchronization. Changes to files MUST be kept up to date with respect to metadata. If one is is changed or deleted and not the other, corruption of the system results. Sakai has this problem and goes to great lengths to avoid it. The synchronization problem doesn't completely go away with a file based solution, since a metadata file needs to be associated with the primary file. If the two become separated or out of synch, the system is also corrupted. However, a file based solution is easier to manage and greatly simplifies backup/retrieval schemes - a problem that Sakai has had from the start. One final consideration is file search. Search generally uses a database of indexes and keywords. Such an approach greatly enhances search response. If database is going to be used for search, it might also be used for metadata support.

Ultimately, two implementations may be needed for metadata - both a file and database solution.

As with metadata, two approaches to file searching are possible: file and database. The database solution is well understood and there are off-the-shelf solutions that could be leveraged, such as Lucene (used by Sakai Search). The file based solution is iterative, nested file tree searching. Depending on the directory view used (ISOLATED, FEDERATED, PLENARY, OR COMPARATIVE), this approach might be fast or prohibitively slow.

Again, dual solutions is probably optional and would allow users to choose the best trade-off for their own uses.