« A "layered" model for interoperability using Dublin Core metadata | Main | Is DCMI hiding its light under a bushel? »

May 29, 2008

FRBR & "Time-Based Media", Part 3: Stills

In my previous post on using the FRBR model for "Time-Based Media", I outlined an example based on a video tutorial and a clip of that tutorial, the former made available in multiple formats, and both of them "versioned" over time.

Another similar, but slightly different, scenario which we want to represent is the case in which one or more still images is created from the content of a video. Suppose I create a sequence of images to use in some sort of summary page describing my video tutorial (or indeed to be used quite independently of the video itself), and I make these available as both a JPEG and a PNG format. The extraction of a still image clearly involves the creation of a new FRBR Work, so each of my stills is a distinct Work (W11, W12, etc), realized in a single Expression (E11, E12, etc), each embodied in two Manifestations (M11, M12, M13, M14, etc), each exemplified in a single Item.

Again, I'm tempted to use a whole-part relationship to express relationships between these new Works and Expressions and my original "complete video" Work (and Expression). It does seem slightly odd to use the same relationship type to express both the relationship between a (moving image) clip and a video and the relationship between a still image and a video, but perhaps from the perspective of film and video as made up of a sequence of discrete events/frames, it can be justified. So in Figure 1 below, I end up with a similar set of relationships to those illustrated in Figure 2 of my previous post. Again, I'll leave out the representation of the Items for conciseness:

Figure 1

And using Ian Davis' and Richard Newman's FRBR RDFS vocabulary again, a Turtle representation would look like:

@prefix frbr: <http://purl.org/vocab/frbr/core#> .
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix ex: <http://example.org/examples/> .

  a frbr:Work ;
  dcterms:title "Second Life Tutorial" ;
  frbr:part ex:W11 ;
  frbr:part ex:W12 ;
  frbr:realization ex:E01 .

  a frbr:Work ;
  dcterms:title "Second Life Tutorial Still 1" ;
  frbr:partOf ex:W01 ;
  frbr:realization ex:E11 .

  a frbr:Work ;
  dcterms:title "Second Life Tutorial Still 2" ;
  frbr:partOf ex:W01 ;
  frbr:realization ex:E12 .

  a frbr:Expression ;
  dcterms:title "Second Life Tutorial, Version 1.0" ;
  frbr:realizationOf ex:W01 ;
  frbr:part ex:E11 ;
  frbr:part ex:E12 ;
  frbr:embodiment ex:M01 ;
  frbr:embodiment ex:M02 ;
  frbr:embodiment ex:M03 .

  a frbr:Expression ;
  dcterms:title "Second Life Tutorial Still 1" ;
  frbr:realizationOf ex:W11 ;
  frbr:partOf ex:E01 ;
  frbr:embodiment ex:M11 ;
  frbr:embodiment ex:M12 .

  a frbr:Expression ;
  dcterms:title "Second Life Tutorial Still 2" ;
  frbr:realizationOf ex:W12 ;
  frbr:partOf ex:E01 ;
  frbr:embodiment ex:M13 ;
  frbr:embodiment ex:M14 .

The first issue raised by the stills example (and also by the previous example of clips/segments, although I didn't address it in my earlier post) is that of how to express the temporal "location" of the part (clip or still) within the whole i.e. to be able to say that my still is taken from a point T1 within the video, or that my clip is taken from the point within the whole video starting at a point in time T1 and continuing until time T2 (or starting at time T1 and having some specified duration). Given that  duration is considered to be an attribute of the Expression, I'd expect this to be represented at the level of the Expression. As far as I can tell, FRBR itself doesn't provide attributes for capturing this level of detail. I can imagine three ways of addressing this:

  • adding an attribute at the Expression level which provided some sort of human-readable note providing the information. This would serve the purpose of presenting the information to a human reader, but it wouldn't be sufficient to support e.g. an application presenting my stills along a timeline or supporting searches for stills or clips extracted from within a specified period of the video
  • adding start-point-within-whole and end-point-within-whole attributes to the "part" Expression. This would enable such processing as suggested above, but would be sufficient only if the  part participated in at most one part-whole relationship
  • modelling the part-whole relationship as a resource in its own right with start point and end point attributes

And the part-whole relationship may have a spatial aspect as well as a temporal aspect e.g. the case where a still is only some of some specific spatial region of the whole screen image. Potentially, there's a great deal of complexity here - as is reflected in the capabilities within MPEG-7 to represent quite complex segmentation/decomposition relationships - but I suspect for the purposes of this exercise we'll be aiming to try to satisfy some of the simple cases required to support the discovery, selection and navigation of resources, probably using some variant of the second bullet above, while acknowledging that there is some complexity which isn't modelled within the DCAP.

Secondly,  I probably want to capture the fact that my set of still images form a set or sequence, distinct from my video clip(s) which also form parts of the same whole. The best fit I can see for this in FRBR would seem to be to express hasSuccessor/isSuccessorTo relationships between the stills in the sequence (ex:E11 frbr:successor ex:E12. ex:E12 frbr:successorOf ex:E11 . And so on.) Whether it's also useful/necessary to represent the sequence as a distinct work, I'm less sure. Probably not.

The final issue raised here I wanted to note is the assumption that both the video and the still images are described using the FRBR model, where whole-part relationships exist between instances of the same Group 1 Entity Type i.e. when a Work is the subject of a has-Part or is-Part-Of relationship, the assumption is that the object of that relationship is also a Work (and the same for Expressions, Manifestations and Items). As I mentioned in a comment on my earlier post, one of the "sibling" projects to the Time-Based Media project is developing at a DCAP for describing still images, and that project has recommended the use of a model which is derived from the FRBR model but which substitutes a single entity type of Image for the Work-Expression pair used in FRBR. Mick Eadie describes the project's reasons for that choice in a recent Ariadne article:

In essence what is being done by FRBR is not the modelling of the simple image and its relationships, but rather an attempt to model the artistic / intellectual process and all resultant manifestations of it. We decided this was inappropriate for the IAP for a number of reasons. While possible, an application profile of this complexity would require detailed explanation that could be a barrier to take-up. Moreover, it strays from the core remit of the images IAP to facilitate a simple exchange of image data between repositories. While the FRBR approach attempts to build relationships between objects, e.g. slides, photographs, objects and digital surrogates, this facility already exists in, for example, the Visual Resources Association Core (VRA) schema. Our intention was not to reinvent or in any way replicate existing standards that are robust and heavyweight enough to deal with most image types. Rather our intention was to build a lightweight layer that could sit above these standards, and work with them, facilitating a simple image search across institutional repositories.

Using the IAP model to describe the still image, there are no distinct Works and Expressions, only Images, so it seems to me that integrating that data within a strictly FRBR-based view would require some mapping between the two models, and the separating out of attributes of the IAP Image entity which in the FRBR model apply to the Work from those which apply to the Expression.


TrackBack URL for this entry:

Listed below are links to weblogs that reference FRBR & "Time-Based Media", Part 3: Stills:


Makes my head spin.

Surely once you move to a 'summary page with still images' you are no longer dealing with time-based media, so these fall outside what you are trying to achieve here?

In terms of the work-to-work relationships, I wonder if there is a need for something else here. 'Is part of' doesn't seem quite right - unlike the idea of a segment of the tutorial which stands alone as an intellectual work - I can't see a still from a film fulfilling either the 'dependent' or 'independent' type of the 'is part of' relationship. That said, I can't see which other relationship might apply. On the otherhand, perhaps the point of timebased media is that by their nature any defined part of their running time (down to a single moment represented in this case by a still) can be regarded as 'discrete components of a work whose content exists as a distinct identifiable segment within a whole'

The more I think about it, the more it seems that there are realistic parallels with more traditional text media. In an edited book, 'Chapter 1 by xxx' would clearly fall into a 'ispartof' relationship. However, 'pages 24-27' doesn't - it only becomes a 'work' by definition of an external party - there is no author intent for this to be regarded as a standalone intellectual work. I'm not sure this helps :(

I have to admit that in reality the idea of cataloguing down to the level of the still image seems quite unlikely, but I suppose this doesn't mean that it wouldn't happen.

Thanks, Owen.

It makes my head spin a bit too, and I've been churning these scenarios around for days wondering what on earth to do with them that doesn't involve a good degree of complexity. I take your point about the level of detail, but OTOH the stills case is based on a real scenario. I guess one alternative would be to decide that the "resource of interest" was some "composite" of video and stills, and stick at describing that as a unit.

As for the "scope" question, and where the "boundaries" lie, this is another question I've been losing sleep over. :-) Increasingly, I worry it's a bit problematic to draw lines around the "worlds" of "Time-Based Media", "ePrints", (Still) "Images", and so on, because so often we need to express relationships between things from these different "worlds"; from the perspective of how the resources are used, the boundaries we create become rather artificial. Many of the scenarios Gayle has sketched around "Time-Based Media" seem to involve links/references to Other Sorts of Stuff.

And just from a personal perspective, when I draw up a list of the "outputs" I've created over the last month, - in some cases at least - I want that list to include textual documents, presentations, audio stuff, videos, diagrams. And, yes, in other cases, I do want to filter it so that I only list the papers.

OTOH, trying to develop a model (or apply an existing model) which satisfactorily deals all that range of resource types risks becoming a somewhat overwhelming endeavour. I think models such as the CIDOC Conceptual Reference Model may have things to offer in this area, but it is a complex model, and I'm not sure that level of detail is required here.

I don't have a good answer right now. At the moment, I feel I'm just trying to highlight some of the issues.

The comments to this entry are closed.



eFoundations is powered by TypePad