Pointers and Links

In section we introduced the simplest pointer elements, ptr and ref. Here we introduce additionally the link element, which represents an association between two (or more) locations by specifying each location explicitly. Its own location is irrelevant to the intended linkage. All three elements use the attribute target, provided by the att.pointing class as a means of indicating the location or locations referenced or pointed to. The ptr element may be called a pure pointer, because its primary function is simply to point. A pointer sets up a connection between an element (which, in the case of a pure pointer, is simply a location in a document), and one or more others, known collectively as its target. The ptr and ref elements point, conceptually, at a single target, even if that target may be discontinuous in the document. The link element specifies at least two targets and represents an association between them, independent of its own location.

These three elements also share a common set of attributes, derived from the att.pointing and att.typed classes:

Double connection among elements could also be expressed by a combination of pointer elements, for example, two ptr elements, or one ptr element and one note element. All that is required is that the value of the target (or other pointing) attribute of the one be the value of the xml:id attribute of the other. What the link element accomplishes is the handling of double connection by means of a single element. Thus, in the following encoding: sa-p1 points to sa-p2, and sa-p2 points to sa-p1. This is logically equivalent to the more compact encoding:

As noted elsewhere, the target attribute may take as value one or more URI reference. In the simplest case, each such reference will indicate an element in the current document (or in some other document), for example by supplying the value used for its global xml:id attribute. It may however carry as value any form of URI, such as a URL pointing to some other document or location on the Internet. Pointing or linking to external documents and pointing and linking where identifiers are not available is described below in section .

Using Pointers and Links

As an example of the use of mechanisms which establish connections among elements, consider the practice (common in 18th century English verse and elsewhere) of providing footnotes citing parallel passages from classical authors.

The figure shows the original page of Pope's Dunciad which is discussed in the text.

Such footnotes can of course simply be encoded using the note element (see section ) without a target attribute, placed adjacent to the passage to which the note refers:The type attribute on the note is used to classify the notes using the typology established in the Advertisement to the work: The Imitations of the Ancients are added, to gratify those who either never read, or may have forgotten them; together with some of the Parodies, and Allusions to the most excellent of the Moderns. In the source text, the text of the poem shares the page with two sets of notes, one headed Remarks and the other Imitations. (Diff'rent our parties, but with equal grace The Goddess smiles on Whig and Tory race, Virg. Æn. 10. Tros Rutulusve fuat; nullo discrimine habebo. —— Rex Jupiter omnibus idem. 'Tis the same rope at sev'ral ends they twist, To Dulness, Ridpath is as dear as Mist)

This use of the note element can be called implicit pointing (or implicit linking). It relies on the juxtaposition of the note to the text being commented on for the connection to be understood. If it is felt that the mere juxtaposition of the note to the text does not make it sufficiently clear exactly what text segment is being commented on (for example, is it the immediately preceding line, or the immediately preceding two lines, or what?), or if it is decided to place the note at some distance from the text, then the pointing or the linking must be made explicit. We now consider various methods for doing that.

Firstly, a ptr element might be placed at an appropriate point within the text to link it with the annotation: (Diff'rent our parties, but with equal grace The Goddess smiles on Whig and Tory race, 'Tis the same rope at sev'ral ends they twist, To Dulness, Ridpath is as dear as Mist) Virg. Æn. 10. Tros Rutulusve fuat; nullo discrimine habebo. —— Rex Jupiter omnibus idem. The note element has been given an arbitrary identifier (note3.284) to enable it to be specified as the target of the pointer element. Because there is nothing in the text to signal the existence of the annotation, the rend attribute has been given the value unmarked.

Secondly, the target attribute of the note element can be used to point at its associated text, provided that an xml:id attribute has been supplied for the associated text: (Diff'rent our parties, but with equal grace The Goddess smiles on Whig and Tory race, 'Tis the same rope at sev'ral ends they twist, To Dulness, Ridpath is as dear as Mist) Given this encoding of the text itself, we can now link the various notes to it. In this case, the note itself contains a pointer to the place in the text which it is annotating; this could be encoded using a ref element, which bears a target attribute of its own and contains a (slightly misquoted) extract from the text marked as a quote element: Verse 283–84. ——. With equal grace Our Goddess smiles on Whig and Tory race. Virg. Æn. 10. Tros Rutulusve fuat; nullo discrimine habebo. —— Rex Jupiter omnibus idem.

Combining these two approaches gives us the following associations: a pointer within one line indicates the note the note indicates the line a pointer within the note indicates the line Note that we do not have any way of pointing from the line itself to the note: the association is implied by containment of the pointer. We do not as yet have a true double link between text and note. To achieve that we will need to supply identifiers for the annotations as well as for the verse lines, and use a link element to associate the two. Note that the ptr element and the target attribute on the note may now be dispensed with: Verse 283–84. ——. With equal grace Our Goddess smiles on Whig and Tory race. Virg. Æn. 10. Tros Rutulusve fuat; nullo discrimine habebo. —— Rex Jupiter omnibus idem.

The target attribute of the link element here bears the identifier of the note followed by that of the verse line. We could also allocate an identifier to the reference within the note and encode the association between it and the verse line in the same way: Verse 283–84. ——. With equal grace Our Goddess smiles on Whig and Tory race. Indeed, the two links could be combined into one, as follows:

Groups of Links

Clearly, there are many reasons for which an encoder might wish to represent a link or association between different elements. For some of them, specific elements are provided in these Guidelines; some of these are discussed elsewhere in the present chapter. The link element is a general purpose element which may be used for any kind of association. The element linkGrp may be used to group links of a particular type together in a single part of the document; such a collection may be used to represent what is sometimes referred to in the literature of Hypertext as a web, a term introduced by the Brown University FRESS project in 1969, and not to be confused with the World Wide Web. As a member of the class att.pointing.group, this element shares the following attributes with other members of that class: It is also a member of the att.pointing and att.typed classes, and therefore also carries the attributes specified in section above, in particular the type attribute.

The linkGrp element provides a convenient way of establishing a default for the type attribute on a group of links of the same type: by default, the type attribute on a link element has the same value as that given for type on the enclosing linkGrp.

Typical software might hide a web entirely from the user, but use it as a source of information about links, which are displayed independently at their referenced locations. Alternatively, software might provide a direct view of the link collection, along with added functions for manipulating the collection, as by filtering, sorting, and so on. To continue our previous example, this text contains many other notes of a kind similar to the one shown above. Here are a few more of the lines to which annotations have to be attached, followed by the annotations themselves: A place there is, betwixt earth, air and seas Where from Ambrosia, Jove retires for ease. Sign'd with that Ichor which from Gods distills. Ovid Met. 12. Orbe locus media est, inter terrasq; fretumq; Cœlestesq; plagas — Alludes to Homer, Iliad 5 ... To avoid having to repeat the specification of type as imitation on each note, we may specify it once for all on a linkGrp element containing all links of this type.

Additional information for applications that use linkGrp elements can be provided by means of special attributes. First, the domains attribute can be used to identify the text elements within which the individual targets of the links are to be found. Suppose that the text under discussion is organized into a body element, containing the text of the poem, and a back element containing the notes. Then the domains attribute can have as its value the identifiers of the body and the back, to enable an application to verify that the link targets are in fact contained by appropriate elements, or to limit its search space:

Note that there must be a single parent element for each domain; if some notes are contained by a section with identifier dunnotes, and others by a section with identifier dunimits, an intermediate pointer must be provided (as described in section ) within the linkGrp and its identifier used instead.

Next, the targFunc attribute can be used to provide further information about the role or function of the various targets specified for each link in the group. The value of the targFunc attribute is a list of names (formally, name tokens), one for each of the targets in the link; these names can be chosen freely by the encoder, but their significance should be documented in the encoding description in the header.Since no special element is provided for this purpose in the present version of these Guidelines, the information should be supplied as a series of paragraphs at the end of the encodingDesc element described in section . In the current example, we might think of the note as containing the source of the imitation and the verse line as containing the goal of the imitation. Accordingly, we can specify the linkGrp in the preceding example thus:

Intermediate Pointers

In the preceding examples, we have shown various ways of linking an annotation and a single verse line. However, the example cited in fact requires us to encode an association between the note and a pair of verse lines (lines 284 and 285); we call these two lines a span.

There are a number of possible ways of correcting this error: one could use the target attribute to indicate one end of the span and the special purpose targetEnd attribute on the note element to point to the other. Another possibility might be to create an element which represents the whole span itself and assign that an xml:id attribute, which can then be linked to the note and ref elements. This could be done using for example the lg element defined in section or the virtual join element discussed in section .

A third possibility would be to use an intermediate pointer as follows: When the target attribute of a ptr or ref element specifies more than one element, the indicated elements are intended to be combined or aggregated in some way to produce the object of the pointer. (Such aggregation is however the task of a processing application, and cannot be defined simply by the markup). The xml:id attribute of the ptr then provides an identifier which can be linked to the note and ref elements:

The all value of evaluate is used on the link element to specify that any pointer encountered as a target of that element is itself evaluated. If evaluate had the value none, the link target would be the pointer itself, rather than the objects it points to.

Where a linkGrp element is used to group a collection of link elements, any intermediate pointer elements used by those link elements should be included within the linkGrp.

Pointing Elsewhere

Like the ubiquitous if misnamed XHTML pointing attribute href, the TEI pointing attributes can point to a document that is not the current document (the one that contains the pointing element) whether it is in the same local filesystem as the current document, or on a different system entirely. In either case, the pointing can be accomplished absolutely (using the entire address of the target document) or relatively (using an address relative to the current base URI in force). The current base URI is defined according to Marsh 2001. If there is none, the base URI is that of the current document. In common practice the current base URI in force is likely to be the value of the xml:base attribute of the closest ancestor that has one. However this may not be the case, since xml:base attributes are accumulated through the hierarchy by concatenation of path segments, beginning at the top of the hierarchy and proceeding down to the context node.

The following example demonstrates an absolute URI reference that points to a remote document: The current base URI in force is as defined in the W3C XML Base recommendation.

This example points explicitly to a location on the Web, accessible via HTTP. Suppose however that we wish to access a document stored locally in a file. Again we will supply an absolute URI reference, but this time using a different protocol: This Debian package is distributed under the terms of the GNU General Public License.

In the following example, we use a relative URI reference to point to a local document:

The figure shows the page from the of Comenius which is discussed in the text.

Since no xml:base is specified here, the location of the resource Figures/compic.png is determined relative to the resource indicated by the current base URI, which is the current document.

In the following example, however, we first change the current base URI by setting a new value for xml:base. The resource required is then identified by means of a relative URI:

As noted above, the current base URI is found on the nearest ancestor. This provides a useful way of abbreviating URIs within a given scope:

Pointing Locally

Because the default base URI is the current document, a pointer that is specified as a bare name fragment identifierIn more recent W3C documents, the term bare name is deprecated in favour of the more explicit shorthand pointer.alone acts as a pointer to an element in the current document, as in the following example.

This method of pointing, by referring to the xml:id of the target element as a bare name only (e.g., #sect106) is the simplest and often the best approach where it can be applied, i.e. where both the source element and target element are in the same XML document, and where the target element carries an identifier. It is the method used extensively in previous sections of this chapter and elsewhere in these Guidelines.

Using Abbreviated Pointers

xml:base is a useful way of handling the repeated use of long external URIs. However, it is less convenient when your text contain many references to a variety of different sources in different locations. Even in the case of relative links on the local file system, ref or target attributes may become quite lengthy and make XML code difficult to read. To deal with this problem, the TEI provides a useful method of using abbreviated pointers and documenting a way to dereference them automatically.

Imagine a project which has a large collection of XML documents organized like this:

anthology poetry poem.xml prose novel.xml references people personography.xml

If you want to link a name in the novel.xml file to a person in the personography.xml file, the link will look like this: Fred If there are many names to tag in a single paragraph, the XML encoding will be congested, and such lengthy links are prone to typographical error. In addition, if the project organization is changed, every relative link will have to be found and altered.

One way to deal with this is to use what is often referred to as a "magic token". You could make such links using the key attribute: Fred and document the meaning of the key using (for instance) a taxonomy element in the TEI header, as described in . However, such a link cannot be mechanically processed by an external system that does not know how to interpret it; a human will have to read the header explanation and write code explicitly to reconstruct the intended link.

A more robust alternative is to use a private URI scheme. This is a method of constructing a simple, key-like token which functions as a data.pointer, and can therefore be used as the value of any attribute which has that datatype, such as ref and target. Such a scheme consists of a prefix with a colon, and then a value. You might, for example, use the prefix psn (for "person"), and structure your name tags like this: Fred How is this different from a magic token? Essentially, it isn't, except that TEI provides a structured method of dereferencing it (turning it into a computable path, such as ../../references/people/personography.xml#fred) by means of a declaration inside encodingDesc in the TEI header, using the elements and attributes for prefix declaration:

This is how you might document a private URI scheme using the psn: prefix:

In the context of this project, private URIs with the prefix "psn" point to person elements in the project's personography.xml file.

This specifies that where a data.pointer value is constructed with a psn: prefix, a regular-expression replace operation can be performed on it to construct the full or relative URI to the target document or fragment. listPrefixDef is a child of encodingDesc, and it contains any number of prefixDef elements. Each prefixDef element provides a method of dereferencing or expanding an abbreviated pointer, based on a regular expression. The ident attribute specifies the prefix to which the expansion applies (without the colon). The matchPattern attribute contains a regular expression which is matched against the component of the pointer following the first colon, and the replacementPattern provides the string which will be used as a replacement. In this example, using psn:fred, the value fred would be matched by the matchPattern, and also captured (through the parentheses in the regular expression); it would then be replaced by the value ../../references/people/personography.xml#fred (with the the $1 in the replacementPattern being replaced by the captured value). The p element inside the prefixDef can be used to provide a human-readable explanation of the usage of this prefix.

Through this mechanism, any processor which encounters a data.pointer with a protocol unknown to it can check the listPrefixDef in the header to see if there is an available expansion for it, and if there is, it can automatically provide the expansion and generate a full or relative URI.

For any given prefix, it may be useful to supply more than one expansion. For instance, in addition to pointing at the person element in the personography file, it might also be useful to point to an external source which is available on the network, representing the same information in a different way. So there might be a second prefixDef like this:

Private URIs with the prefix "psn" can be converted to point to a fragment on the Personography page of the project Website.

Any number of prefixDef elements may be provided for the same prefix. A processor may decide to process one or all of them; if it processes only one, it should choose the first one with the correct ident value, so the primary or most important prefixDef for any given prefix should appear first in its parent listPrefixDef.

When creating private URI schemes, it is recommended that you avoid using any existing registered prefix. A list of registered prefixes is maintained by IANA at http://www.iana.org/assignments/uri-schemes.html.

Note that this mechanism can also be used to dereference other abbreviated pointing systems which are based on prefixes, such as Tag URIs.

The matchPattern and replacementPattern attributes are also used in dereferencing canonical reference patterns, and further examples of the use of regular expressions are shown in .

TEI XPointer Schemes

The pointing schemes described in this chapter are part of a number of such schemes envisaged by the W3C, which together constitute a framework for addressing data within XML documents, known as the XPointer Framework (Grosso et al 2003). This framework permits the definition of many other named addressing methods, each of which is known as an XPointer Scheme. The W3C has predefined a set of such schemes, and maintains a register for their expansion.

One important scheme, also defined by the W3C, and recommended by these Guidelines is the xpath() pointer scheme, which allows for any part of an XML structure to be selected using the syntax defined by the XPath specification. This is further discussed below, . These Guidelines also define six other pointer schemes, which provide access to parts of an XML document such as points within data content or stretches of data content. These additional TEI pointer schemes are defined in sections to below.

Canonical References

By canonical reference we mean any means of pointing into documents, specific to a community or corpus. For example, biblical scholars might understand Matt 5:7 to mean the book called , chapter 5, verse 7. They might then wish to translate the string Matt 5:7 into a pointer into a TEI-encoded document, selecting the element which corresponds to the seventh div element within the fifth div element within the div element with the n attribute valued Matt.

Several elements in the TEI scheme (gloss, ptr, ref, and term) bear a special attribute, cRef, just for this purpose. Using the system described in this section, an encoder may specify references to canonical works in a discipline-familiar format, and expect software to derive a complete URI from it. The value of the cRef attribute is processed as described in this section, and the resulting URI reference is treated as if it were the value of the target attribute. The cRef and target attributes are mutually exclusive: only one or the other may be specified on any given occurrence of an element.

For the cRef attribute to function as required, a mechanism is needed to define the mapping between (for example) the book called and the part of the XML structure which corresponds with it. This is provided by the refsDecl element in the TEI header, which contains an algorithm for translating a canonical reference string (like Matt 5:7) into a URI such as #xpath(//div[@n='Matt']/div[5]/div[7]). The refsDecl element is described in section ; the following example is discussed in more detail below in section .

This pointer pattern extracts and references the book, chapter, and verse parts of a biblical reference.

This pointer pattern extracts and references the book and chapter parts of a biblical reference.

This pointer pattern extracts and references just the book part of a biblical reference.

When an application encounters a canonical reference as the value of cRef attribute, it might follow this sequence of specific steps to transform it into a URI reference: Ascertain the correct refsDecl following the rules summarized in section . For each cRefPattern element encountered in the appropriate refsDecl, in the order encountered: match the value of the cRef attribute to the regular expression found as the value of the matchPattern attribute if the value of the cRef attribute matches: take the value of the replacementPattern attribute and substitute the back references ($1, $2, etc.) with the corresponding matched substrings the result is taken as if it were a relative or absolute URI reference specified on the target attribute; i.e., it should be used as is or combined with the current xml:base attribute value as usual no further processing of this value of the cRef attribute against the refsDecl should take place if, however, the value of the cRef attribute does not match the regular expression specified in the value of the matchPattern attribute, proceed to the next cRefPattern If all the cRefPattern elements are examined in turn and none matches, the pointer fails.

The regular expression language used as the value of the matchPattern attribute is that used for the pattern facet of the World Wide Web Consortium's XML Schema Language in an Appendix to XML Schema Part 2.As always seems to be the case, no two regular expression languages are precisely the same. For those used to Perl regular expressions, be warned that while in Perl the pattern tei matches any string that contains tei, in the W3C language it only matches the string tei. The value of the replacementPattern attribute is simply a string, except that occurrences of $1 through $9 are replaced by the corresponding substring match. Note that since a maximum of nine substring matches are permitted, the string $18 means the value of the first matched substring followed by the character 8 as opposed to the eighteenth matched substring. If there is a need for an actual string including a dollar sign followed by a digit that is not supposed to be replaced, the dollar sign should be written as $$. Implementations must convert $$ to $ during processing.

Correspondence

A common requirement in text analysis is to represent correspondences between two or more parts of a single document, or between places in different documents. Provided that explicit elements are available to represent the parts or places to be linked, then the global linking attribute corresp may be used to encode such correspondence, once it has been identified. This is one of the attributes made available by the mechanism described in the introduction to this chapter (). Correspondence can also be expressed by means of the link element introduced in section .

Where the correspondence is between spans, the seg element should be used, if no other element is available. Where the correspondence is between points, the anchor element should be used, if no other element is available.

The use of the corresp attribute with spans of content is illustrated by the following example: , which made its Friday night debut only a month ago, was not listed on NBC's new schedule, although the network says the show still is being considered. Here the anaphoric phrases the network and the show have been associated directly with the elements to which they refer by means of corresp attributes. This mechanism is simple to apply, but has the drawback that it is not possible to specify more exactly what kind of correspondence is intended. Where this attribute is used, therefore, encoders are encouraged to specify their intent in the associated encoding description in the TEI header.

Essentially, what the corresp attribute does is to specify that elements bearing this attribute and those to which the attribute points are doubly linked. In the example above, the use of the corresp attribute indicates that the seg element containing the show and the title element containing Shirley correspond to each other: the correspondence relationship is not from one to the other, but between the two objects. It is thus different from the target attribute, and provides functionality more similar to that of the link and linkGrp elements defined in section , although it lacks the ability to indicate more precisely what kind of correspondence is intended as in the following retagging of the preceding example. , which made its Friday night debut only a month ago, was not listed on NBC's new schedule, although the network says the show still is being considered.

In the following example, we use the same mechanism to express a correspondence amongst the anchors introduced following the fifth word after English in a text: English language. Except for not very English at all at the time English was still full of flaws English. This was revised by young

Alignment of Parallel Texts

One very important application area for the alignment of parallel texts is multilingual corpora. Consider, for example, the need to align translation pairs of sentences drawn from a corpus such as the Canadian Hansard, in which each sentence is given in both English and French. Concerning this problem, Gale and Church write: Most English sentences match exactly one French sentence, but it is possible for an English sentence to match two or more French sentences. The first two English sentences [in the example below] illustrate a particularly hard case where two English sentences align to two French sentences. No smaller alignments are possible because the clause ...sales...were higher... in the first English sentence corresponds to (part of) the second French sentence. The next two alignments ... illustrate the more typical case where one English sentence aligns with exactly one French sentence. The final alignment matches two English sentences to a single French sentence. These alignments [which were produced by a computer program] agreed with the results produced by a human judge.See , from which the example in the text is taken.

The alignment produced by Gale and Church's program can be expressed in four different ways. The encoder must first decide whether to represent the alignment in terms of points within each text (using the anchor element) or in terms of whole stretches of text, using the seg element. To some extent the choice will depend on the process by which the software works out where alignment occurs, and the intention of the encoder. Secondly, the encoder may elect to represent the actual encoding using either corresp attributes attached to the individual anchor or seg elements, or using a free-standing linkGrp element.

We present first a solution using anchor elements bearing only corresp attributes:

There is no requirement that the corresp attribute be specified in both English and French texts, since (as noted above) this attribute is defined as representing a mutual association. However, it may simplify processing to do so, and also avoids giving the impression that the English is translating the French, or vice versa. More seriously, this encoding does not make explicit that it is in fact the entire stretch of text between the anchors which is being aligned, not simply the points themselves. If for example one text contained material omitted from the other, this approach would not be appropriate.

We now present the same passage using the alternative linkGrp mechanism and marking explicitly the segments which have been aligned:

Note that use of the ab element allows us to mark up the orthographic sentences in both languages independently of the alignment: the first translation pair in this example might be marked up as follows:

A Three-way Alignment

The preceding encoding of the alignment of parallel passages from two texts requires that those texts and the alignment all be part of the same document. If the texts are in separate documents, then complete URIs, whether absolute or relative (section ), will be required. These external pointers may appear anywhere within the document, but if they are created solely for use in encoding links, they may for convenience be grouped within the linkGrp (or other grouping element that uses them for linking).

To demonstrate this facility, we consider how we might encode the alignments in an extract from Comenius' , in the English translation of Charles Hoole (1659).

Each topic covered in this work has three parts: a picture, a prose text in Latin describing the topic, and a carefully-aligned translation of the Latin into English, German, or some other vernacular. Key terms in the two texts are typographically distinct, and are linked to the picture by numbers, which appear in the two texts and within the picture as well.

First, we consider the text portions. The English and Latin portions have been encoded as distinct div elements. Identifiers have been attached to each typographic line, but no other encoding added, to simplify the example.

Next we consider the non-textual parts of the page. Encoding this requires providing two distinct components: firstly a digitized rendering of the page itself, and secondly a representation of the areas within that image which are to be aligned. In section we present a simple way of doing this using the TEI-defined markup for alignment of facsimiles. In the present chapter we demonstrate a more powerful means of aligning arbitrary polygons and points, which uses the XML notation SVG (see SVG). This provides appropriate facilities for both these requirements: This example of SVG defines two rectangles at the locations with the specified x and y coordinates. A view is defined on these, enabling them to be mapped by an SVG processor to the image found at the URL specified (p1764.png). It also defines unique identifiers for the whole image, and the two views of it, which we will use within our alignment, as shown next (for further discussion of the handling of images and graphics, see section ; for further discussion of using non-TEI XML vocabularies such as SVG within a TEI document, see section ).

As printed, the Comenius text exhibits three kinds of alignment. The English and Latin portions are printed in two parallel columns, with corresponding phrases, (represented above by seg elements), more or less next to each other. Particular words or phrases are marked as terms in the two languages by a change of rendition: the English text, which otherwise uses black letter type throughout, has the words The Study, a Student, Studies, and Books in a roman font; in the Latin text, which is printed in roman, the corresponding words (Museum, Studiosus, Studiis, and Libros) are all in italic. Numbered labels appear within the text portions, linking keywords to each other and to sections of the picture. These labels, which have been left out of the above encoding, are attached to the first, third, and last segments in each language quoted below, and also appear (rather indistinctly) within the picture itself. Thus, the images of the study, the student, and his books are each aligned with the correct term for them in the two languages.

The first kind of alignment might be represented by using the corresp attribute on the seg element. The second kind might be represented by using the gloss and term mechanism described in section . The third kind of alignment might be represented using pointers embedded within the texts, for example: where a Student, ubi Studiosus, We choose however to use the link element, since this provides a more efficient way of representing the three-way alignment between English, Latin, and picture without redundancy.

This map, of course, only aligns whole segments and image portions, since these are the only parts of our encoding which bear identifiers and can therefore be pointed to. To add to it the alignment between the typographically distinct words mentioned above, new elements must be defined, either within the text itself or externally by using stand off techniques. Encoding these word pairs as term and gloss, although intuitively obvious, requires a non-trivial decision as to whether the Latin text is glossing the English, or vice versa. Tagging all the marked words as term avoids the difficult decision, but might be thought by some encoders to convey the wrong information about the words in question. Simply tagging them as additional embedded seg elements with identifiers that can be aligned like the others is also a possibility.

These solutions all require the addition of further markup to the text. This may pose no problems, or it may be infeasible, for example because the text is held on a read-only medium. If it is not feasible to add more markup to the original text, some form of stand-off markup will be needed. Any item within the text that can be pointed to using the various pointer schemes discussed in this chapter may be used, not simply those which rely on the existence of an xml:id attribute. Suppose our example had been more lightly tagged, as follows:

To express the same alignment mentioned above, we could use an XPath expression to identify the required seg elements: In the absence of any markup around individual substrings of the element content, the string-range pointer scheme discussed in may also be helpful: for example, to indicate that the words Studies and Studiis correspond, we might express the link between them as follows:

Aligning Synchronous Events

Provided that explicit elements are available to represent the parts or places to be synchronized, then the global linking attribute synch may be used to encode such synchronization, once it has been identified. This is another of the attributes made globally available by the mechanism described in the introduction to this chapter. Alternatively, the link and linkGrp elements may be used to make explicit the fact that the synchronous elements are aligned.

To illustrate the use of these mechanisms for marking synchrony, consider the following representation of a spoken text:

This representation uses numbers in brackets to mark the points at which speakers overlap each other. For example, the [1] in A's first speech is to be understood as coinciding with the [1] in B's second speech.This sample is taken from a conversation collected and transcribed for the British National Corpus.

To encode this we use the spoken texts module, described in chapter , together with the module described in the present chapter. First, we transcribe this text, marking the synchronous points with anchor elements, and providing a synch attribute on one of each of the pairs of synchronous anchors. As noted in the example given above (section ), correspondence, and hence synchrony, is a symmetric relation; therefore the attribute need only be specified on one of the pairs of synchronous anchors.

We can encode this same example using link and linkGrp elements to make the temporal alignment explicit. A back element has been used to enclose the linkGrp element, but the links may be located anywhere the encoder finds convenient: The xml:id attributes are provided for the link and linkGrp elements here for reasons discussed in the next section, .

As with other forms of alignment, synchronization may be expressed between stretches of speech as well as between points. When complete utterances are synchronous, for example, if one person says What? and another No! at the same time, that can be represented without anchor elements as follows. What? No!

A simple way of expressing overlap (where one speaker starts speaking before another has finished) is thus to use the seg element to encode the overlapping portions of speech. For example, So you're It will be nice in a way, but, be strange. Yeah , yeah, cos it, its the not Note in this encoding how synchronization has been effected between an empty unclear element and the content of a seg element, and between the content of an u element and that of another seg, using the synch attribute. Alternatively, a linkGrp could be used in the same way as above.

Placing Synchronous Events in Time

A synchronous alignment specifies which points in a spoken text occur at the same time, and the order in which they occur, but does not say at what time those points actually occur. If that information is available to the encoder it can be represented by means of the when and timeline elements, whose description and attributes are the following:

Each when element indicates a point in time, either directly by means of the absolute attribute, whose value is a string which specifies a particular time, or indirectly by means of the since attribute, which points to another when. If the since is used, then the interval and unit attributes should also be used to indicate the amount of time that has elapsed since the time specified by the element pointed to by the since attribute; the value -1 can be given to indicate that the interval is unknown.

If the when elements are uniformly spaced in time, then the interval and unit values need be given once in the timeline, and not repeated in any of the when elements. If the intervals vary, but the units are all the same, then the unit attribute alone can be given in the timeline element, and the interval attribute given in the when element.

The origin attribute in the timeline element points to a when element which specifies the reference or origin for the timings within the timeline; this must, of course, specify its position in time absolutely. If the origin of a timeline is unknown, then this attribute may be omitted.

The following timeline might be used to accompany the marked up conversation shown in the preceding section: The information in this timeline could now be linked to the information in the linkGrp which provides the temporal alignment (synchronization) for the text, as follows:

To avoid the need for two distinct link groups (one marking the synchronization of anchors with each other, and the other marking their alignment with points on the time line) it would be better to link the when elements with the synchronous points directly:

Finally, suppose that a digitized audio recording is also available, and an XML file that assigns identifiers to the various temporal spans of sound is available. For example, the following Synchronized Multimedia Integration Language (SMIL, pronounced "smile") fragment: URIs pointing to the audio elements could also be included as a fourth component in each of the above link elements, thus providing a synchronized audio track to complement the transcribed text.

For further discussion of this and related aspects of encoding transcribed speech, refer to chapter .

We had fun at the beach today. We had sun at the beach today.

We had fun at the beach today. We had sun at the beach today.

We had fun sun at the beach today.

We had fun at the beach today. We had sun at the beach today. Lee had fun at the beach today.

We had fun at the beach today. We had sun at the beach today. Lee had fun at the beach today.

We had fun at the beach today. We had sun at the beach today.

Her skin is tender as Dimaggio's a leather a baseball glove, and she bats from right to left. now ain't that too damn bad.

Introduction

Most of the mechanisms defined in this chapter rely to a greater or lesser extent on the fact that tags in a marked-up document can both assert a property for a span of text which they enclose, and assert the existence of an association between themselves and some other span of text elsewhere. In stand-off markup, there is a clear separation of these two behaviours: the markup does not directly contain any part of the text, but instead includes it by reference. One specific mechanism recommended by these Guidelines for this purpose is the standard XInclude mechanism defined by the W3C; another is to use pointers as demonstrated elsewhere in this chapter.

There are many reasons for using stand-off markup: the source text might be read-only so that additional markup cannot be added, or a single text may need to be marked up according to several hierarchically incompatible schemes, or a single scheme may need to accommodate multiple hierarchical ambiguities, so that a single markup tree is not the most faithful representation of the source material.

This section describes a generic mechanism for expressing all kinds of markup externally as stand-off tags, for use whenever it is appropriate.

Throughout this section the following terms will be systematically used in specific senses. source document a document to which the stand-off markup refers (a source document can be either XML or plain text); there may be more than one source document. internal markup markup that is already present in an XML source document stand-off markup markup that is either outside of the source document and points in to it to the data it describes, or alternatively is in another part of the source document and points elsewhere within the document to the data it describes external document a document that contains stand-off markup that points to a different, source document internalize the action of creating a new XML document with external markup and data integrated with the source document data, and possibly some source document markup as well externalize a process applied to markup from a pre-existing XML document, which splits it into two documents, an XML (external) document containing some of the markup of the original document, and another (source) XML document containing whatever text content and markup has not been extracted into the stand-off document; if all markup has been externalized from a document, the new source may be a plain text document

The three major requirements satisfied by this scheme for stand-off markup are: any valid TEI markup can be either internal or external, external markup can be internalized by applying it to the document content by either substituting the existing markup or adding to it, to form a valid TEI document, and the external markup itself specifies whether an internalized document is to be created by substituting the existing internal markup or by adding to it.

Overview of XInclude

Stand-off markup which relies on the inclusion of virtual content is adequately supported by the W3C XInclude recommendation, which is also recommended for use by these Guidelines.The version on which this text is based is the W3C Recommendation dated 20 December 2004.. XInclude defines a namespace (http://www.w3.org/2001/XInclude), which in these Guidelines will be associated with the prefix xi:, and exactly two elements, xi:include and xi:fallback. XInclude relies on the XPointer framework discussed elsewhere in this chapter to point to the actual fragments of text to be internalized. Although XInclude only requires support for the element() scheme of XPointer, these Guidelines permit the use of any of the pointing schemes discussed in section .

XInclude is a W3C recommendation which specifies a syntax for the inclusion within an XML document of data fragments placed in different resources. Included resources can be either plain text or XML. XInclude instructions within an XML document are meant to be replaced by a resource targetted by a URI, possibly augmented by an XPointer that identifies the exact subresource to be included.

The xi:include element uses the href attribute to specify the location of the resource to be included; its value is an URI containing, if necessary, an XPointer. Additionally, it uses the parse attribute (whose only valid values are text and xml) to specify whether the included content is plain text or an XML fragment, and the encoding attribute to provide a hint, when the included fragment is text, of the character encoding of the fragment. An optional xi:fallback element is also permitted within an xi:include; it specifies alternative content to be used when the external resource cannot be fetched for some reason. Its use is not however recommended for stand-off markup.

Stand-off Markup in TEI

The operations of internalizing and externalizing markup are very useful and practically important. XInclude processing as defined by the W3C is internalization of one or more source documents' content into a stand-off document. TEI use of XInclude for stand-off markup enables use of XInclude-conformant software to perform this useful operation. However, internalization is not clearly defined for all stand-off files, because the structure of the internal and external markup trees may overlap. In particular, when an external markup document selects a range that overlaps partial elements in the source document, it is not clear how the semantics of internalization (inclusion) should work, since partial elements are not XML objects.This corresponds to the observation that overlapping XML tags reflecting a textual version of such an inclusion would not even be well-formed XML. This kind of overlap in textual phenomena of interest is in fact the major reason that stand-off markup is needed. XInclude defines a semantics for this case that involves only complete elements.

When a range selection partially overlaps a number of elements in a source document, XInclude specifies that the partially overlapping elements should be included as well as all completely overlapping elements and characters (partially overlapping characters are not possible). The effect of this is that elements that straddle the start or end of a selected range will be included as wrappers for those of their children that are completely or partially selected by the range. For example, given the following source document:

home, home on Brokeback Mountain.

That was the song that I sang

and the following external document:

]]> the resulting document after XInclude processing of this external document would be:

home, home on Brokeback Mountain.

That was the song that I sang

]]> The result of the inclusion is two paragraph elements, while the original range designated in the source document overlapped two paragraph fragments. The semantics of XInclude require the creation of well-formed XML results even though the pointing mechanisms it uses do not necessarily respect the hierarchical structure of XML documents, as in this case. While this is a good way to ensure that internalization is always possible, it has implications for the use of XInclude as a notation for the description of overlapping markup structures.

When overlapping hierarchies need to be represented for a single document, each hierarchy must be represented by a separate set of XInclude tags pointing to a common source document. This sort of structure corresponds to common practice in work with linguistic text corpora. In such corpora, each potentially overlapping hierarchy of elements for the text is represented as a separate stream of stand-off markup. Generally the source text contains markup for the smallest significant units of analysis in the corpus, such as words or morphemes, this information and its markup representing a layer of common information that is shared by all the various hierarchies. As a way of organizing the representation of complex data, this technique generally allows a large number of xml:id attributes to be attached to the shared elements, providing robust anchors for links and facilitating adjustments to the source document without breaking external documents that reference it.

Any tag can be externalized by removing its content and replacing it with an xi:include element that contains an XPointer pointing to the desired content.

For instance the following portion of a TEI document: 1755 To make a prairie it takes a clover and one bee, One clover, and a bee, And revery. The revery alone will do, If bees are few. can be externalized by placing the actual text in a separate document, and providing exactly the same markup with the xi:include elements: Source.xml To make a prairie it takes a clover and one bee,\n One clover, and a bee,\n And revery.\n The revery alone will do,\n If bees are few.\n ]]> External.xml 1755 ]]>

Please note that this specification requires that the XInclude namespace declaration is present in all cases. The xi:fallback element contains text or XML fragments to be placed in the document if the inclusion fails for any reason (for instance due to inaccessibility of an external resource). The xi:fallback element is optional; if it is not present an XInclude processor must signal a fatal error when a resource is not found. This is the preferred behaviour for use with stand-off markup. These Guidelines recommend against the use of xi:fallback for stand-off markup.

Well-formedness and Validity of Stand-off Markup

The whole source fragment identified by an XInclude element, as well as any markup therein contained is inserted in the position specified, and an XInclude processor is required to ensure that the resulting internalized document is well-formed. This has obvious implications when the external document contains XML markup. A plain text source document will always create a well-formed internalized document.

While a TEI customization may permit xi:include elements in various places in a TEI document instance, in general these Guidelines suggest that validity be verified after the resolution of all the xi:include elements.

Including Text or XML Fragments

When the source text is plain text the overall form of the XPointer pointing to it is of minimal importance. The form of the XPointer matters considerably, on the other hand, when the source document is XML.

In this case, it is rather important to distinguish whether we intend to substitute the source XML with the new one, or just to add new markup to it. The XPointers used in the references can express both cases.

A simple way is to make sure to select only textual data in the XPointer. For instance, given the following document: Source.xhtml

the expression range(element(/1/2/1.0),element(/1/2/11.1)) will select the whole poem, text content and div elements and hypertext links (NB: in XPointer whitespace-only text nodes count).

On the contrary, the expressions xpointer(//text()/range-to(.)) and xpointer(string-range(//text(),"To")/range-to(//text(),"few.") will only select the text of the poem, with no markup inside.

Thus, the following could be a valid stand-off document for the document: External2.xml 1755 ]]>