IMS Question & Test Interoperability: ASI Selection & Ordering Final Specification Version 1.2

Table of Contents

1. Introduction

1.1 Question & Test Interoperability Overview

The IMS Question & Test Interoperability (QTI) specification describes a basic structure for the representation of question (item) and test (assessment) data and their corresponding results reports [QTI, 02i]. Therefore, the specification enables the exchange of this item, assessment and results data between Learning Management Systems, as well as content authors and, content libraries and collections. The IMS QTI specification is defined in XML to promote the widest possible adoption. XML is a powerful, flexible, industry standard mark-up language used to encode data models for Internet-enabled and distributed applications. The IMS QTI specification is extensible and customizable to permit immediate adoption, even in specialized or proprietary systems. Leading suppliers and consumers of learning products, services and content contributed time and expertise to produce this final specification. The IMS QTI specification, like all IMS specifications, does not limit product designs by specifying user interfaces, pedagogical paradigms, or establishing technology or policies that constrain innovation, interoperability, or reuse.

This 'Selection & Ordering' specification contains the description of how the sequence in which Sections and/or Items are presented can be controlled. The selection and ordering process is a two-stage operation in which the child objects are selected according to some defined criteria e.g. meta-data content, etc. and the order of their presentation is then determined. The selection and ordering process within an object is limited to the immediate children of the object and so complex requirements must be based upon the appropriate usage of Sections to contain the Section/Item hierarchies. This document contains the relevant information model, XML binding and best practices guidance but it should be read in the context of the core ASI documents.

1.2 Scope & Context

This document is the IMS Question & Test Interoperability ASI Selection & Ordering Specification and must be read in conjunction with the core documents:

• IMS QTI: ASI Information Model [QTI, 02a];
• IMS QTI: ASI XML Binding [QTI, 02b];
• IMS QTI: ASI Best Practice & Implementation Guide [QTI, 02c].

It defines the selection and ordering features that are to be applied to the Section and/or Items. These new Assessment and Section features are not backwards compatible with the other versions of the IMS QTI specifications. These new features have no effect on the IMS QTILite specification [QTI, 02h] because that refers only to the Item data structure.

1.3 Structure of this Document

The structure of this document is:

2. Information Model	The description of the information model of the selection & ordering component for the full IMS QTI ASI;
3. XML Binding	The description of the XML binding of the selection & ordering component for the full IMS QTI ASI;
4. Best Practice & Implementation Guide	The description of the best practices and implementation guide for the selection & ordering component of the full IMS QTI ASI;
Appendix A - Glossary of Terms	A brief description of all of the elements and attributes that have been used to support selection & ordering.
Appendix B - Logic Rules	The set of logic rules that are supported by the or_selection, and_selection and not_selection elements.

1.4 Nomenclature

API	Application Programming Interface
ASI	Assessment, Section, Item
BAT	Balanced Adaptive Testing
CAT	Computer Adaptive Testing
CBT	Computer Based Training
DTD	Document Type Definition
IRT	Item Response Theory
QTI	Question & Test Interoperability
VLE	Virtual Learning Environment
W3C	World Wide Web Consortium
XML	Extensible Mark-up Language
XSD	XML Schema Data

1.5 References

[IMS, 01]	IMS Persistent, Location-Independent Resource Identifier Implementation Handbook, M.McKell, Version 1.0, IMS, April 2001.
[QTI, 02a]	IMS Question & Test Interoperability ASI Information Model Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02b]	IMS Question & Test Interoperability: ASI XML Binding Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02c]	IMS Question & Test Interoperability: ASI Best Practice & Implementation Guide, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02d]	IMS Question & Test Interoperability: ASI Outcomes Processing Specification, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02e]	IMS Question & Test Interoperability: Results Reporting Information Model, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02f]	IMS Question & Test Interoperability: Results Reporting XML Binding, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02g]	IMS Question & Test Interoperability: Results Reporting Best Practice & Implementation Guide, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02h]	IMS Question & Test Interoperability: QTILite Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02i]	IMS Question & Test Interoperability: Overview, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.

2. Information Model

2.1 Exchanging ASI Objects

The possible advanced Assessment structures that can be exchanged using QTI are shown in Figure 2.1:

• The Assessment could contain a single Section block (Figure 2.1a);
• The Assessment could consist of a reference to an external Section block (Figure 2.1b);
• The Assessment could consist of a mixture of contained and externally referenced Section blocks (Figure 2.1c). There is no preferential order in the blocks and references.

(a)	(c)
(b)

In the case of Sections, some of the possible data structures that can be exchanged are shown in Figure 2.2:

• The Section could contain one or more Items (Figure 2.2a);
• The Section could contain one or more references to external Items (Figure 2.2b);
• The Section could contain one or more Sections (Figure 2.2c);
• The Section could contain one or more references to external Sections (Figure 2.2d);
• The Section could contain a mixture of blocks and references to Section and Items. In V1.1 and earlier releases, the only constraint was that all of the Section blocks and references occur before the Item blocks and references. This has been changed in V1.2 and the Section and Item structures can be interleaved in any combination.

(a)	(b)
(c)	(d)

(e)

The wide range of different data structures that can be exchanged is especially significant when developing mechanisms that support 'Selection & Ordering'. 'Selection & Ordering' is the capability to define the rules that should be used for the selection and the order of Sections and/or Items to be presented to the participant. This selection and ordering mechanism must support blocks that are contained and referenced.

2.2 Use Cases

The following use-cases have been identified for support within QTI ASI selection and ordering V1.2:

1. Simple selection - this is what QTI supports currently i.e. an Assessment contains a list of Sections and/or Items specified precisely. In some instances, these items should be required to be answered in order, in other cases, the participant should be free to choose their own order, or range freely through the items;
2. Basic random selection - an assessment is to consist of 'x' Items chosen randomly from 'y'. For example, an Assessment has 50 Items, but each time it is presented, 20 of them are to be selected at random. This includes the two cases of when repetition is and is not permitted;
3. Topic random selection - as basic random selection, but there are to be a number of Items from different "topics", where topic is some kind of property or structure of items or an item bank. For example, an Assessment is to contain 20 items, 5 chosen at random from topic 1, 5 chosen at random from topic 2, and 10 chosen at random from topic 3. Topic could be extended for QTI to include meta-data and Item attributes, for instance, 10 Items on the subject 'Geometry' with a difficulty level of 'easy' instead of 10 Items from a topic;
4. Golden questions - a common requirement is to make some items "golden", i.e. always selected. This modifies (b) or (c) above to allow some items always to be selected. For example, an Assessment is to contain 10 Items chosen at random from topic 1, and 10 Items chosen at random from topic 2, but always Items 1a, 1b and 1c from topic 1 (a common method of implementing golden Items is to say that random selection should never include an Item already in the Assessment thus if the Assessment includes items 1a, 1b and 1c and seven chosen at random from topic 1, this implicitly means that the seven should not be 1a, 1b or 1c);
5. Mixing up or not - in some cases the order of topics would need to be mixed and some would not. For example, if there are 10 Items from topic 1 and 10 Items from topic 2, sometimes all the Items in one topic need to be presented first, followed by the Items in the second topic. At other times they will need to be mixed randomly';
6. Testlets - several Items that share a common stimulus. For example, a reading passage followed by three multiple-choice items. For fixed selection and ordering, these can be expressed as a single Item with multiple responses/outcomes. However, often Items are written such that there are 10 or so Items for a stimulus of which only 3-5 are used on a given test form;
7. More complex use-cases - other things are required:
   • Constraining the random selection by other criteria, e.g. level of difficulty
   • Making some questions require another question, e.g. if Item 'X' is presented then Item 'Y' must also be presented (this is an overlap inclusion requirement) - for further study in V2.0
   • Making some questions force removal of another question, e.g. never present both Items 'X' and 'Y' in the same assessment (this is an overlap exclusion requirement) - for further study in V2.0.

The next use-cases will be considered for inclusion in later releases of the specification:

1. Greco-Latin Squares - the name comes from Statistical Design literature where Greek and Latin letters are filled in a grid to determine balancing among the factors in an experimental design. From the statistical literature we get Block designs, Spiral designs, Latin Hypercubes, Partially-balanced incomplete blocks, etc. An example of this is the National Assessment for Educational Progress (NAEP), a US regular survey of educational achievement. In NAEP, the goal is to gain an overall impression on how all the students in the Country (or State) are doing, and not focused on one individual student's achievement. NAEP uses a block spiral design where the first student's booklet has sections 1,2 and 3. The second has sections 1, 3 and 4. The third section 1, 4 and 5, etc;
2. Exposure Control - in high stakes testing situations the number of times an Item is given must be limited as this reduces the chances of it becoming compromised. Balancing exposure across Items in a pool is important because it is another aspect which works across candidates;
3. Conditional selection on Item Outcome/Response - this comes from survey questions which the instructions read, if your answer is "No", skip to Item 'XXXX'. For example, if the answer to "Do you have a computer at home?" is 'No', there is no need to ask "How many hours a week do you use if for homework?" A related example is if the participant was always asked two multiple-choice questions. The first asked for the answer to some problem and the second asked what strategy the student used to arrive at the answer. Obviously, the text of the second question is dependent on the first - this is being studied by the IMS Sequencing working-group;
4. Simulated Cases - a more complex generalization of the previous case e.g. a "simulated medical exam" by having a medical student make a series of choices about patient treatment. The next question would follow on in sequence from the previous decisions;
5. Content Balancing - this is based upon selection that requires balanced selection e.g. "Two from Column A and three from Column B". Items or Sections are classified according to a content code and selection must balance across these codes. Often, there are upper and lower bounds rather than exact values;
6. Multiple Content Balancing - simultaneously balancing across multiple content categories;
7. A form of Computer Adaptive Testing (CAT) that refers to those kinds of adaptivity that are based on current estimates of examinee ability e.g. the result of outcome processing and not just response processing. Typically, Items are chosen to maximize "value of information" or "weight of evidence". For example, in IRT-CAT Items whose difficulty matches the current best estimate of examine proficiency can be selected;
8. Balanced Adaptive Testing (BAT) - adaptive testing subject to content balancing, overlap and exposure control constraints. Sometimes these are call "Barely Adaptive Tests" because unless the constraints are well balanced, the adaptivity can be overwhelmed by content balancing and exposure control constraints;
9. Multistage Testing - this is a test which is composed of stages (or Sections) and it is adaptive at the stage level but not within stages;
10. Content Balancing - this is based upon selection that requires balanced selection e.g. "Two from Column A and three from Column B". Items or Sections are classified according to a content code and selection must balance across these codes. Often, there are upper and lower bounds rather than exact values;
11. Multiple Content Balancing - simultaneously balancing across multiple content categories.

2.3 ASI Issues

The core features within the ASI structures that are related to the process of selection and ordering are shown in Figure 2.3.

The relationship of these features to selection and ordering are:

• <qtimetadata> - contains the qti-specific meta-data about the object. This meta-data may be used by the selection and ordering algorithms;
• <precondition> - defines the preconditions that must be 'True' in order that the objects can be processed by the assessment rendering engine (for further study in V2.0);
• <postcondition> - defines the postconditions that must be set as 'True' once the object processing has been completed (for further study in V2.0);
• <selection_ordering> - the selection and ordering rules that are to be applied to the set of child objects contained within the parent;
• Internal and externally referenced Section and Item objects - the set of contained and referenced objects which can be selected and ordered;
• External metadata - the external meta-data descriptions that are linked to the object and which conform to the IMS Meta-data specification.

The scoping of the selection and ordering algorithms that are supported is summarized using the example Section/Item combination shown in Figure 2.4. The scope of the selection and ordering is:

• For Section 'Section Ident_1' the set of children objects is - 'Item Ident_1', 'Section Ident_2', 'Section Ident-3' and 'Item Ident_2'. This means that any combination of these four objects can be selected and ordered;
• For Section 'Section Ident_2' the set of children objects is - 'Item Ident_2.1', 'Item Ident_2.2' and 'Item Ident_2.3';
• For Section 'Section Ident_3' the set of children objects is - 'Item Ident_3.1', 'Item Ident_3.2' and 'Item Ident_3.3';
• If for all of these three Sections all of the objects are selected and the order of presentation is linear (the default configuration) then the Items will be presented in the following order - 'Item Ident_1', 'Item Ident_2.1', 'Item Ident_2.2', 'Item Ident_2.3', 'Item Ident_3.1', 'Item Ident_3.2', 'Item Ident_3.3', 'Item Ident_2'. If the objects were ordered randomly, but all were selected, then the order could be - 'Item Ident_1', 'Item Ident_2.2', 'Item Ident_2.1', 'Item Ident_2.3', 'Item Ident_3.1', 'Item Ident_3.3', 'Item Ident_3.2', 'Item Ident_2'. In both examples the position of 'Item Ident_1' and 'Item Ident_2' is unchanged.

2.4 Selection & Ordering Solution

The 'Selection & Ordering' solution is based upon the separation of the process into three distinct but related stages:

• Object sequence - this is the set of constraints that are applied to the selection and ordering rules e.g. whether or not an object can be presented more than once;
• Object selection - this is the first process by which the set of rules for the selection of the objects are applied to the objects. This application is limited to the direct children of the parent containing the selection rules. The application of these rules is at run-time;
• Object ordering - this is the second process by which the order of the selected objects is determined according to the defined rules. The application of these rules is at run-time.

Object selection and ordering are applied to the fixed properties of the objects i.e. the dynamic properties of the object are not used. This combination of the selection and ordering is the 'sequencing' operation.

2.4.1 Selection

The following selection rules are supported:

1. 'All' - all of the contained objects are selected. The element <selection_number> must not be used;
2. 'Parameterized All' - select all of the objects that have particular properties (the properties are characterized by the object's meta-data). The element <selection_number> must not be used;
3. 'Partial' - the random selection of 'x' objects from the set of 'y' objects (this assumes that x=y). The number of objects is defined using the instruction
   
         <selection_number>value</selection_number>
   
   where the 'value' is the required number of objects;
4. 'Parameterized Partial' - the random selection of 'x' objects that have particular properties from the set of 'y' objects (the properties are characterized by the object's meta-data). This algorithm is implied from the usage of the <selection_metadata> and <selection_number> elements;
5. 'Logical' - the logical association of the objects such that the selection of one object is based upon the selection of another. This includes mutual exclusion and mutual inclusion. This is supported through the appropriate structuring of Sections and Items.

The rules that are to be applied to the selection of objects are:

1. The default condition is equivalent to the unique selection of all of the objects contained within the parent object;
2. Multiple selection rules can be applied to the same group of objects e.g. all of the Items within a Section. These rules can be combined in a variety of ways and so the full selection criteria can only be applied once all of the rules have been parsed - multiple rules can be contained in one or more selection structures. The final selection is the union of all of the selected objects - rules form different selection structures are additive in nature;
3. Rules must be applied explicitly to each object and are relevant to the direct children of the identified object. The order in which the rules are applied has no effect as the order of presentation of the objects to the participant is controlled using other functions;
4. If there is any selection conflict due to the hierarchical nature of the objects then the outermost rules take precedence;
5. An object could in fact be an object-bank i.e. a container of many Sections and/or Items. Only one object-bank can be referenced by each selection structure hence reference to multiple object-banks is via multiple selection structures.

2.4.2 Ordering

The following ordering rules are supported:

1. 'Fixed' - the specifically identified object is to be ordered as identified by the relative position of the 'order 'statement in relation to the other 'order' statements. This will be supported through the appropriate combination Section and Item structures;
2. 'Sequential' - all of the selected objects are presented in the order in which they were selected. This is defined using the attribute 'order_type="Sequential"';
3. 'Random' - all of the selected objects are presented in a random order and the order will change from instantiation to instantiation. This is defined using the attribute 'order_type="Random"'.

The rules that are to be applied to the ordering of the objects are:

1. The default condition is that the objects are presented in the order in which they were selected. If all of the objects have been selected then that order is defined by the order in which they are stored within the ASI structure;
2. The randomization of order is applied only to those objects that have not been specifically identified and explicitly positioned within the order;
3. If there is any ordering conflict due to the hierarchical nature of the objects then the outermost rules take precedence.

2.4.3 Sequencing

At the current time there are two sequencing rules that can be applied to the selection and ordering rules:

1. Normal - each object can be presented only once i.e. there is no repetition. This is the default condition and as such is assumed if the 'sequence_type' attribute is absent. The unique identity of the object is implied through its 'ident' and so if these are not uniquely allocated (including their usage in object-banks) then errors in the selection and ordering will occur;
2. Repeat - each object can be presented any number of times. The total number of objects that can be selected is defined as the termination clause and is set using the associated instruction
   
        '<sequence_parameter pname="totalobjectnumber">10</sequence_parameter>'
   
   where the value '10' is replaced by the total number of objects that must be selected.

2.4.4 Preconditions & Postconditions

For further study in V2.0.

2.5 Tabular Representation

The tables in this Section provide a conceptual, informative description of the elements in the data objects. The columns in these tables refer to:

No:	The number of the data element. An element may be composed of sub-elements. The numbering scheme reflects these relationships.
Name:	The descriptive name of the element.
Explanation:	A brief functional description of the element.
Required:	Indicates if the element is required: M = Mandatory Element that must be included in the data object, if the element at the higher level is included; C = Conditional Element. Existence is dependent on values of other Elements; O = Optional Element.
Multi:	Multiplicity of the element: Blank = single instance; Number = maximum number of times the element is repeatable; n = multiple occurrences allowed, no limit; Repeatability of an element implies that all sub-elements repeat with the same element.
Type:	A description of formatting rules for the data element. Type includes the maximum length of the element: ID = element used to uniquely identify an object; Code = element value from a list of codes; Description = descriptive element, human language Flag = binary flag Enumerated = list of predefined non-numeric options i.e. the definitive list of objects The international character set specified by ISO 10646 will be used for all fields. The type will also include a description of the set of valid values for the sub-element: Coding schemes using numerical values; The set of values as defined in the Domain i.e. making it closed. The list of values cannot be extended to include values not defined in the specification. If there is a need for values not included in the domain set of values then the extension should be done defining a new element under the Extension element that is a part of each data object definition.

Note: Additional descriptive information about the element.

Table 2.1 describes the data objects that are used in the construction of the QTI results reporting package itself.

No	Name	Explanation	Reqd	Mult	Type	Note
1	selection_ordering	The container for the set of selection and ordering rules that will be applied to the contained set of children Sections and/or Items.	O
1.1	sequence_type	Defines the nature of the total sequencing algorithm that is composed of the selection and ordering rules.	O			The set of possible values are 'Normal' and 'RandomRepeat'. 'Normal' is the assumed default value. Proprietary names can also be included.
1.2	qticomment	The comments used to annotate the XML instance file.	O			Comments should be used to aid human readability of the XML file itself.
1.2.1	xml:lang	The language that is used for the comment.	O		String. 32 chars.	The language entries will be defined as per the ISO standard.
1.3	sequence_parameter	The container for parameters that are to be passed to bias the sequencing rules.	O	n	PCDATA String 64 chars.
1.3.1	pname	The name of the parameterized sequence rule that is to be applied.	M		String. 64 chars.	The name of the parameterized sequence rule that is to be applied.
1.4	selection	The rules that are to be applied to select the set of objects that will then be order for presentation.	O	n		The absence of this element means that all of the objects are to be selected.
1.4.1	sourcebank_ref	The identifier of the objectbank that is to be used as the source of objects to which the selection rules are to be applied.	O		String 255 chars.	This would have been assigned to the objectbank using the 'ident' attribute as defined within the IMS QTI ASI Information Model.
1.4.2	selection_number	A selection rule that is used to define the number of objects that are to be randomly but uniquely selected from the available set of objects.	O		String 4 chars.	Used to identify the number of partially selected objects.
1.4.3	selection_metadata	A selection rule that is applied to the object's meta-data fields. Each rule is applied to a single meta-data field.	O	n	String. 64 chars.	Contains the value of the named meta-data field against which the operator- defined test is to be applied.
1.4.3.1	mdname	The name of the meta-data field whose value is to be examined.	M		String. 64 chars.	The named meta-data field may by QTI-specific or may be externally referenced.
1.4.3.2	mdoperator	The operation on the meta-data field that is to be applied in the rule.	M		Enumerated list consisting of: EQ, NEQ, LT, LTE, GT, GTE.
1.4.4	and_selection	Used to construct complex rules that are based upon the logical 'AND' of the contained rules.	C			The rules for this logic are detailed in Appendix B.
1.4.4.1	selection_metadata		O	n	As per structure 1.4.4
1.4.4.2	and_selection		O	n	As per structure 1.4.5
1.4.4.3	or_selection		O	n	As per structure 1.4.6
1.4.4.4	not_selection		O	n	As per structure 1.4.7
1.4.5	or_selection	Used to construct complex rules that are based upon the logical 'OR' of the contained rules.	C	n		The rules for this logic are detailed in Appendix B.
1.4.5.1	selection_metadata		O	n	As per structure 1.4.4
1.4.5.2	and_selection		O	n	As per structure 1.4.5
1.4.5.3	or_selection		O	n	As per structure 1.4.6
1.4.5.4	not_selection		O	n	As per structure 1.4.7
1.4.6	not_selection	Used to construct complex rules that are based upon the logical 'NOT' of the contained rules.	C			The rules for this logic are detailed in Appendix B.
1.4.461	selection_metadata		C		As per structure 1.4.4
1.4.6.2	and_selection				As per structure 1.4.5
1.4.6.3	or_selection				As per structure 1.4.6
1.4.6.4	not_selection				As per structure 1.4.7
1.4.7	selection_extension	The selection rule extension facility.	O	n	ANY	This extension approach should be used if the rule is to be based upon new and/or established elements.
1.5	order	The rule that is to be applied to establish the order of the objects that have been selected.	O		There is no content for this element.	The absence of this element means that the objects are to be presented in the order that they are declared within the parent object.
1.5.1	order_type	The type of ordering that is to be applied.	O			An enumerated set consisting of: Random, Sequential (default) Proprietary names can also be included.
1.5.2	order_extension	The order rule extension facility.	O	n	ANY	This extension approach should be used if the rule is to be based upon new and/or established elements.

3. XML Binding

3.1 <selection_ordering> Elements

Description: The <selection_ordering> element is the container for all of the selection and ordering instructions that are to be applied to the corresponding objects (Sections and Items). The positioning of the Section and Item objects within the parent object defines the default coverage of the ordering with respect to the selected objects.

Multiplicity: The <selection_ordering> occurs zero or once within the <assessment> and <section> elements.

Attributes: None.

Elements:

• qticomment
• sequence_parameter
• selection
• order

Example:

<selection_ordering>
  <selection sequence_type="Normal"/>
  <order order_type="All"/>
</selection_ordering>

3.1.1 <qticomment>

Description: This element contains the comments that are relevant to the selection and ordering structure as a whole.

Multiplicity: Occurs zero or once within the <selection_ordering> element.

Attributes:

• xml:lang (optional). Identifies the language that is to be used within the instance. The default is set as English but the potential range of languages is defined as per the XML W3C specification.
  Data-type = string.

3.1.2 <sequence_parameter>

Description: This element contains the comments that are relevant to the selection and ordering structure as a whole.

Multiplicity: Occurs zero or once within the <selection_ordering> element.

Attributes:

• pname (mandatory). The name of the proprietary parameterized sequence rule that is to be used by the selection and ordering engine.
  Data-type = string (1-64 chars).

3.1.3 <selection>

Description: This element contains the selection instructions that are to be applied to the objects contained within the parent object.

Multiplicity: Occurs zero or more times within the <selection_ordering> element.

Attributes: See sub-section 3.2.

3.1.4 <order>

Description: This element contains the ordering instructions that are to be applied to the objects that have been previously selected.

Multiplicity: Occurs zero or more times within the <selection_ordering> element.

Attributes: See sub-section 3.6.

3.2 <selection> Elements

Description: The <selection> element is used to express the selection rules that are to be applied to the identified objectbank or the set of child objects contained within the parent. The full set of rules must be parsed before the consistent selection can be achieved.

Multiplicity: The <selection> occurs once or more times in the <selection_ordering> element.

Attributes: None.

Elements:

• sourcebank_ref
• selection_number
• selection_metadata
• or_selection
• and_selection
• not_selection
• selection_extension

3.2.1 <sourcebank_ref>

Description: Identifies the objectbank to which the selection and ordering rules are to be applied. This objectbank may or may not be contained in the same <questestinterop> package.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes: None.

Elements: None.

Example:

<selection_ordering>
  <selection>
    <sourcebank_ref>objectbankidentifier01</sourcebank_ref>
    <selection_number>10</selection_number>
  </selection>
</selection_ordering>

3.2.2 <selection_number>

Description: This element defines the partial selection rule i.e. chose 'x' objects from the set of objects contained in the identified object or parent object. This data is an integer number in the range 1-4096.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes: None.

Elements: None.

Example:

<selection>
  <selection_number>7</selection_number>
</selection>

3.2.3 <selection_metadata>

Description: This element defines the rule that is applied to the IMS QTI-specific meta-data and/or IMS Meta-data fields of the object. The content contains the value of the meta-data field that is being tested for within the rule. This data is a string of up to 64 characters length.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes:

• mdname (mandatory). Identifies the IMS QTI-specific or IMS Meta-data field that is to be used for the selection rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

Example:

<selection>
  <selection_metadata mdname="qmd_timelimit" mdoperator="LTE">5</selection_metadata>
</selection>

3.2.4 <or_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'OR' operator. The object is added to the list of selected objects if any or more of the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes: See sub-section 3.3.

3.2.5 <and_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'AND' operator. The object is added to the list of selected objects if all of the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes: See sub-section 3.4.

3.2.6 <not_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is not added to the list of selected objects if the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <selection> element.

Attributes: See sub-section 3.5.

3.2.7 <selection_extension>

Description: This element allows proprietary extensions to be made to the selection rules. The nature of these extensions is limited to that of the 'ANY' definition for an element within the XML schema.

Multiplicity: Occurs zero or many times within the <selection> element.

Attributes: None.

Elements: None.

3.3 <or_selection> Elements

Description: The <or_selection> element is used to express the selection of the object if at least one of the rules is found to be relevant. It is used to select objects that have particular metadata content or through the parameterized extension mechanism. The rules for the operation of this logic are described in Appendix B.

Multiplicity: The <or_selection> occurs zero or once in the <selection> element.

Attributes: None.

Elements:

• selection_metadata
• or_selection
• and_selection
• not_selection

Example:

<or_selection>
  <selection_metadata mdname="qmd_topic" mdoperator="EQ">algebra</selection_metadata>
  <selection_metadata mdname="qmd_topic" mdoperator="EQ">geometry</selection_metadata>
</or_selection>

3.3.1 <selection_metadata>

Description: This element defines the rule that is applied to the IMS QTI-specific and/or IMS Meta-data meta-data fields of the object. The content contains the value of the meta-data field that is being tested for within the rule. This data is a string of up to 64 characters length.

Multiplicity: Occurs zero or many times within the <or_selection> element.

Attributes:

• mdname (mandatory). Identifies the QTI-specific or IMS Meta-data field that is to be used for the selection rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.3.2 <or_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'OR' operator. The object is added to the list of selected objects if any or more of the contained rules are 'True'.

Multiplicity: Occurs zero or many times within the <or_selection> element.

Attributes: See sub-section 3.3.

3.3.3 <and_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'AND' operator. The object is added to the list of selected objects if all of the contained rules are 'True'.

Multiplicity: Occurs zero or many times within the <or_selection> element.

Attributes: See sub-section 3.4.

3.3.4 <not_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is not added to the list of selected objects if the contained rules are 'True'.

Multiplicity: Occurs zero or many times within the <or_selection> element.

Attributes: See sub-section 3.5.

3.4 <and_selection> Elements

Description: The <and_selection> element is used to express the selection of the object if all of the contained rules are found to be 'True'. It is used to select objects that have particular metadata content or through the parameterized extension mechanism. The rules for the operation of this logic are described in Appendix B.

Multiplicity: The <and_selection> occurs zero or once within the <selection> element.

Attributes: None.

Elements:

• selection_metadata
• or_selection
• and_selection
• not_selection

Example:

<and_selection>
  <selection_metadata mdname="qmd_topic" mdoperator="EQ">algebra</selection_metadata>
  <selection_metadata mdname="qmd_weighting" mdoperator="EQ">2</selection_metadata>
</and_selection>

3.4.1 <selection_metadata>

Description: This element defines the rule that is applied to the IMS QTI-specific and/or IMS Meta-data meta-data fields of the object. The content contains the value of the meta-data field that is being tested for within the rule. This data is a string of up to 64 characters length.

Multiplicity: Occurs zero or many times within the <and_selection> element.

Attributes:

• mdname (mandatory). Identifies the QTI-specific or IMS Meta-data field that is to be used for the selection rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.4.2 <or_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'OR' operator. The object is added to the list of selected objects if any or more of the contained rules are 'True'.

Multiplicity: Occurs zero or more times within the <and_selection> element.

Attributes: See sub-section 3.3.

3.4.3 <and_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'AND' operator. The object is added to the list of selected objects if all of the contained rules are 'True'.

Multiplicity: Occurs zero or more times within the <and_selection> element.

Attributes: See sub-section 3.4.

3.4.4 <not_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is not added to the list of selected objects if the contained rules are 'True'.

Multiplicity: Occurs zero or more times within the <and_selection> element.

Attributes: See sub-section 3.5.

3.5 <not_selection> Elements

Description: The <not_selection> element is used to exclude the identified objects from selection. The object will not be selected if the contained rule is 'True'. It is used to select objects that have particular metadata content or through the parameterized extension mechanism. The rules for the operation of this logic are described in Appendix B.

Multiplicity: The <not_selection> occurs zero or once within the <selection> element.

Attributes: None.

Elements:

• selection_metadata
• or_selection
• and_selection
• not_selection

Example:

<not_selection>
  <or_selection>
    <selection_metadata mdname="qmd_topic" mdoperator="EQ">algebra</selection_metadata>
    <selection_metadata mdname="qmd_topic" mdoperator="EQ">geometry</selection_metadata>
  </or_selection>
</not_selection>

3.5.1 <selection_metadata>

Description: This element defines the rule that is applied to the QTI-specific and/or IMS Meta-data meta-data fields of the object. The content contains the value of the meta-data field that is being tested for within the rule. This data is a string of up to 64 characters length.

Multiplicity: Occurs zero or once within the <not_selection> element.

Attributes:

• mdname (mandatory). Identifies the IMS QTI-specific or IMS Meta-data field that is to be used for the selection rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.5.2 <or_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'OR' operator. The object is added to the list of selected objects if any or more of the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <not_selection> element.

Attributes: See sub-section 3.3.

3.5.3 <and_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'AND' operator. The object is added to the list of selected objects if all of the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <not_selection> element.

Attributes: See sub-section 3.4.

3.5.4 <not_selection>

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is not added to the list of selected objects if the contained rules are 'True'.

Multiplicity: Occurs zero or once within the <not_selection> element.

Attributes: See sub-section 3.5.

3.6 <order>

Description: This element contains the ordering instructions that are to be applied to the objects that have been previously selected.

Multiplicity: Occurs zero or once within the <selection_ordering> element.

Attributes:

• order_type (Required). Identifies the way in which the enclosed objects are to be ordered. 'Sequential' means that the order follows the order in which the objects are contained in the parent child after all of the selection rules have been applied. 'Random' means that the order is to be randomized. Proprietary names can also be used but these will only have localized meaning.
  Data-type = string (1-16 chars).

3.6.1 <order_extension>

Description: This element allows proprietary extensions to be made to the order rules. The nature of these extensions is limited to that of the 'ANY' definition for an element within the XML schema.

Multiplicity: Occurs zero or once within the <order> element.

Attributes: None.

Elements: None.

4. Best Practice & Implementation Guide

4.1 Overall Data Model

The overall IMS QTI ASI Selection & Ordering data model is shown in Figure 4.1.

4.2 Relationship to the Other IMS QTI Specifications

The relationship of this 'Selection & Ordering' specifications to the other parts of the IMS QTI specifications is:

• The 'Selection & Ordering' root element is a part of the Assessment and Section core data objects in the IMS QTI ASI: Information Model and IMS QTI ASI: XML Binding;
• The 'Outcomes Processing' component of the IMS QTI specification interacts with the 'Selection & Ordering' part. This interaction is a result of the fact that the aggregated score for a Section and/or Assessment depends upon which Items and Section were selected for presentation to the participant. The manner of this interaction is addressed in the "Outcomes Processing' specification [QTI, 02d];
• The IMS QTI Results Reporting specifications contain the results recorded for a participant that has undertaken an evaluation. These results will depend upon the order in which the Items and Sections are presented however there is no direct linkage between the 'Results Reporting' and the 'Selection & Ordering' specifications.

4.3 Basic Example XML Instances

The following set of examples demonstrates how the different use-cases are supported. Table 4.1 lists the examples used to show each selection and ordering algorithm and Table 4.2 lists the examples that support each use-case.

Algorithm	Example
	1	2	3	4	5	6	7	8	9	10
Sequencing
Normal	*	*	*	*	*	*	*	*		*
RandomRepeat									*
Selection
All	*	*	*		*
Parameterized All						*
Partial				*	*					*
Parameterized Partial							*	*
Ordering
Sequential	*		*		*
Random		*	*	*	*	*	*	*	*	*

Use-case
	1	2	3	4	5	6	7	8	9	10
(a) Simple selection	*							*
(b) Basic random selection		*		*	*			*	*
I Topic random selection						*	*
(d) Golden questions			*		*
(e) Topic mixing						*	*
(f) Testlet										*

4.3.1 Example (1)

This is an example where "All Items are defined and are displayed in the order in which they are defined". This is the default condition and is equivalent to the <selection_ordering> element being excluded from an instance. Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_01"> <selection_ordering sequence_type="Normal"> <selection/> <order order_type="Sequential"> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ...

51 52 53 54 55 56 57 58

</item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample01.xml'. The sequence of Items presented to the participant will be:

1. IMS_QTIV1p2_I_SAO_01
2. IMS_QTIV1p2_I_SAO_02
3. IMS_QTIV1p2_I_SAO_03
4. IMS_QTIV1p2_I_SAO_04
5. IMS_QTIV1p2_I_SAO_05
6. IMS_QTIV1p2_I_SAO_06
7. IMS_QTIV1p2_I_SAO_07
8. IMS_QTIV1p2_I_SAO_08
9. IMS_QTIV1p2_I_SAO_09
10. IMS_QTIV1p2_I_SAO_10

4.3.2 Example (2)

This is an example where "All Items are defined but are displayed in random order and the random order is different each time". Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_02"> <selection_ordering> <selection/> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample02.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_03
2. IMS_QTIV1p2_I_SAO_07
3. IMS_QTIV1p2_I_SAO_08
4. IMS_QTIV1p2_I_SAO_04
5. IMS_QTIV1p2_I_SAO_02
6. IMS_QTIV1p2_I_SAO_05
7. IMS_QTIV1p2_I_SAO_03
8. IMS_QTIV1p2_I_SAO_06
9. IMS_QTIV1p2_I_SAO_09
10. IMS_QTIV1p2_I_SAO_01

4.3.3 Example (3)

This is an example where "All Items are defined but are displayed in random order with the exception of one Item which is always displayed first and another which is always displayed last". Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_03"> <selection_ordering> <selection/> <order order_type="Sequential"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <section ident="IMS_QTIV1p2_S_SAO_03p1"> <selection_ordering> <selection/> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05">

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> </section> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample03.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_01      (fixed position)
2. IMS_QTIV1p2_I_SAO_03
3. IMS_QTIV1p2_I_SAO_07
4. IMS_QTIV1p2_I_SAO_04
5. IMS_QTIV1p2_I_SAO_09
6. IMS_QTIV1p2_I_SAO_06
7. IMS_QTIV1p2_I_SAO_05
8. IMS_QTIV1p2_I_SAO_08
9. IMS_QTIV1p2_I_SAO_02
10. IMS_QTIV1p2_I_SAO_10      (fixed position)

4.3.4 Example (4)

This is an example where "A set number of Items are selected at random from a larger number, e.g. 4 Items from 10. The random selection is different each time and the order in which they are displayed is normally randomized". Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_03"> <selection_ordering> <selection> <selection_number>4</selection_number> </selection> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample04.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_10
2. IMS_QTIV1p2_I_SAO_03
3. IMS_QTIV1p2_I_SAO_07
4. IMS_QTIV1p2_I_SAO_04

4.3.5 Example (5)

This is an example where "A specified Item is displayed followed by 3 Items selected at random from 8 and in random order followed by another specified Item". Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_10"> <selection_ordering> <selection/> <order order_type="Sequential"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <section ident="IMS_QTIV1p2_S_SAO_10p1"> <selection_ordering> <selection> <selection_number>3<selection_number> </selection> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ...

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample05.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_01 (fixed position)
2. IMS_QTIV1p2_I_SAO_07
3. IMS_QTIV1p2_I_SAO_04
4. IMS_QTIV1p2_I_SAO_06
5. IMS_QTIV1p2_I_SAO_10 (fixed position)

4.3.6 Example (6)

This is an example where "Items are selected according to specified attributes e.g. 'topic=qtiv1p2test' and 'difficulty=basic'. All Items with these attributes are selected and presented randomly". Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15	<questestinterop> <section ident="IMS_QTIV1p2_I_SAO_06"> <selection_ordering> <selection> <and_selection> <selection_metadata mdname="qmd_topic" mdoperator="EQ">qtiv1p2test </selection_metadata> <selection_metadata mdname="qmd_levelofdifficulty" mdoperator="EQ">basic </selection_metadata> </and_selection> </selection> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01">
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75	<itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry>
76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135	</qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> <itemmetadata>
136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186	<qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample06.xml'. The sequence of Items presented to the participant could be

1. IMS_QTIV1p2_I_SAO_03
2. IMS_QTIV1p2_I_SAO_07
3. IMS_QTIV1p2_I_SAO_06
4. IMS_QTIV1p2_I_SAO_05
5. IMS_QTIV1p2_I_SAO_08
6. IMS_QTIV1p2_I_SAO_02

4.3.7 Example (7)

This is an example where "Items are selected according to their difficulty 'difficulty=basic' and subject 'subject=algebra'. A specified number of these Items (4) are selected randomly and presented in order of selection. Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45	<questestinterop> <section ident="IMS_QTIV1p2_I_SAO_07"> <selection_ordering> <selection> <selection_number>4</selection_number> <and_selection> <selection_metadata mdname="qmd_topic" mdoperator="EQ">qtiv1p2test </selection_metadata> <selection_metadata mdname="qmd_levelofdifficulty" mdoperator="EQ">basic </selection_metadata> </and_selection> </selection> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata>
46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105	... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel>
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165	<fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ...
166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187	abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>algebra</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample07.xml'. The sequence of Items presented to the participant could be

1. IMS_QTIV1p2_I_SAO_07
2. IMS_QTIV1p2_I_SAO_06
3. IMS_QTIV1p2_I_SAO_08
4. IMS_QTIV1p2_I_SAO_02

4.3.8 Example (8)

This is an example where the number of Items selected is constrained using the parameter 'maxscorepossible'. For the sake of this example we are assuming that this parameter is used by an algorithm (this is proprietary to the assessment engine) that chooses the Items to ensure that the maximum possible score that can be attained is '8'. Each Item is shown only once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60	<questestinterop> <section ident="IMS_QTIV1p2_I_SAO_08"> <selection_ordering> <sequence_parameter pname="maxscorepossible">4</sequence_parameter> <selection/> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> <itemmetadata> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04">
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120	<itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry>
121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180	</qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample08.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_07
2. IMS_QTIV1p2_I_SAO_06
3. IMS_QTIV1p2_I_SAO_08
4. IMS_QTIV1p2_I_SAO_02
5. IMS_QTIV1p2_I_SAO_09
6. IMS_QTIV1p2_I_SAO_10
7. IMS_QTIV1p2_I_SAO_03
8. IMS_QTIV1p2_I_SAO_04

4.3.9 Example (9)

This is an example where "All Items are defined but are displayed in random order and the random order is different each time". Each Item may be presented more than once.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

<questestinterop> <section ident="IMS_QTIV1p2_S_SAO_02"> <selection_ordering sequence_type="Repeat"> <sequence_parameter pname="totalnumberobjects">10</sequence_parameter> <selection/> <order order_type="Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample09.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_03
2. IMS_QTIV1p2_I_SAO_07
3. IMS_QTIV1p2_I_SAO_08
4. IMS_QTIV1p2_I_SAO_04
5. IMS_QTIV1p2_I_SAO_02
6. IMS_QTIV1p2_I_SAO_08
7. IMS_QTIV1p2_I_SAO_03
8. IMS_QTIV1p2_I_SAO_06
9. IMS_QTIV1p2_I_SAO_03
10. IMS_QTIV1p2_I_SAO_01

4.3.10 Example (10)

In this example a testlet consists of ten Items within a single Section. The <presentation_material> element within the Section contains the common content that must be used for all ten Items. Only six of the Items are presented to the participant.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

<questestinterop> <section ident = "IMS_QTIV1p2_S_SAO_10"> <presentation_material> <flow_mat> <flow_mat> <material> <matimage uri="periodictable.jpg" x0="100" y0="100" height="400" width="600"/> </material> </flow_mat> <flow_mat> <material> <mattext>Answer the following questions on the Periodic Table shown above. </mattext> </material> </flow_mat> </flow_mat> </presentation_material> <selection_ordering> <selection> <selection_number>6</selection_number> </selection> <order order_type = "Random"/> </selection_ordering> <item ident="IMS_QTIV1p2_I_SAO_01"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_02"> ...

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77

abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_03"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_04"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_05"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_06"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_07"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_08"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_09"> ... abbreviated example ... </item> <item ident="IMS_QTIV1p2_I_SAO_10"> ... abbreviated example ... </item> </section> <questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/basicsaoexample10.xml'. The sequence of Items presented to the participant could be:

1. IMS_QTIV1p2_I_SAO_07
2. IMS_QTIV1p2_I_SAO_08
3. IMS_QTIV1p2_I_SAO_02
4. IMS_QTIV1p2_I_SAO_05
5. IMS_QTIV1p2_I_SAO_03
6. IMS_QTIV1p2_I_SAO_01

4.4 Advanced Example XML Instances

4.4.1 Example (1)

This is an example where the set of Items are distributed within four Sections that are themselves structured in two hierarchical groups.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50	<questestinterop> <section ident = "IMS_QTIV1p2_S_SAO_11"> <selection_ordering> <selection> <selection_number>1</selection_number> </selection> <order order_type = "Sequential"/> </selection_ordering> <section ident= "IMS_QTIV1p2_S_SAO_11p1"> <qticomment>Set of True/false questions</qticomment> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_09" maxattempts = "2"> <itemmetadata>
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110	<qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <section ident = "IMS_QTIV1p2_S_SAO_11p2"> <qticomment>Set of multiple-choice/multiple-response questions</qticomment> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield>
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170	<fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_QTILiveTalkv1_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata>
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230	... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> </section> <section ident = "IMS_QTIV1p2_S_SAO_12"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <section ident = "IMS_QTIV1p2_S_SAO_12p1"> <qticomment>Set of True/false questions</qticomment> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel>
231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290	<fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <section ident = "IMS_QTIV1p2_S_SAO_12p2"> <selection_ordering> <sequence_parameter pname = "maxscorepossible">8</sequence_parameter> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield>
291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350	</qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_QTILiveTalkv1_06" maxattempts = "2"> <itemmetadata> <qtimetadata>
351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384	<qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> </section> </questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/advsaoexample01.xml'. The sequence of Items presented to the participant from Section 'IMS_QTIV1p2_S_SAO_11' could be:

Section IMS_QTIV1p2_S_SAO_11p1

1. IMS_QTIV1p2_I_SAO_08
2. IMS_QTIV1p2_I_SAO_02
3. IMS_QTIV1p2_I_SAO_09
4. IMS_QTIV1p2_I_SAO_10

or

Section IMS_QTIV1p2_S_SAO_11p2

1. IMS_QTIV1p2_I_SAO_04
2. IMS_QTIV1p2_I_SAO_07
3. IMS_QTIV1p2_I_SAO_01
4. IMS_QTIV1p2_I_SAO_06
5. IMS_QTIV1p2_I_SAO_03
6. IMS_QTIV1p2_I_SAO_05

4.4.2 Example (2)

This is an example where the set of Items are distributed within four Sections that are themselves structured in two hierarchical groups within the Assessment.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52	<questestinterop> <assessment ident="IMS_QTIV1p2_A_SAO_01"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <section ident = "IMS_QTIV1p2_S_SAO_11"> <selection_ordering> <selection> <selection_number>1</selection_number> </selection> <order order_type = "Sequential"/> </selection_ordering> <section ident= "IMS_QTIV1p2_S_SAO_11p1"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item>
53 54 54 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111	<item ident = "IMS_QTIV1p2_I_SAO_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <section ident = "IMS_QTIV1p2_S_SAO_11p2"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_03" maxattempts = "2"> <itemmetadata>
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170	<qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_QTILiveTalkv1_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry>
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229	</qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> </section> <section ident = "IMS_QTIV1p2_S_SAO_12"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <section ident = "IMS_QTIV1p2_S_SAO_12p1"> <selection_ordering> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry>
230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288	</qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> <section ident = "IMS_QTIV1p2_S_SAO_12p2"> <selection_ordering> <sequence_parameter pname = "maxscorepossible">8</sequence_parameter> <selection/> <order order_type = "Random"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_SAO_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield>
289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347	<qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>advanced</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example
348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387	... </item> <item ident = "IMS_QTIV1p2_I_QTILiveTalkv1_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> <item ident = "IMS_QTIV1p2_I_SAO_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_levelofdifficulty</fieldlabel> <fieldentry>basic</fieldentry> </qtimetadatafield> <qtimetadatafield> <fieldlabel>qmd_topic</fieldlabel> <fieldentry>qtiv1p2test</fieldentry> </qtimetadatafield> </qtimetadata> </itemmetadata> ... abbreviated example ... </item> </section> </section> </assessment> </questestinterop>

This example is available in the file: 'ims_qtiasiv1p2/selection/advsaoexample03.xml'. The sequence of Items presented to the participant could be:

Section IMS_QTIV1p2_S_SAO_12

Section IMS_QTIV1p2_S_SAO_12p1

     1.  IMS_QTIV1p2_I_SAO_02

     2.  IMS_QTIV1p2_I_SAO_09

     3.  IMS_QTIV1p2_I_SAO_10

     4.  IMS_QTIV1p2_I_SAO_18

Section IMS_QTIV1p2_S_SAO_12p2

     5.  IMS_QTIV1p2_I_SAO_03

     6.  IMS_QTIV1p2_I_SAO_01

Section IMS_QTIV1p2_S_SAO_11

Section IMS_QTIV1p2_S_SAO_11p1

     7.  IMS_QTIV1p2_I_SAO_08

     8.  IMS_QTIV1p2_I_SAO_02

     9.  IMS_QTIV1p2_I_SAO_09

     10.  IMS_QTIV1p2_I_SAO_10

4.5 Implementation Guidance

4.5.1 Section & Item Sequencing

Currently there are only two sequencing rules but more will be added once support for adaptive-based sequencing is added. Repeated object selection does not necessarily result in the repetitive selection of Items - the Items may all be encapsulated in a set of hierarchical Sections.

4.5.2 Section & Item Selection

The selection of Sections and Items is based upon the same mechanism, namely the <selection> element. Sections and Items are treated in the same manner i.e. they are objects that may or may not be selected. Multiple selection rules can be applied to the pool of objects and these rules are applied only to the immediate set of children contained within the parent object. It is the parent object that contains the <selection_ordering> instructions for the child objects. The order in which the objects are selected is important but the default order is determined by the order in which the objects are defined in the corresponding ASI structure. The parent-constrained selection mechanism creates a pool of child objects that are to be presented to the participant. The selection rules are additive both in terms of within each selection structure but also across multiple selection structures. The selection rules may be applied to an object-bank that is identified by external reference i.e. the Assessment and/or Section may not actually contain the Sections and/or Items to be used.

4.5.3 Section & Item Ordering

At the current time the ordering of the selected objects is restricted to two mechanisms:

• Sequential - the presentation in the order in which the objects were selected or occur within the ASI structure. If a subset of objects has been selected then the order in which they are chosen is used otherwise their order within the ASI structure is used;
• Random - a randomization of the order in which the selected objects are defined within the relevant ASI structures. This assumes that all of the objects are placed within a single pool (multiple occurrences is not possible) from which an object is randomly selected. Once an object has been selected for presentation i.e. its relative order defined, then the object is either removed from the poll (sequencing defined as 'Normal') or it is returned to allow multiple presentation (sequencing defined as 'Repeat').

Complex ordering of objects is achieved by their appropriate grouping in Sections and the interleaving of those Sections with Items. It should be noted that the macro order of the full set of objects follows the hierarchical structure of the objects i.e. there is no interleaving of child objects from different parent objects. This form of interleaving is only accomplished by referencing the same child object in more than one parent object.

4.5.4 Using Meta-data

The meta-data of the objects is an important selection mechanism. This selection must operate on the IMS QTI-specific meta-data and the IMS Meta-data definitions. There are three meta-data entry techniques used within IMS QTI:

• For the explicitly identified IMS QTI-specific meta-data fields (the elements named using the 'qmd_**...***' scheme) the name of the meta-data element is used as the string name in the 'mdname' attribute of the <selection_metadata> element. The actual meta-data field value is that content of the <qmd_***...***> element. This approach should not be used as this form of IMS QTI-specific meta-data has been deprecated in favour of the vocabulary-based mechanism;
• For the IMS QTI-specific meta-data fields named using the generic vocabulary mechanism (using the element <qtimetadatafield>) the string name will be that identified using the <qtimetadatafield><fieldlabel> structure. The actual meta-data field value is the content of the structure <qtimetadatafield><fieldentry>;
• For the IMS Meta-data entries the string naming convention will adopt the '.' mechanism to denote the concatenation of nested tags e.g. the string "lom:metametadata.contribute.role.source" is used to denote the IMS Meta-data structure of '<metametadata><contribute><role><source>'. The 'lom:' namespace prefix should also be used.

The logic rules for the construction of complex meta-data selection rules are given in Appendix B.

4.6 Example XML Instances

The full set of example files, as referred to in Sections 4.3 and 4.4 are available as part of the Q7TI Resource Kit. The set of example XML instances are listed in Table 4.3.

Filename	Nature	Description
basicsaoexample01.xml	S(1)I(10)	All Items are selected and they are presented in the order defined in the ASI structure (also the default condition). Each object can be presented only once.
basicsaoexample02.xml	S(1)I(10)	All Items are selected and they are presented in random order (the order will differ each time selection occurs). Each object can be presented only once.
basicsaoexample03.xml	S(1){I(1)[S(1)I(8)]I(1)}	All Items are selected and they are presented in random order with the exception of the first and last Items that are fixed in order. Each object can be presented only once.
basicsaoexample04.xml	S(1)I(10)	A predefined subset of the Items are selected and they are presented in random order. Each object can be presented only once.
basicsaoexample05.xml	S(1){I(1)[S(1)I(8)]I(1)}	A subset of Items are selected and they are presented in random order with the exception of the first and last Items that are fixed in order and which are always selected. Each object can be presented only once.
basicsaoexample06.xml	S(1)I(10)	All Items that have a particular set of meta-data properties are selected and they are presented in random order. Each object can be presented only once.
basicsaoexample07.xml	S(1)I(10)	A subset of all of the Items that have a particular set of meta-data properties are selected and they are presented in random order. Each object can be presented only once.
basicsaoexample08.xml	S(1)I(10)	A subset of the Items are selected such that they obey a proprietary algorithm that receives a particular parameter unique to the algorithm. Each object can be presented only once.
basicsaoexample09.xml	S(1)I(10)	All Items are selected and they are presented in random order (the order will differ each time selection occurs) but multiple presentations of an Item are permitted.
basicsaoexample10.xml	S(1)I(10)	A testlet that consist of 10 Items all of which use a common stimulus. Only a subset (6) of the Items are to be presented and no Item is to be presented more than once.
advsaoexample01.xml	S(2)[S(1)I(4) + S(1)I(6)]	Two Sections each of which contains two other Sections. Each Section supports its own selection and ordering instruction.
advsaoexample02.xml	S(2)[S(1)I(4) + S(1)I(6)]	Two Sections each of which contains two other Sections. Each Section supports its own selection and ordering instruction.
advsaoexample03.xml	A(1){S(2)[S(1)I(4) + S(1)I(6)]}	An Assessment that contains two Sections each of which contains two other Sections. The Assessment and each Section support their own selection and ordering instructions.
advsaoexample04.xml	A(1){S(2)[S(1)I(4) + S(1)I(6)]}	An Assessment that contains two Sections each of which contains two other Sections. The Assessment and each Section support their own selection and ordering instructions.

4.7 Proprietary Extensions

The proprietary extensions facilities listed in Table 4.4 are supported as elements within the specifications:

Extension Element Name	Host Element	Description
selection_extension	selection	Inclusion of alternative selection instructions.
order_extension	order	Inclusion of alternative ordering instructions.

Note: These elements are only used if the suppliers of the ASIs require proprietary features that are not supported by the available range of elements. It is recommended that extensions are used sparingly. Backwards compatibility with proprietary extensions will NOT be guaranteed in future versions of these specifications. Proprietary sequencing, selection and ordering rules should be supported using the alternative mechanisms that use the explicit parameterization mechanisms without using the extension elements listed in Table 4.4.

4.8 V1.x/V2.0 Issues & Compatibility

The 'Selection & Ordering' functionality is not backwards compatible with early versions of the specification. These functions that have been replaced in earlier specifications are listed in Table 4.5.

Previous Element Name	New Element Name	Description
sectionselection	selection	These elements are used to replace the Section selection within an Assessment and Section.
sectionsequence	order
itemselection	selection	These elements are used to replace the Item selection within a Section.
itemsequence	order

V1.0, V1.01 and V1.1 QTI-XML instances that do not use the elements listed within Table 4.5 are backwards compatible because the <selection_ordering> element within V1.2 is optional.

4.9 IMS Harmonization

The issues of harmonization of this part of the IMS QTI specification with other IMS specifications is addressed by:

• IMS Persistent Location-independent Resource Identifier - all of the identifiers should conform to this IMS global labelling recommendation;
• IMS Meta-data - the <selection_metadata> element should be capable of operating on any form of the entries defined within the IMS Meta-data specification;
• IMS Sequencing - harmonization with this specification will be undertaken once it has been competed and accepted by the IMS Technical Board.

Appendix A - Glossary of Terms

and_selection	The and_selection element is used to create a complex logical rule. In this rule all of the clauses must be true before the object can be selected. This construct is commonly used to identify the set of meta-data values that must be true before the object can be selected. Appendix B contains the logic rules associated with this element.
mdname	The mdname attribute is used with the selection_metadata element to identify the meta-data field that is to be used. The meta-data field may be contained within either the QTI-specific fields or the externally linked IMS meta-data field of the object. It is the responsibility of the host system to determine the existence or otherwise of the meta-data field.
mdoperator	The mdoperator attribute is used with the selection_metadata element to identify the comparison operation to be applied on the meta-data field. The set of possible operators is defined as and enumerated set and consists of: EQ, NEQ, LT, LTE, GT, GTE. This set is derived from the operators normally supported by database access languages.
not_selection	The not_selection element is used to create a complex logical selection rule. This construct is commonly used to identify the set of objects that must not be included due to the defined meta-data values. Appendix B contains the logic rules associated with this element.
or_selection	The or_selection element is used to create a complex logical rule. In this rule if any of the clauses is true then the object is selected. This construct is commonly used to identify the set of meta-data values that may be true before the object can be selected. Appendix B contains the logic rules associated with this element.
order	This element contains the rules for the presentation order of the objects previously selected. The nature of the ordering is defined using the order_type attribute.
order_extension	This element is the extension facility for the order element. It is used to support the definition of ordering selection rules. Any new ordering rule is exclusive with respect to the other rules i.e. only one rule can be used.
order_type	This attribute is used on the order element to define the nature of the ordering to be applied by the assessment engine. The possible values are 'Sequential' and 'Random'. The 'Sequential' value orders the selected objects as they are contained within the parent object whereas the 'Random' value ensures that the order is random.
pname	The pname attribute is used by the sequence_parameter element to define the name of the associated parameterized proprietary selection rule. The content of the sequence_parameter element is used as the data for the named parameterized selection.
qticomment	This is the commenting facility within the XML schemas. The comments can take any form supported as #PCDATA. The key difference between this comment style and the standard '<!--***‡' is that the former is passed through the XML parser to the host system.
selection	This element is used to contain all of the selection rules that are to be applied to the set of child objects contained in the parent. All of the child objects are treated as equivalent. All of the contained rules are additive i.e. each rule adds more objects to the pool already selected. Multiple selection elements are also treated additively i.e. the final pool of selected objects is the sum of the application of all the rules within all of the selection elements.
selection_extension	This element is the extension facility for the selection element. It is used to support the definition of proprietary selection rules. Any new selection rule is additive with respect to the other rules i.e. each rule can add more objects to the pool of objects selected.
selection_metadata	The selection_metadata element is used to contain the rules that are applied to the meta-data fields of the object. This element uses the mdname and mdoperator attributes to define the meta-data label and the type of comparison respectively. Complex combinations of the meta-data values can be constructed using the and_selection, or_selection and not_selection elements.
selection_number	This element contains the part of the selection rule that defines the number of objects to be selected randomly from the set of objects available. When used in conjunction with the selection_metadata element then each of the objects selected must also have the described meta-data properties.
selection_ordering	This element is used to contain all of the selection and ordering instructions that are to be applied to the host Assessment or Section structure. All of the selection and ordering instructions are contained in a single instance of the element and these instructions are only applied to the immediate children of the parent object. The sequence_type attribute is used to define the overall sequencing algorithm for the selection and ordering.
sequence_parameter	The sequence_parameter element is used to define proprietary parameterized selection rules. The name of the rule is defined by the pname attribute and the data for the rule is passed within the element's content. The significance of the parameter name and the data content is host dependent.
sequence_type	The sequence_type attribute is used by the selection_ordering element to define the nature of the overall sequencing algorithm. At the current time there are two possible values, namely 'Normal' and 'Repeat'. The 'Repeat' value allows each object to be selected more than once whereas the value 'Normal' imposes a single occurrence for any selected object.
sourcebank_ref	The sourcebank_ref element is used to identify the object-bank upon which the selection rules are to be applied. The selection rules are applied to the full object-bank identified.

Appendix B - Logic Rules

The elements or_selection, and_selection and not_selection are used within the selection element and by each other to control the ways in which the selection rules are combined with each other. A selection rule takes the form such as:

<selection_metadata mdname="qmd_timelimit" mdoperator="LTE">5</selection_metadata>

i.e. select all of the objects that have a meta-data entry of "qmd_timelimit" that is less than or equal to 5.

The selection element will contain a set of rules, i.e. R1, R2, R3...Rk. The logic operators between these rules are defined as:

• ¬R1 means 'NOT R1' which is represented in QTIASI-XML as <not_selection>R1</not_selection>
• R1 and R2 means 'R1 AND R2' which is represented in QTIASI-XML as <and_selection>R1, R2</and_selection>
• R1 or R2 means 'R1 OR R2' which is represented in QTIASI-XML as <or_selection>R1, R2</or_selection>

Using this initial set of constructions gives rise to the following representations:

• ¬ (R1 OR R2) which is represented in QTIASI-XML as

  <not_selection>
    <or_selection>
        R1
        R2
    </or_selection>
  </not_selection>

• ¬ (R1 AND R2) which is represented in QTIASI-XML as

  <not_selection>
    <and_selection>
        R1
        R2
    </and_selection>
  </not_selection>

• (¬R1 AND ¬R2) which is represented in QTIASI-XML as

  <and_selection>
    <not_selection>R1</not_selection  >    
    <not_selection>R2</not_selection  >
  </and_selection>

• (¬R1 OR ¬R2) which is represented in QTIASI-XML as

  <or_selection>
    <not_selection>R1</not_selection  >    
    <not_selection>R2</not_selection  >
  </or_selection>

• ((R1 OR R2) AND (R3 OR R4)) which is represented in QTIASI-XML as

  <and_selection>
    <or_selection>
        R1
        R2
    </or_selection>
    <or_selection>
        R3
        R4
    </or_selection>
  </and_selection>

• ((R1 OR R2 OR R3) AND (R4 OR R5 OR (R6 AND R7 AND R8)) AND ¬(R9 OR R10 OR R11)) which is represented in QTIASI-XML as

  <and_selection>
    <or_selection>
        R1
        R2
        R3
    </or_selection>
    <or_selection>
        R4
        R5 
      <and_selection>
          R6
          R7
          R8
      </and_selection>
    </or_selection>
    <or_selection>
      <not_selection>
        <or_selection>
            R9
            R10
            R11
        </or_selection>
        </not-selection>
    </or_selection>
  </and_selection>

This set of rules gives rise to the following guidelines:

• Only a single <and>, <or> or <not> structure is required at the top-most level of the logic statement;
• Multiple occurrences of the <and>, <or> and <not> elements is permitted within <and> and <or> elements to allow the construction of complex logic statements;
• The <not> element can only contain a single element the result of which is to invert the logic of the test.

About This Document

Title	IMS Question & Test Interoperability ASI Selection & Ordering
Editors	Colin Smythe, Eric Shepherd, Lane Brewer, and Steve Lay
Version	1.2
Version Date	11 February 2002
Status	Final Specification
Summary	This document presents the IMS QTI ASI Selection & Ordering Specification. This specification is one of the set of the IMS Question & Test Interoperability specifications. This Selection & Ordering specification is an extension to the IMS QTI ASI V1.2 specification.
Revision Information	22 January 2002
Purpose	Defines the selection and ordering features that have been added to IMS QTI V1.1 as part of the development of the IMS QTI V1.2 specification.
Document Location	http://www.imsglobal.org/question/v1p2/imsqti_asi_saov1p2.html

List of Contributors

The following individuals contributed to the development of this document:

Russell Almond	ETS, USA
Lane Brewer	Galton Technologies Inc.
Todd Brewer	Galton Technologies Inc.
Russell Grocott	Can Studios Ltd.
Andy Heath	CETIS/JISC, UK
Paul Hilton	Can Studios Ltd.
Steven Lay	University of Cambridge Local Examinations Syndicate, UK
Jez Lord	Can Studios Ltd.
John Kleeman	Question Mark Computing Ltd.
Paul Roberts	Question Mark Computing Ltd.
Nial Sclater	CETIS/University of Strathclyde, UK
Colin Smythe	Dunelm Services Ltd.

Revision History

Version No.	Release Date	Comments
Final Version 1.2	11 February 2002	The first release of the IMS QTI ASI Selection & Ordering final specification.

Index

A
ASI 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
Assessment 1, 2, 3, 4, 5, 6, 7, 8, 9
Assessment Elements
     assessment 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Attributes
     case 1, 2
     columns 1
     height 1
     ident 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37
     label 1
     maxattempts 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
     mdname 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     mdoperator 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
     order_type 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23
     pname 1, 2, 3, 4, 5, 6, 7, 8, 9
     sequence_type 1, 2, 3, 4, 5, 6
     uri 1
     width 1
     x0 1
     xmllang 1, 2
     y0 1

B
Basic 1, 2

C
Common Elements
     fieldentry 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     flow_mat 1
     material 1
     matimage 1
     mattext 1
     order 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39
     other 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     outcomes 1
     presentation_material 1
     qticomment 1, 2, 3, 4, 5, 6
     qtimetadata 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     qtimetadatafield 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     reference 1, 2, 3
     selection 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55
     selection_ordering 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
     vocabulary 1

E
Elements
     and_selection 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
     assessment 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     fieldentry 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     fieldlabel 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     flow 1
     flow_mat 1
     itemmetadata 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24
     material 1
     matimage 1
     mattext 1
     not_selection 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13
     objectbank 1, 2, 3
     or_selection 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
     order 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39
     order_extension 1, 2, 3, 4
     other 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     outcomes 1
     presentation 1, 2, 3, 4, 5, 6, 7, 8
     presentation_material 1
     qmd_levelofdifficulty 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     qmd_timelimit 1, 2
     qmd_topic 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28
     qmd_weighting 1
     qticomment 1, 2, 3, 4, 5, 6
     qtimetadata 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     qtimetadatafield 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
     questestinterop 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19
     reference 1, 2, 3
     section 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29
     selection 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55
     selection_extension 1, 2, 3, 4, 5
     selection_metadata 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
     selection_number 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     selection_ordering 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
     sequence_parameter 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
     solution 1
     sourcebank_ref 1, 2, 3
     vocabulary 1
Extension Elements
     order_extension 1, 2, 3, 4
     selection_extension 1, 2, 3, 4, 5