IMS Question & Test Interoperability: ASI Outcomes Processing Final Specification Version 1.2

Table of Contents

1. Introduction

1.1 Question & Test Interoperability Overview

The IMS Question & Test Interoperability (QTI) specification describes a basic structure for the representation of question (item) and test (assessment) data and their corresponding results reports [QTI, 02i]. Therefore, the specification enables the exchange of this item, assessment and results data between Learning Management Systems, as well as content authors and, content libraries and collections. The IMS QTI specification is defined in XML to promote the widest possible adoption. XML is a powerful, flexible, industry standard mark-up language used to encode data models for Internet-enabled and distributed applications. The QTI specification is extensible and customizable to permit immediate adoption, even in specialized or proprietary systems. Leading suppliers and consumers of learning products, services and content contributed time and expertise to produce this final specification. The IMS QTI specification, like all IMS specifications, does not limit product designs by specifying user interfaces, pedagogical paradigms, or establishing technology or policies that constrain innovation, interoperability, or reuse.

The 'Outcomes Processing' specification contains the description of how the aggregated scores at the Assessment and Section levels can be derived. These scoring outcomes are based upon the child Sections and/or Items. A wide range of scoring algorithms is supported through the usage of a predefined set of parameterized instructions; these avoid the realization of the algorithms within the XML. This document contains the relevant information model, XML binding and best practices guidance but it should be read in the context of the core ASI documents.

1.2 Scope & Context

This document is the IMS Question & Test Interoperability ASI Outcomes Processing Specification and must be read in conjunction with the core documents:

• IMS QTI: ASI Information Model [QTI, 02a];
• IMS QTI: ASI XML Binding [QTI, 02b];
• IMS QTI: ASI Best Practice & Implementation Guide [QTI, 02c].

It defines the outcomes processing features that are to be applied to the Section and/or Assessments. These new Assessment and Section features are not backwards compatible with the other versions of the IMS QTI specifications. These new features have no effect on the IMS QTILite specification [QTI, 02h] because that refers only to the Item data structure.

1.3 Structure of this Document

The structure of this document is:

2. Information Model	The description of the information model of the outcomes processing component for the full IMS QTI ASI;
3. XML Binding	The description of the XML binding of the outcomes processing component for the full IMS QTI ASI;
4. Best Practice & Implementation Guide	The description of the best practices and implementation guide for the outcomes processing component of the full IMS QTI ASI;
Appendix A - Glossary of Terms	A brief description of all of the elements and attributes that have been used to support outcomes processing;
Appendix B - In-Built Scoring Algorithms	A description of the operation of each of the default scoring algorithms;
Appendix C - Logic Rules	The logic rules that are supported by the or_object, and_object and not_object elements.

1.4 Nomenclature

API	Application Programming Interface
ASI	Assessment, Section, Item
CBT	Computer Based Training
DTD	Document Type Definition
QTI	Question & Test Interoperability
VLE	Virtual Learning Environment
W3C	World Wide Web Consortium
XML	Extensible Mark-up Language

1.5 References

[IMS, 01]	IMS Persistent, Location-Independent Resource Identifier Implementation Handbook, M.McKell, Version 1.0, IMS, April 2001.
[QTI, 02a]	IMS Question & Test Interoperability: ASI Information Model Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02b]	IMS Question & Test Interoperability: ASI XML Binding Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02c]	IMS Question & Test Interoperability: ASI Best Practice & Implementation Guide, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02d]	IMS Question & Test Interoperability: ASI Selection & Ordering Specification, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02e]	IMS Question & Test Interoperability: Results Reporting Information Model, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02f]	IMS Question & Test Interoperability: Results Reporting XML Binding, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02g]	IMS Question & Test Interoperability: Results Reporting Best Practice & Implementation Guide, C.Smythe, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02h]	IMS Question & Test Interoperability: QTILite Specification, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.
[QTI, 02i]	IMS Question & Test Interoperability: Overview, C.Smythe, E.Shepherd, L.Brewer and S.Lay, Final Specification, Version 1.2, IMS, February 2002.

2. Information Model

2.1 Exchanging ASI Objects

The possible advanced Assessment structures that can be exchanged using QTI are shown in Figure 2.1:

• The Assessment could contain a single Section block (Figure 2.1a);
• The Assessment could consist of a reference to an external Section block (Figure 2.1b);
• The Assessment could consist of a mixture of contained and externally referenced Section blocks (Figure 2.1c). There is no preferential order in the blocks and references.

(a)	(c)
(b)

In the case of Sections, some of the possible data structures that can be exchanged are shown in Figure 2.2:

• The Section could contain one or more Items (Figure 2.2a);
• The Section could contain one or more references to external Items (Figure 2.2b);
• The Section could contain one or more Sections (Figure 2.2c);
• The Section could contain one or more references to external Sections (Figure 2.2d);
• The Section could contain a mixture of blocks and references to Section and Items. There is no constraint on the order in which the Sections and Items are declared/referenced i.e. interleaving is supported.

(a)	(b)
(c)	(d)

(e)

The wide range of different data structures that can be exchanged is especially significant when developing mechanisms that support 'Outcomes Processing'. 'Outcomes Processing' is responsible for:

• The aggregation of scores assigned to the Items and/or Sections to create a score or set of scores for the parent Section. The ways in which this roll-up is achieved reflects the different scoring algorithms;
• The aggregation of scores assigned to Sections to create a score or set of scores for the parent Assessment. The ways in which this roll-up is achieved reflects the different scoring algorithms;
• Any assignment of scores based upon more than one Item and/or Section.

It is important to note that the type of scoring algorithm that is to be used is not mandated by the IMS specification. Instead, the specification is capable of supporting most of the available scoring algorithms. Also, the algorithms themselves are not encoded within the specification itself. Instead the type of algorithm is specified and the necessary parametric information supplied to enable the assessment engine to execute the corresponding algorithm.

2.2 Use Cases

The following representative use-cases have been identified for support within QTI ASI outcomes processing V1.2:

1. Multiple-choice quiz - this is a simple "end of chapter" quiz given by an LMS. The scores are reported back to the student for self-evaluation, so there is no particular need for high reliability. All of the questions are discrete (single response) questions whose outcomes are a dichotomous variable e.g. multiple-choice questions. For the sake of being definite we assume there are 10 Items and that we have a defined outcome, "CORRECT" for each Item. This takes on the value "True" if the response was correct and "False' if the response was not correct. All items are given equal weight. We wish to know, (1) the number of correct items, (2) the percentage of total score obtained by a candidate and (3) the percentage of items actually attempted which were correct;
2. True/false quiz - this is a simple "end of chapter" quiz given by an LMS. The scores are reported back to the student for self-evaluation, so there is no particular need for high reliability. All of the questions are discrete true/false. For the sake of being definite we assume there are 10 Items and that we have a defined outcome, "isCorrect" for each Item. This takes on the value '1' if the response was correct and "-1' if the response was not correct. All items are given equal weight. We wish to know, (1) the number of correct items, (2) the total score and (3) the weighted total score;
3. Multiple-response quiz - this is a simple "end of chapter" quiz given by an LMS. The scores are reported back to the student for self-evaluation, so there is no particular need for high reliability. All of the questions are multiple-response questions (two answers for each) with partial scores being awarded when at only one correct answer is given. For the sake of being definite we assume there are 5 Items and that we have a defined outcome, "SCORE" for each Item. This takes on the value "+1" for each correct response "-1' for each incorrect response. All items are given equal weight. We wish to know, (1) the number of correct items, (2) the total score and (3) the percentage of items actually attempted which were correct;
4. End of chapter test - this is the same as use-case (a) except that all of the items have different weights. We wish to see (1) the number of correct items, (2) the percentage of total weight answered correctly by the candidate and (3) the "weighted" percentage of questions attempted correctly of those questions answered;
5. English comprehension/composition test with essay - this is a placement test designed to assess English language ability. The test consists of two sections: (1) Short answer items (multiple-choice) and (2) the Essay. Short answer items come in two kinds. The first are 15 discrete vocabulary, grammar and usage Items that all produce a single dichotomous outcome "isCorrect". The second are the reading comprehension Items (although they may actually test other skills than just reading) that consist of a stimulus followed by five discrete questions. We assume that these are done as Items with five responses and hence five outcomes: these are Boolean variables whose meanings are as above. All of these Items can be given different weights (as can the 5 outcomes of the two reading comprehension items). The essay question is graded "A,B,C,D,E,F" by either a human or computer rater. The divisions do not necessarily represent equal spacings. The essay represents 1/3 of the total grade. Out of ten points, the mapping of the grades is as follows A=10, B=8, C=7, D=5, E=2, F=0. The total score for the assessment is required;
6. Diagnostic scoring - this is an additional score report which has been retrofitted onto the 'English comprehension test ((e) above). In addition to the overall score, "diagnostic" scores are going to be given in four different sub-areas: "Reading", "Writing", "Vocabulary" and "Grammar" (the dependencies among these sub-scores and the overall score are ignored for computational simplicity). The 15 discrete items and each of the 10 outcomes from the reading items all load onto these sub-scores differently. Some outcomes are completely irrelevant to some of the sub-scores. An Item/outcome can have a high "vocabulary" load and a low "grammar" load, or vice-versa. The essay is also added onto each of the sub-scores, but with a different loading;
7. Diagnostics complex response biology lite - this is a classroom assessment designed to assess the students understanding of the scientific method as it applies to Biology. The assessment returns not a single score but three scores reflecting "Domain Knowledge" (knowledge of the biological subjects in the assessment), "Methodological Knowledge" (understanding of the scientific method) and "Integrated Knowledge" (the ability to apply the scientific method to a given problem). The items are a series of complex "tasks". Each task gives a piece of scenario of a scientific investigation and has the students work through several steps of the process, answering questions about what they are doing at each step. Each task has many complex responses, and hence many observable outcomes. The outcomes are related to the number of steps in the task. Each is a variable given one of the values "Low", "Medium", or "High" depending on how the student did on one part of one step. The outcomes are labelled "OUT***" where '***' is a sequence number unique to the item. For more definite, assume that there are five tasks: Task1 has 7 outcomes, Task2 has 5, Task3 has 8, Task4 has 6 and Task5 has 13. For each of the reporting variables, the scoring model should add an appropriate "weight" if the outcome has reached an appropriate level (either "Medium" and above or "High"). The weight and the level will, in general, be different for each of the reporting variables. Not all outcomes are used for all of the reporting variables;

2.3 ASI Issues

The core features within the ASI structures that are related to the process of selection and ordering are shown in Figure 2.3.

The relationship of these features to outcomes processing are:

• <qtimetadata> - contains the QTI-specific meta-data about the object. This meta-data may be used to decide which objects are selected to support a particular aggregation mechanism;
• <outcomes_processing> - the set of aggregation processing instructions that are to be applied to the child objects. The variables to contain the aggregated scores are declared using the <outcomes> element;
• <feedback> - the feedback that is to be presented if the conditions defined within the <outcomes_processing> occur;
• <selection_ordering> - the selection and ordering rules that are to be applied to the set of child objects contained within the parent. Only those objects that are presented to the participant can contribute to the scoring algorithm but these may or may not be attempted by the participant;
• Internal and externally referenced Section and Item objects - the set of contained and referenced objects which can be selected and ordered;
• External metadata - the external meta-data descriptions that are linked to the object and which conform to the IMS Meta-data specification.

The scoping of the scoring algorithms that are supported is summarized using the example Section/Item combination shown in Figure 2.4. The scope of the scoring algorithm is:

• For Section 'Section Ident_1' the set of children objects is - 'Item Ident_1', 'Section Ident_2', 'Section Ident-3' and 'Item Ident_2'. This means that the scoring algorithm identified in Section 'Section Ident_1' is applied to the default variables associated with each child object;
• For Section 'Section Ident_2' the set of children objects is - 'Item Ident_2.1', 'Item Ident_2.2' and 'Item Ident_2.3'. The Section-level default variables are formed from the aggregation of these child objects;
• For Section 'Section Ident_3' the set of children objects is - 'Item Ident_3.1', 'Item Ident_3.2' and 'Item Ident_3.3'. The Section-level default variables are formed from the aggregation of these child objects;
• If for all of these three Sections only some of the objects are selected then the corresponding aggregation is based upon those objects presented.

It is the responsibility of the scoring engine that is implementing the scoring algorithm to maintain the scoping of the variables. If the default variable names are used then all of the variables will be named 'SCORE'. For instance, the scoping should ensure that:

Section_Ident_2.SCORE = Item_Ident_2.1.SCORE + Item_Ident_2.1.SCORE + Item_Ident_2.1.SCORE

At the level of the 'Section_Ident_1' then the variables 'Item_Ident_2.1.SCORE' etc. are only perceived through the variable 'Section_Ident_2.SCORE'.

2.4 Outcomes Processing Solution

The outcomes processing capability is based upon two complimentary mechanisms:

• In-built scoring algorithms - in which the appropriate scoring algorithm is just named and so the corresponding assessment engine is responsible for applying the algorithm to the default input and output variables;
• Defined proprietary algorithms - in which the relationship between the input and output scoring variables is defined using a parameterized approach.

2.4.1 The In-built Scoring Algorithms

The algorithms that are to be supported are:

1. Number of right answers (including multivariate responses) - this is identified by the algorithm name "NumberCorrect" and has associated with it the default integer variable 'COUNT'. This is the store for the number of correctly answered objects. An object is defined as correctly answered if the value assigned to the default Boolean variable 'COUNT' for the object is 'True'. The number correct algorithm can only operate on Boolean scoring variables. The default integer scoring variables COUNT.max (equal to the number of objects attempted), COUNT.min (set to zero) and COUNT.normalized (a value of between zero and one) are also available. The value of COUNT is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
2. Number of right answers normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'NumberCorrectAttempted';
3. Weighted number right (including multivariate responses) - this is similar to the 'number of rights answers' with the addition of the weighting of the individual components and is identified by the algorithm name "WeightedNumberCorrect". The same set of variables are used but their values reflect the object weighting as defined in the associated meta-data field 'qmd_weighting'. The value of COUNT is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
4. The weighted number of right answers normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'WeightedNumberCorrectAttempted';
5. Parameterized weighted number correct - name 'ParameterWeightedNumberCorrect' this is similar to the 'WeightedNumberCorrect' algorithm but the weighting value is passed using the <objects_parameter> element in preference to the meta-data field. The value of CORRECT is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
6. The parameterized weighted number of right answers normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'ParameterWeightedNumberCorrectAttempted';
7. Percentage correct (including multivariate responses) - this score is derived from both the "NumberCorrect" and "NumberCorrectAttempted" algorithms in which the percentage correct is defined by the equation-

1. Sum-of-scores - this is identified by the algorithm name "SumofScores" and has associated with it four default integer variables SCORE, SCORE.min, SCORE.max and SCORE.normalized. These variables store the sum of the scores, the minimum possible score (set to the sum of the 'minvalue' attributes for all of the corresponding objects), the maximum possible score (set to the sum of the 'maxvalue' attributes for all of the corresponding objects) and the normalized score (a value of between zero and one) respectively for the set of objects (the minimum, maximum and normalized values are only valid if they have been explicitly defined for all of the corresponding objects). The value of 'SCORE' is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
2. Sum of scores normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'SumofScoresAttempted';
3. Weighted sum of scores (including multivariate responses) - this is similar to the 'sum of scores' with the addition of the weighting of the individual components and is identified by the algorithm name "WeightedSumofScores". The same set of variables are used but their values reflect the object weighting as defined in the associated meta-data field 'qmd_weighting'. The value of SCORE is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
4. The weighted sum of scores normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'WeightedSumofScoresAttempted';
5. Parameterized weighted sum of scores - name 'ParameterWeightedSumofScores' this is similar to the 'WeightedSumofScores' algorithm but the weighting value is passed using the <objects_parameter> element in preference to the meta-data field. The value of SCORE is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
6. The parameterized weighted sum of scores normalized with respect to those objects that have been attempted and not just selected and presented. This algorithm is named 'ParameterWeightedSumofScoresAttempted';
7. Best 'K' out of 'N' (including multivariate responses) - this is identified by the algorithm name "BestKofN" and is used to calculate the outcome by using the highest 'K' scores from the set of presented objects generically defined as 'N' objects. The values are placed in the default variables 'SCORE', 'SCORE.min', 'SCORE.max' and 'SCORE.normalized'. The value of SCORE is determined with respect to the number of objects that have been selected and presented to the participant irrespective if an attempt is made to provide and answer;
8. Negative scores (including multivariate responses) - called 'GuessingPenalty'. Each object has three associated variables for the values of the number of correct answers ('COUNT.correct'), the number of incorrect answers ('COUNT.incorrect') and the number of unattempted answers ('COUNT.unattempted');
9. Item Response Theory (IRT) - supported using the processing parameter mechanism;
10. Time remaining - for further study in V2.0;
11. Par scores - for further study in V2.0;
12. Cut scores (pass/fail, right/wrong and including multivariate responses) - for further study in V2.0;
13. Multiple forms/equating (table based) - for further study in V2.0;
14. Partial credit/Graded response - for further study;
15. Critical Item - for further study in V2.0;
16. Critical Outcome - for further study in V2.0;
17. Factor analysis/ Multivariate IRT (research) -supported through extensions. For further study in V2.0;
18. Bayes net (research) - supported through extensions. For further study in V2.0;
19. Computer Adaptive Testing - for further study in V2.0.

2.4.2 Defined Proprietary Algorithms

Proprietary algorithms can be used but these must be characterized using the parameter definition mechanisms supported in the specification i.e. the <objects_parameter> and <processing_parameter> elements. Any number of parameters can be passed using these elements and there is no restriction on the naming convention used for these parameters - all parameters are specific to an algorithm. The names of the proprietary algorithms (passed in the scoremodel attribute) should not clash with the intrinsic names and some form of naming convention based upon the creating organization should be adopted.

2.5 Tabular Representation

The tables in this Section provide a conceptual, informative description of the elements in the data objects. The columns in these tables refer to:

No:	The number of the data element. An element may be composed of sub-elements. The numbering scheme reflects these relationships.
Name:	The descriptive name of the element.
Explanation:	A brief functional description of the element.
Required:	Indicates if the element is required: M = Mandatory Element that must be included in the data object, if the element at the higher level is included; C = Conditional Element. Existence is dependent on values of other Elements; O = Optional Element.
Multi:	Multiplicity of the element: Blank = single instance; Number = maximum number of times the element is repeatable; n = multiple occurrences allowed, no limit; Repeatability of an element implies that all sub-elements repeat with the same element.
Type:	A description of formatting rules for the data element. Type includes the maximum length of the element: ID = element used to uniquely identify an object; Code = element value from a list of codes; Description = descriptive element, human language Flag = binary flag Enumerated = list of predefined non-numeric options i.e. the definitive list of objects The international character set specified by ISO 10646 will be used for all fields. The type will also include a description of the set of valid values for the sub-element: Coding schemes using numerical values; The set of values as defined in the Domain i.e. making it closed. The list of values cannot be extended to include values not defined in the specification. If there is a need for values not included in the domain set of values then the extension should be done defining a new element under the Extension element that is a part of each data object definition.

Note: Additional descriptive information about the element.

2.5.1 QTI Outcomes Processing Data Objects

Table 2.1 describes the data objects that are used in the construction of the QTI outcomes processing elements.

No	Name	Explanation	Reqd	Mult	Type	Note
1	outcomes_processing	Accumulated outcomes processing and feedback applied within Sections and Assessments.	O	n
1.1	scoremodel	The type of scoring model being adopted.	O		CDATA string describing the model. String 32 chars.	Default string is "SumofScores".
1.2	qticomment	The comments used to annotate the XML file.	O			Comments should be used to aid human readability of the XML file itself.
1.2.1	xml:lang	The language that is being used for the information.	O		String 32 chars.	The language entries will be defined as per the ISO639 and ISO3166 standards.
1.3	outcomes	To create the variables required for the assessment accumulated scores.	M			The assessment accumulated processing variables group.
1.3.1	qticomment	As per structure 1.2.
1.3.2	decvar	Declaration of a variable to be used for scoring.	M	n		Each type of variable must be declared before it is used.
1.3.2.1	varname	The name of the variable to be declared.	M		String 256 chars.	Default is set as 'SCORE".
1.3.2.2	vartype	The type of variable.	M		Enumerated: String Integer (default) Decimal Scientific Boolean Enumerated Set	Default is set to 'Integer'.
1.3.2.3	defaultval	The default value for the variable.	O		Numerical 32 chars String 32 chars True/False	Can be set to any value. Default is set to '0'.
1.3.2.4	minvalue	The minimum value permitted for a numeric score.	O		String 32chars.	Applies to the value of the score after all of the item processing has been completed.
1.3.2.5	maxvalue	The maximum score permitted for a numeric score.	O		String 32chars.	Applies to the value of the score after all of the item processing has been completed.
1.3.2.6	cutvalue	The value above which the participant will have been defined to have mastery of the subject.	O		String 32chars.	The type of this cut value is set by the variable type.
1.3.2.7	members	The set of enumerated values.	O		String 1024chars.	This is a comma separated list without enclosing parentheses.
1.3.3	interpretvar	The interpretation to be applied to the variable in terms relevant to an actor.	O	n		At present this element will be a comment string however it will be further developed in version 1.2.
1.3.3.1	varname	The name of the input variable being described.	O		As per structure 1.3.2.1.
1.3.3.2	view	The view to which the interpretation is applied.	O	n	Enumerated: All (default) Administrator AdminAuthority Assessor Author Candidate Invigilator Proctor Psychometrician Scorer Tutor	The 'All' view is the default value.
1.4	objects_condition	This contains the conditions that are applied to define the ways in which the outcomes variables are combined to create the aggregated value.	O	n
1.4.1	qticomment	As per structure 1.2.
1.4.2	outcomes_metadata	Contains the rules that are applied to the IMS QTI-specific and/or IMS Meta-data meta-data fields of the object to decide if the object scoring is to be aggregated.	C		String 1-256 chars.
1.4.2.1	mdname	Identifies the IMS QTI-specific or IMS Meta-data field that is to be used for the aggregation rule.	M		String 1-64 chars.
1.4.2.2	mdoperator	Identifies the nature of the meta-data field comparison that is to be applied.	M		Enumerated list: EQ NEQ LT LTE GT GTE
1.4.3	and_objects	Contains the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'.	C			See Appendix B for the logic rules.
1.4.3.1	outcomes_metadata	As per structure 1.4.2
1.4.3.2	and_objects	As per structure 1.4.3
1.4.3.3	or_objects	As per structure 1.4.4
1.4.3.4	not_objects	As per structure 1.4.5
1.4.4	or_objects	Contains the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'.	C			See Appendix B for the logic rules.
1.4.4.1	outcomes_metadata	As per structure 1.4.2
1.4.4.2	and_objects	As per structure 1.4.3
1.4.4.3	or_objects	As per structure 1.4.4
1.4.4.4	not_objects	As per structure 1.4.5
1.4.5	not_objects	Contains the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'.	C			This element contains only ONE of the sub-elements. See Appendix B for the logic rules.
1.4.5.1	outcomes_metadata	As per structure 1.4.2
1.4.5.2	and_objects	As per structure 1.4.3
1.4.5.3	or_objects	As per structure 1.4.4
1.4.5.4	not_objects	As per structure 1.4.5
1.4.6	objects_parameter	Contains the value of a particular parameter that is to be used by the corresponding scoring algorithm variable selection. Each parameter has a particular meaning to each scoring algorithm i.e. there is no established vocabulary for these parameters.	O	n		These parameters only apply to the objects within the containing <objects_condition> element.
1.4.6.1	pname	The name of the parameter that is being defined.	M		String 256 chars.	Parameters of the same name may have different significance to different algorithms.
1.4.7	map_input	This element is used to re-map the default input variable to another variable.	O	n
1.4.7.1	varname	The name of the input variable whose name is to be remapped.	O		As per structure 1.3.2.1.
1.4.8	objectscond_extension	The objects condition extension facility.	O	n	ANY
1.5	processing_parameter	The value of a parameter that is passed to the scoring algorithm that is being used.	O	n	String 128 chars.	These parameters apply to the algorithm as a whole and not to specific objects operated on by the algorithm.
1.5.1	pname	The name of the parameter that is being defined.	M		String 256 chars.	Parameters of the same name may have different significance to different algorithms.
1.6	map_output	This is used to remap the result of the scoring algorithm from the default/named variable to another named variable.	O	n	String 256 chars.	The new variable name should have been declared in the <outcomes> element of the parent object. This remapping operates for all of the associated variable names e.g. the *.min , *.max and *.normalized variables.
1.6.1	varname	The name of the output variable whose name is to be remapped.	O		As per structure 1.3.2.1.
1.7	outcomes_feedback_test	Contains the tests to be applied to determine if any and the type of feedback to be presented.	O	n
1.7.1	title	The title of the feedback test.	O		String 256 chars.
1.7.2	test_variable	The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.	M
1.7.2.1	variable_test	The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.	C
1.7.2.1.1	varname	The name of the variable whose state is to be tested.	O		As per structure 1.3.2.1.
1.7.2.1.2	testoperator	Identifies the nature of the variable comparison that is to be applied.	M		Enumerated list: EQ NEQ LT LTE GT GTE
1.7.2.2	and_test	This element allows the construction of complex variable test rules to be built based upon the logical 'AND' operator.	C
1.7.2.2.1	variable_test	As per structure 1.7.2.1.
1.7.2.2.2	and_test	As per structure 1.7.2.2.
1.7.2.2.3	or_test	As per structure 1.7.2.3.
1.7.2.2.4	not_test	As per structure 1.7.2.4.
1.7.2.3	or_test	This element allows the construction of complex variable test rules to be built based upon the logical 'OR' operator.	C
1.7.2.3.1	variable_test	As per structure 1.7.2.1.
1.7.2.3.2	and_test	As per structure 1.7.2.2.
1.7.2.3.3	or_test	As per structure 1.7.2.3.
1.7.2.3.4	not_test	As per structure 1.7.2.4.
1.7.2.4	not_test	This element allows the construction of complex variable test rules to be built based upon the logical 'NOT' operator.	C			Only one of the contained sub-elements can be used within each usage of the NOT operator.
1.7.2.4.1	variable_test	As per structure 1.7.2.1.
1.7.2.4.2	and_test	As per structure 1.7.2.2.
1.7.2.4.3	or_test	As per structure 1.7.2.3.
1.7.2.4.4	not_test	As per structure 1.7.2.4.
1.7.3	displayfeedback	The trigger for displaying feedback.	M	n
1.7.3.1	feedbacktype	The type of feedback to be displayed.	M		Enumerated: Response (default) Solution Hint	The default value is 'Response'.
1.7.3.2	linkrefid	The identifier of the material to be referenced.	M		String 32 chars.	Consistency checking is beyond the scope of this specification. Usage rules are given in the Q&TI Best Practice Guide.

3. XML Binding

3.1 outcomes_processing> Elements

Description: The <outcomes_processing> element is the container for all of the outcomes processing instructions for Assessments and Sections. Multiple outcomes processing containers can be use when multiple scoring algorithms are to be applied to produce the aggregated outcomes. If multiple outcomes_processing elements are supplied, it is the intention that all should be run and the outcomes from all of them together should be reported as the outcomes of the enclosing Section or Assessment. The outcome variables defined by each <outcomes_processing> element should be unique across all outcomes_processing elements defined by a section. In particular, it is an error for multiple <outcomes_processing> elements to set the same outcome variable and the results will be undefined.

Multiplicity: The <outcomes_processing> occurs zero or more times within the <assessment> and <section> elements.

Attributes:

• scoremodel (optional. Default='SumofScores'). The type of the scoring algorithm that is to be used.
  Data-type = String (max of 32 chars).

Elements:

• qticomment
• outcomes
• objects_condition
• processing_parameter
• map_outcome
• outcomes_feedback_test

Example:

<outcomes_processing scoremodel="SumofScores">
  <outcomes>
    <decvar/>
  </outcomes>
</outcomes_processing>

3.1.1 <qticomment> Element

Description: This element contains the comments that are relevant to the outcomes processing structure as a whole.

Multiplicity: Occurs zero or once within the <outcomes_processing> element.

Attributes:

• xml:lang (optional). Identifies the language that is to be used within the instance. The vocabulary is defined as per the XML W3C specification.
  Data-type = string.

3.1.2 <outcomes> Element

Description: The <outcomes> element contains all of the variable declarations that are to be made available to the scoring algorithm. Each variable is declared using the <decvar> element apart from the default variable called 'SCORE' that is an integer and has a default value of zero (0).

Multiplicity: This occurs once within the <outcomes_processing> element.

Attributes: See sub-section 3.2.

3.1.3 <objects_condition> Element

Description: This element contains the conditions that are applied to define the ways in which the outcomes variables are combined to create the aggregated value using the in-built set of algorithm definitions.

Multiplicity: Occurs zero or more times within the <outcomes_processing> element.

Attributes: See sub-section 3.3.

3.1.4 <processing_parameter> Element

Description: This element contains the value of a particular parameter that is to be used by the corresponding scoring algorithm. Each parameter has a particular meaning to each scoring algorithm i.e. there is no established vocabulary for these parameters.

Multiplicity: Occurs zero or more times within the <outcomes_processing> element.

Attributes:

• pname (mandatory). The name of the proprietary parameterized value that is to be used by the scoring algorithm.
  Data-type = string (1-64 chars).

3.1.5 <map_output> Element

Description: This element is used to re-map the named variable to another named variable (given in the body of the element). The target variable name must have been declared using <decvar> in the <outcomes> element of the enclosing <outcomes_processing> element. When a variable is remapped, all of its derived variables are to be remapped as well. Thus if remapping 'SCORE' to 'myScore', 'SCORE.min', 'SCORE.max' and 'SCORE.normalized' would be remapped to 'myScore.min', 'myScore.max' and 'myScore.normalized' respectively.
Data-type = string (1-256 chars).

Multiplicity: This occurs zero or many times within the <outcomes_condition> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is normally used by the scoring algorithm e.g. 'SCORE', 'CORRECT', etc. The default name is 'SCORE' but the variables SCORE.min, SCORE.max and SCORE.normalized are also to be remapped.
  Data-type = String (max of 256 chars).

3.1.6 <outcomes_feedback_test> Element

Description: The <outcomes_feedback> element contains the tests to be applied to determine if any and the type of feedback to be presented. This feedback could include information about passing the assessment etc.

Multiplicity: This occurs zero or many times within the <outcomes_processing> element.

Attributes: See sub-section 3.4.

3.2 <outcomes> Element

Description: The <outcomes> element contains all of the variable declarations that are to be made available to the scoring algorithm. Each variable is declared using the <decvar> element apart from the default variable called 'SCORE' that is an integer and has a default value of zero (0). The declaration of each variable is accompanied by the implicit declaration of the <vname>.min, <vname>.max and <vname>.normalized variables. These are used to store the minimum value, maximum value and normalized value (in the range zero to one) of the scoring variable.

Multiplicity: This occurs once within the <outcomes_processing> element.

Attributes: None.

Elements:

• qticomment
• decvar
• interpretvar

3.2.1 <qticomment> Element

Description: This element contains the comments that are relevant to the outcomes.

Multiplicity: Occurs zero or once within the <outcomes> element.

Attributes:

• xml:lang (optional). Identifies the language that is to be used within the instance. The vocabulary is defined as per the XML W3C specification.
  Data-type = string (1-2048 chars).

3.2.2 <decvar> Element

Description: The <decvar> element permits the declaration of the scoring variables. The default name is 'SCORE' but the variables SCORE.min, SCORE.max and SCORE.normalized are also declared and made available from the scoring algorithms. A <decvar> with no attributes is assumed to define the integer variable 'SCORE' with a default value of zero.

Multiplicity: This occurs one or more times within the <outcomes> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is to be declared.
  Data-type = String (max of 256 chars).
• vartype (required. Enumerated list: String, Decimal, Scientific, Boolean, Integer, Enumerated, Set. Default=Integer). The type of the variable declared.
  Data-type = Enumerated list.
• defaultval (optional). The default value to which the variable is to be initialized.
  Data-type = String (max of 16 chars).
• cutvalue (optional). The value/grade above which the subject is considered to have been mastered.
  Data-type = String (max of 16 chars).
• minvalue (optional). The minimum value permitted for a numeric score.
  Data-type = String (max of 32 chars).
• maxvalue (optional). The maximum value permitted for a numeric score.
  Data-type = String (max of 32 chars).
• members (optional). The set of enumerated values that constitute the member.
  Data-type = String (max of 1024 chars).

3.2.3 <interpretvar> Element

Description: The <interpretvar> element is used to provide statistical interpretation information about the associated variables.

Multiplicity: This occurs zero or more times within the <outcomes> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable whose interpretation details are to be described. The default name is 'SCORE'.
  Data-type = String (max of 256 chars).
• view (optional with selection from the enumerated list of: All, Administrator, AdminAuthority, Assessor, Author, Candidate, InvigilatorProctor, Psychometrician, Scorer, Tutor. Default=All). The view defines the scope for the display of the associated information i.e. to whom the material can be presented.
  Data-type = Enumerated list.

Another description of this element and its sub-components is given in IMS QTI: ASI XML Binding V1.2
[QTI, 02b].

3.3 <objects_condition>

Description: Each <objects_condition> element defines a subset of the objects (Items and Sections) selected by the selection algorithm that are to be used in this score. This allows for the construction of subscores. If no <outcomes_metadata> element is present within the <objects_conditions> element to select a subset of objects, then the <objects_conditions> applies to all objects selected by the selection algorithm. If multiple <objects_conditions> are given within a single <outcomes_processing> element, then the algorithm is applied to the union of all objects selected. The <objects_parameter>, <map_input> and <objectscond_extension> elements within the <objects_condition> element apply only to those objects selected in the condition. If an object is selected by more than one <objects_condition> element, it should receive parameter assignments and input mapping as if it were the first element so selected. These conditions include the identification and mapping of the variables for input.

Multiplicity: This occurs zero or many times within the <outcomes_processing> element.

Attributes: None.

Elements:

• qticomment
• outcomes_metadata
• and_objects
• or_objects
• not_objects
• objects_parameter
• map_input
• objectscond_extension

3.3.1 <qticomment> Element

Description: This element contains the comments that are relevant to the presentation of the Item.

Multiplicity: Occurs zero or once within the <objects_condition> element.

Attributes:

• xml:lang (optional). Identifies the language that is to be used within the instance. The vocabulary is defined as per the XML W3C specification.
  Data-type = string (1-2048 chars).

3.3.2 <outcomes_metadata> Element

Description: This element defines the rule that is applied to the QTI-specific and/or IMS Meta-data meta-data fields of the object to decide if the object scoring is to be aggregated. The content contains the value of the meta-data field that is being tested for within the rule. This data is a string of up to 64 characters length.

Multiplicity: Occurs zero or once within the <objects_condition> element.

Attributes: See Sub-section 3.5.

Elements: None.

3.3.3 <and_out> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'.

Multiplicity: This occurs zero or once within the <objects_condition> element.

Attributes: See Sub-section 3.6.

3.3.4 <or_out> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'.

Multiplicity: This occurs zero or once within the <objects_condition> element.

Attributes: See Sub-section 3.7.

3.3.5 <not_out> Element

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'.

Multiplicity: This occurs zero or once within the <objects_condition> element.

Attributes: See Sub-section 3.8.

3.3.6 <objects_parameter> Element

Description: This element contains the value of a particular parameter that is to be used by the corresponding scoring algorithm variable selection. Each parameter has a particular meaning to each scoring algorithm i.e. there is no established vocabulary for these parameters. These parameters are applied only to the objects selected by the enclosing <outcomes_condition> element. If multiple <outomes_condition> select the same object, the parameters are taken from the first <outcomes_condition> that selects the element.

Multiplicity: Occurs zero or more times within the <objects_condition> element.

Attributes:

• pname (mandatory). The name of the proprietary parameterized value that is to be used by the scoring algorithm.
  Data-type = string (1-64 chars).

3.3.7 <map_input> Element

Description: This element is used to re-map the named input variable to another variable. The default variable names are derived from the type of scoring algorithm identified in <outcomes_processing> element. The target variable name must have been declared in the evaluation objects that undergoing aggregation and must be of the same type as the default variable. When a variable is remapped, all of its derived variables are to be remapped as well. The input mapping is applied only to the objects selected by the enclosing <outcomes_condition> element. If multiple <outcomes_condition> select the same object, the input mapping used is the one contained in the first <outcomes_condition> that selects the element.
Data-type = String (1-256 chars).

Multiplicity: This occurs zero or many times within the <objects_condition> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is normally used in the scoring algorithm e.g. 'SCORE', 'COUNT', etc.
  Data-type = String (max of 256 chars).

3.3.8 <objectscond_extension> Element

Description: This element contains the proprietary extensions that can be used to extend the functionally capabilities of the <outcomes_condition> element.

Multiplicity: Occurs zero or once within the <objects_condition> element.

Attributes: None.

3.4 <outcomes_feedback_test> Element

Description: The <outcomes_feedback> element contains the tests to be applied to determine if any and the type of feedback to be presented. This feedback could include information about passing the assessment etc.

Multiplicity: This occurs zero or many times within the <outcomes_processing> element.

Attributes:

• title (optional). The title of the feedback test.
  Data-type = String (max of 256 chars).

Elements:

• test_variable
• displayfeedback

3.4.1 <test_variable> Elements

Description: The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.

Multiplicity: This occurs once within the <outcomes_feedback_test> element.

Attributes: See Sub-section 3.9.

3.4.2 <displayfeedback> Element

Description: The <displayfeedback> element is responsible for assigning an associated feedback to the aggregated scoring if the 'True' state results.

Multiplicity: This occurs zero or more times within the <outcomes_feedback_test> element.

Attributes:

• feedbacktype (optional - enumerated list: Response, Solution, Hint. Default = Response). The type of feedback that has been triggered by the associated outcomes condition. Within the Assessment and Section feedback elements only the 'Response' type of feedback is supported.
  Data-type = Enumerated list.
• linkrefid (required). The identifier of the associated feedback. An <assessfeedback> or <sectionfeedback> element must exist with this identifier as defined by the 'ident' attribute.
  Data-type = String (max of 256 chars).

3.5 <outcomes_metadata> Element

Description: This element defines the rule that is applied to the IMS QTI-specific and/or IMS Meta-data fields of the object to decide if the object scoring is to be aggregated. The content contains the value of the meta-data field that is being tested for within the rule.
Data-type = String (1-64 chars).

Multiplicity: Occurs zero or once within the <objects_condition> element.

Attributes:

• mdname (mandatory). Identifies the QTI-specific or IMS Meta-data field that is to be used for the aggregation rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-256 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

Example:

<objects_condition>
  <outcomes_metadata mdname="qmd_topic" mdoperator="EQ">Trigonometry</outcomes_metadata>
</objects_condition>

3.6 <and_objects> Element

Description: The <and_objects> element supports the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'. The logic rules for this element are described in Appendix B.

Multiplicity: This occurs zero or once within the <objects_condition>, and zero or many times within the <and_objects>, <or_objects> and <not_objects> elements.

Attributes: None.

Elements:

• outcomes_metadata
• and_objects
• or_objects
• not_objects

3.6.1 <outcomes_metadata> Element

Description: This element defines the rule that is applied to the IMS QTI-specific and/or IMS Meta-data fields of the object to decide if the object scoring is to be aggregated. The content contains the value of the meta-data field that is being tested for within the rule.
Data-type = string (1-64 chars).

Multiplicity: Occurs zero or many times within the <and_objects> element.

Attributes:

• mdname (mandatory). Identifies the IMS QTI-specific or IMS Meta-data field that is to be used for the aggregation rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.6.2 <and_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'.

Multiplicity: This occurs zero or many times within the <and_objects> element.

Attributes: See Sub-section 3.6.

3.6.3 <or_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'.

Multiplicity: This occurs zero or many times within the <and_objects> element.

Attributes: See Sub-section 3.7.

3.6.4 <not_objects> Element

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'.

Multiplicity: This occurs zero or many times within the <and_objects> element.

Attributes: See Sub-section 3.8.

3.7 <or_objects> Element

Description: The <or_objects> element supports the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'. The logic rules are described in Appendix B.

Multiplicity: This occurs zero or once within the <objects_condition>, and zero or more times within the <and_objects>, <or_objects> and <not_objects> element.

Attributes: None.

Elements:

• outcomes_metadata
• and_objects
• or_objects
• not_objects

3.7.1 <outcomes_metadata> Element

Description: This element defines the rule that is applied to the IMS QTI-specific and/or IMS Meta-data fields of the object to decide if the object scoring is to be aggregated. The content contains the value of the meta-data field that is being tested for within the rule.
Data-type = string (1-64 chars).

Multiplicity: Occurs zero or many times within the <or_objects> element.

Attributes:

• mdname (mandatory). Identifies the QTI-specific or IMS Meta-data field that is to be used for the aggregation rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.7.2 <and_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'.

Multiplicity: This occurs zero or many times within the <or_objects> element.

Attributes: See Sub-section 3.6.

3.7.3 <or_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'.

Multiplicity: This occurs zero or many times within the <or_objects> element.

Attributes: See Sub-section 3.7.

3.7.4 <not_objects> Element

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'.

Multiplicity: This occurs zero or many times within the <or_objects> element.

Attributes: See Sub-section 3.8.

3.8 <not_objects> Element

Description: The <not_objects> element supports the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'. The logic rules are listed in Appendix B.

Multiplicity: This occurs zero or once within the <objects_condition>, and zero or many times within the <and_objects>, <or_objects> and <not_objects> element.

Attributes: None.

Elements:

• outcomes_metadata
• and_objects
• or_objects
• not_objects

3.8.1 <outcomes_metadata> Element

Description: This element defines the rule that is applied to the QTI-specific and/or IMS Meta-data fields of the object to decide if the object scoring is to be aggregated. The content contains the value of the meta-data field that is being tested for within the rule.
Data-type = string (1-64 chars).

Multiplicity: Occurs zero or once within the <not_objects> element.

Attributes:

• mdname (mandatory). Identifies the IMS QTI-specific or IMS Meta-data field that is to be used for the aggregation rule. No validation check is made in the instance on the existence or otherwise of this field.
  Data-type = string (1-64 chars).
• mdoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the meta-data field comparison that is to be applied.
  Data-type = string (1-16 chars).

Elements: None.

3.8.2 <and_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'AND' operator. The object is selected for aggregation if all of the contained rules are 'True'.

Multiplicity: This occurs zero or once within the <not_objects> element.

Attributes: See Sub-section 3.6.

3.8.3 <or_objects> Element

Description: This element allows the construction of complex score condition rules to be built based upon the logical 'OR' operator. The object is selected for aggregation if at least one of the contained rules is 'True'.

Multiplicity: This occurs zero or many times within the <not_objects> element.

Attributes: See Sub-section 3.7.

3.8.4 <not_objects> Element

Description: This element allows the construction of complex rules to be built based upon the logical 'NOT' operator. The object is selected for aggregation if the contained rule(s) is 'False'.

Multiplicity: This occurs zero or many times within the <not_objects> element.

Attributes: See Sub-section 3.8.

3.9 <test_variable> Element

Description: The <test_variable> element contains the conditions that are applied to a defined set of input scoring variables to determine if feedback is to be presented. Complex test structures can be constructed using the associated logic elements.

Multiplicity: This occurs once within the <outcomes_feedback_test> element.

Attributes: None.

Elements:

• variable_test
• and_test
• or_test
• not_test

3.9.1 <variable_test> Element

Description: The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.

Multiplicity: This occurs zero or once within the <test_variable> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable whose condition is to be tested. The default name is 'SCORE'.
  Data-type = String (max of 256 chars).
• testoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the variable comparison that is to be applied.
  Data-type = string (1-16 chars).

3.9.2 <and_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'AND' operator.

Multiplicity: This occurs zero or once within the <test_variable> element.

Attributes: See Sub-section 3.10.

3.9.3 <or_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'OR' operator.

Multiplicity: This occurs zero or many times within the <test_variable> element.

Attributes: See Sub-section 3.11.

3.9.4 <not_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'NOT' operator.

Multiplicity: This occurs zero or once within the <test_variable> element.

Attributes: See Sub-section 3.12.

3.10 <and_test> Element

Description: The <and_test> element is used to define variable test conditions all of which must be defined as 'True' to result in the output variables being assigned the corresponding values.

Multiplicity: This occurs zero or once within the <test_variable> element, and zero or more times in the <and_test>, <or_test> and <not_test> elements.

Attributes: None.

Elements:

• variable_test
• and_test
• or_test
• not_test

3.10.1 <variable_test> Element

Description: The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.

Multiplicity: This occurs zero or more times within the <and_test> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is to be tested. The default name is 'SCORE'.
  Data-type = String (max of 256 chars).
• testoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the variable comparison that is to be applied.
  Data-type = string (1-16 chars).

3.10.2 <and_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'AND' operator.

Multiplicity: This occurs zero or many times within the <and_test> element.

Attributes: See Sub-section 3.10.

3.10.3 <or_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'OR' operator.

Multiplicity: This occurs zero or many times within the <and_test> element.

Attributes: See Sub-section 3.11.

3.10.4 <not_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'NOT' operator.

Multiplicity: This occurs zero or many times within the <and_test> element.

Attributes: See Sub-section 3.12.

3.11 <or_test> Element

Description: The <or_test> element is used to define variable test conditions at least one of which must be defined as 'True' to result in the output variables being assigned the corresponding values.

Multiplicity: This occurs zero or more times within the <test_variable> element.

Attributes: None.

Elements:

• variable_test
• and_test
• or_test
• not_test

3.11.1 <variable_test> Element

Description: The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.

Multiplicity: This occurs zero or more times within the <or_test> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is to be tested. The default name is 'SCORE'.
  Data-type = String (max of 256 chars).
• testoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the variable comparison that is to be applied.
  Data-type = string (1-16 chars).

3.11.2 <and_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'AND' operator.

Multiplicity: This occurs zero or many times within the <or_test> element.

Attributes: See Sub-section 3.10.

3.11.3 <or_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'OR' operator.

Multiplicity: This occurs zero or many times within the <or_test> element.

Attributes: See Sub-section 3.11.

3.11.4 <not_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'NOT' operator.

Multiplicity: This occurs zero or many times within the <or_test> element.

Attributes: See Sub-section 3.12.

3.12 <not_test> Element

Description: The <not_test> element is used to define variable test conditions that result in the condition being defined as 'False' to result in the output variables being assigned the corresponding values.

Multiplicity: This occurs zero or more times within the <test_var> element.

Attributes: None.

Elements:

• variable_test
• and_test
• or_test
• not_test

3.12.1 <variable_test> Element

Description: The conditional test that is to be applied to the aggregated score variables. A wide range of separate and combinatorial tests can be applied.

Multiplicity: This occurs zero or once within the <not_test> element.

Attributes:

• varname (optional. Default = 'SCORE'). The name of the variable that is to be tested. The default name is 'SCORE'.
  Data-type = String (max of 256 chars).
• testoperator (mandatory with selection from the enumerated list of: EQ, NEQ, LT, LTE, GT, GTE). Identifies the nature of the variable comparison that is to be applied.
  Data-type = string (1-16 chars).

3.12.2 <and_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'AND' operator.

Multiplicity: This occurs zero or once within the <not_test> element.

Attributes: See Sub-section 3.10.

3.12.3 <or_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'OR' operator.

Multiplicity: This occurs zero or once within the <not_test> element.

Attributes: See Sub-section 3.11.

3.12.4 <not_test> Element

Description: This element allows the construction of complex variable test rules to be built based upon the logical 'NOT' operator.

Multiplicity: This occurs zero or many times within the <not_test> element.

Attributes: See Sub-section 3.12.

4. Best Practice & Implementation Guide

4.1 Overall Data Model

The overall IMS QTI ASI Outcomes Processing data model is shown in Figure 4.1.

4.2 Relationship to the Other IMS Specifications

The relationship of this 'Outcomes Processing' specification to the other parts of the IMS QTI specifications is:

• The 'Outcomes Processing' root element is a part of the Assessment and Section core data objects in the IMS QTI ASI: Information Model and IMS QTI ASI: XML Binding. The <response_processing> element (defined within the QTI ASI Information model and XML Binding documents) defines the variables that are used as inputs to the response processing when the algorithm operates on items. Similarly, the <outcomes_processing> element (defined in this document) defines the variables that are used as inputs when the outcomes processing operates on sections. Note that in predefined algorithms, these operate on either an integer variable 'SCORE' or a boolean variable 'CORRECT'. To be used with the maximum number of outcomes processing algorithms an item's <response_processing> element should set both variables. Also, some algorithms treat items that were not attempted or not presented differently from those which were attempted so there must be some way of querying the items attempted or presented status. Finally, other algorithms assume that a default value for the outcome variable (usually a null score) is available even if the item was never presented. All of these data can be extracted from a IMS QTI results report, however, most implementations will probably use equivalent internal data structures;
• The 'Selection & Ordering' component of the IMS QTI specification [QTI, 02d] interacts with the 'Outcomes Processing' part. This interaction is a result of the fact that the aggregated score for a Section and/or Assessment depends upon which Items and Section were selected for presentation to the participant;
• The IMS QTI Results Reporting specification contains the description of how the results are recorded for a participant that has undertaken an evaluation. These results will depend upon the aggregated scores derived from the outcomes processing mechanism. The QTI Results Reporting specification describes how the results from any IMS QTI-compliant Assessment, Section and Item can be reported.

4.3 Basic Example XML Instances

The following set of basic examples demonstrates how the different use-cases are supported. Table 4.1 lists the examples used to show the operation of each of the outcomes processing algorithms.

Algorithm	Example
	1	2	3	4	5	6	7	8	9
Number Correct	*
Number Correct Attempted	*
Weighted Number Correct		*
Weighted Number Correct Attempted		*
Parameter Weighted Number Correct		*
Parameter Weighted Number Correct Attempted		*
Sum of Scores			*				*	*	*
Sum of Scores Attempted			*
Weighted Sum of Scores				*
Weighted Sum of Scores Attempted				*
Parameter Weighted Sum of Scores				*
Parameter Weighted Sum of Scores Attempted				*
Best K from N					*
Guessing Penalty						*

4.3.1 Example ('Number Correct' and 'Number Correct Attempted')

A Section consists of 10 Items each of which has a 'score' assigned to its default scoring boolean variable (CORRECT) that is used to denote if the Item has been answered correctly i.e. 'True'. The number of correct responses is assigned to the default score variable for the Section (COUNT). The outcomes processing algorithms for calculating the number of correct responses both in terms of the Items presented and the Items presented and attempted are given below.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S01" ident = "IMS_QTIV1p2_S_OUT_01"> <outcomes_processing scoremodel = "NumberCorrect"> <outcomes> <decvar defaultval = "0" varname = "COUNT" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> </outcomes_processing> <outcomes_processing scoremodel = "NumberCorrectAttempted"> <outcomes> <decvar defaultval = "0" varname = "COUNT_Attempted" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <map_output varname="COUNT">COUNT_Attempted</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2">
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110	... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2">
111 112 113 114 115 116 117 118 119 120 121	... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'NumberCorrect' scoring algorithm that is supported for the Section is:

  COUNT = 0
  COUNT.min = 0
  COUNT.max = 0
  COUNT.normalized = 0.0
  for ALL_PRESENTED_ITEMS do
    if ITEM_HAS_VARIABLE(CORRECT) = True
    then
      begin
        COUNT.max = COUNT.max + 1
        if item.CORRECT = True
        then COUNT = COUNT + 1
      endif
    enddo
  COUNT.normalized = COUNT / COUNT.max

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

COUNT = 7

COUNT.min = 0

COUNT.max = 10

COUNT.normalized = 0.7

The 'NumberCorrectAttempted' scoring algorithm that is supported for the Section is (the variable 'COUNT_Attempted' must be used to ensure that there is no scoping clash when two algorithms attempt to operate on the COUNT variable):

  COUNT_Attempted = 0
  COUNT_Attempted.min = 0
  COUNT_Attempted.max = 0
  COUNT_Attempted.normalized = 0
  for ALL_ATTEMPTED_ITEMS do
    if ITEM_HAS_VARIABLE(CORRECT) = True
    then
      begin
        COUNT_Attempted.max = COUNT_Attempted.max + 1
        if item.CORRECT = True
        then COUNT_Attempted = COUNT_Attempted + 1
      endif
    enddo
  COUNT_Attempted.normalized = COUNT_Attempted / COUNT_Attempted.max

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

COUNT_Attempted = 7

COUNT_Attempted.min = 0

COUNT_Attempted.max = 8

COUNT_Attempted.normalized = 0.875

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample01.xml'. All more complete description of the 'Number Correct' and 'NumberCorrectAttempted' scoring algorithms is given in Appendices B1 and B2.

4.3.2 Example ('Weighted Number Correct', 'Weighted Number Correct Attempted', 'Parameter Weighted Number Correct' and 'Parameter Weighted Number Correct Attempted')

A Section consists of 10 Items (a mixture of implicit and referenced) and each Item has a 'score' assigned to its default scoring boolean variable (CORRECT) that is used to denote if the Item has been answered correctly i.e. 'True'. The four different weighting algorithms are presented using the appropriate weighting calculation mechanism. The number of correct responses is assigned to the default score variable for the Section (COUNT).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S02" ident = "IMS_QTIV1p2_S_OUT_02"> <outcomes_processing scoremodel = "WeightedNumberCorrect"> <outcomes> <decvar defaultval = "0" varname = "COUNT_WNC" vartype = "Integer" minvalue = "0" maxvalue = "15" cutvalue = "10"/> </outcomes> <map_output varname="COUNT">COUNT_WNC</map_output> </outcomes_processing> <outcomes_processing scoremodel = "WeightedNumberCorrectAttempted"> <outcomes> <decvar defaultval = "0" varname = "COUNT_WNCA" vartype = "Integer" minvalue = "0" maxvalue = "15" cutvalue = "10"/> </outcomes> <map_output varname="COUNT">COUNT_WNCA</map_output> </outcomes_processing> <outcomes_processing scoremodel = "ParameterWeightedNumberCorrect"> <outcomes> <decvar defaultval = "0" varname = "COUNT_PWNC" vartype = "Integer" minvalue = "0" maxvalue = "20" cutvalue = "10"/> </outcomes> <objects_condition> <objects_parameter pname="qmd_weighting">2<objects_parameter> </objects_condition> <map_output varname="COUNT">COUNT_PWNC</map_output> </outcomes_processing> <outcomes_processing scoremodel = "ParameterWeightedNumberCorrectAttempted"> <outcomes> <decvar defaultval = "0" varname = "COUNT_PWNC" vartype = "Integer" minvalue = "0" maxvalue = "20" cutvalue = "10"/> </outcomes> <objects_condition> <objects_parameter pname="qmd_weighting">2<objects_parameter> </objects_condition> <map_output varname="COUNT">COUNT_PWNC</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2">
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100	<itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ...
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160	<resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2">
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212	<itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'WeightedNumberCorrect' scoring algorithm that is supported for the Section is:

  COUNT_WNC = 0
  COUNT_WNC.min = 0
  COUNT_WNC.max = 0
  COUNT_WNC.normalized = 0
  for ALL_SELECTED_ITEMS do 
    if ITEM_HAS_VARIABLE(COUNT) = True
    then
      begin
        item.weighting = item(qmd_weighting)
        COUNT_WNC.max = COUNT_WNC.max + item.weighting
        if item.CORRECT = True
        then COUNT_WNC = COUNT_WNC + item.weighting
      endif
    enddo
  COUNT_WNC.normalized = COUNT_WNC / COUNT_WNC.max

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

COUNT_WNC = 10

COUNT_WNC.min = 0

COUNT_WNC.max = 15

COUNT_WNC.normalized = 0.667

The 'WeightedNumberCorrectAttempted' scoring algorithm that is supported for the Section is:

  COUNT_WNCA = 0
  COUNT_WNCA.min = 0
  COUNT_WNCA.max = 0
  COUNT_WNCA.normalized = 0
  for ALL_ATTEMPTED_ITEMS do 
    if ITEM_HAS_VARIABLE(COUNT) = True
    then
      begin
        item.weighting = item(qmd_weighting)
        COUNT_WNCA.max = COUNT_WNCA.max + item.weighting
        if item.CORRECT = True
        then COUNT_WNCA = COUNT_WNCA + item.weighting
      endif
    enddo
  COUNT_WNCA.normalized = COUNT_WNCA / COUNT_WNCA.max

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

COUNT_WCNA = 10

COUNT_WCNA.min = 0

COUNT_WCNA.max = 12

COUNT_WCNA.normalized = 0.833

The 'ParameterWeightedNumberCorrect' scoring algorithm that is supported for the Section is:

  COUNT_PWNC = 0
  COUNT_PWNC.min = 0
  COUNT_PWNC.max = 0
  COUNT_PWNC.normalized = 0
  for ALL_SELECTED_ITEMS do 
    if ITEM_HAS_VARIABLE(COUNT) = True
    then
      begin
        item.weighting = objects_parameter(qmd_weighting)
        COUNT_PWNC.max = COUNT.max + item.weighting
        if item.CORRECT = True
        then COUNT_PWNC = COUNT_PWNC + item.weighting
      endif
    enddo
  COUNT_PWNC.normalized = COUNT_PWNC / COUNT_PWNC.max

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

COUNT_PWNC = 14

COUNT_PWNC.min = 0

COUNT_PWNC.max = 20

COUNT_PWNC.normalized = 0.7

The 'ParameterWeightedNumberCorrectAttempted' scoring algorithm that is supported for the Section is:

  COUNT_PWNCA = 0
  COUNT_PWNCA.min = 0
  COUNT_PWNCA.max = 0
  COUNT_PWNCA.normalized = 0
  for ALL_ATTEMPTED_ITEMS do 
    if ITEM_HAS_VARIABLE(COUNT) = True
    then
      begin
        item.weighting = objects_parameter(qmd_weighting)
        COUNT_PWNCA.max = COUNT_PWNCA.max + item.weighting
        if item.CORRECT = True
        then COUNT_PWNCA = COUNT_PWNCA + item.weighting
      endif
    enddo
  COUNT_PWNCA.normalized = COUNT_PWNCA / COUNT_PWNCA.max

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

COUNT_PWNCA = 14

COUNT_PWNCA.min = 0

COUNT_PWNCA.max = 16

COUNT_PWNCA.normalized = 0.875

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample02.xml'. A more complete description of the weighted number correct' scoring algorithms is given in Appendices B3, B4, B5 and B6.

4.3.3 Example ('SumofScores' and 'SumofScoresAttempted')

A Section consists of 10 Items each of which has a score assigned to its default scoring integer variable (SCORE). A score of 1 is given for a correct answer and 0 for an incorrect answer. The sum of these scores is assigned to the default score variable for the Section (SCORE). The outcomes processing algorithms for calculating the sum of scores both in terms of the Items presented and the Items presented and attempted are given below.

1 2 3 4 5 6 7 8 9 10	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S03" ident = "IMS_QTIV1p2_S_OUT_03"> <outcomes_processing scoremodel = "SumofScores"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> </outcomes_processing> <outcomes_processing scoremodel = "SumofScoresAttempted"> <outcomes>
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70	<decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <map_output varname="SCORE">SCORE_Attempted</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> ...
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120	<resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'SumOfScores' scoring algorithm that is supported for the Section is:

  SCORE = 0
  SCORE.min = 0
  SCORE.max = 0
  SCORE.normalized = 0.0
  for ALL_PRESENTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
    then
      begin
        SCORE.min = SCORE.min + item.SCORE.minvalue
        SCORE.max = SCORE.max + item.SCORE.maxvalue
        SCORE = SCORE + item.SCORE
      endif
    enddo
  SCORE.normalized = (SCORE-SCORE.min) / (SCORE.max-SCORE.min)

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

SCORE = 7

SCORE.min = 0

SCORE.max = 10

SCORE.normalized = 0.7

The 'SumofScoresAttempted' scoring algorithm that is supported for the Section is (the default output variable name has been remapped to avoid overloading the 'SCORE' variable during the concurrent application of the two scoring algorithms):

  SCORE_Attempted = 0
  SCORE_Attempted.min = 0
  SCORE_Attempted.max = 0
  SCORE_Attempted.normalized = 0.0
  for ALL_ATTEMPTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
    then
      begin
        SCORE_Attempted.min = SCORE_Attempted.min + item.SCORE.minvalue
        SCORE_Attempted.max = SCORE_Attempted.max + item.SCORE.maxvalue
        SCORE_Attempted = SCORE_Attempted + item.SCORE
      endif
    enddo
  SCORE_Attempted.normalized = (SCORE_Attempted - SCORE_Attempted.min) / 
                      (SCORE_Attempted.max - SCORE_Attempted.min)

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

SCORE_Attempted = 7

SCORE_Attempted.min = 0

SCORE_Attempted.max = 8

SCORE_Attempted.normalized = 0.875

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample03.xml'. All more complete description of the 'Sum of Scores' and 'Sum of Scores Attempted' scoring algorithms are given in Appendix B7 and B8.

4.3.4 Example ('Weighted Sum of Scores', 'Weighted Sum of Scores Attempted', 'Parameter Weighted Sum of Scores' and 'Parameter Weighted Sum of Scores Attempted')

A Section consists of 10 Items each of which has a score assigned to its default scoring integer variable (SCORE). A score of '1' is given for a correct answer and '0' for an incorrect answer. The four different weighting algorithms are presented using the appropriate weighting calculation mechanism. The sum of these scores is assigned to the default score variable for the Section (SCORE).

1 2	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S04" ident = "IMS_QTIV1p2_S_OUT_04">
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50	<outcomes_processing scoremodel = "WeightedSumofScores"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "19" cutvalue = "12"/> </outcomes> <map_output varname="SCORE">SCORE_WSOS</map_output> </outcomes_processing> <outcomes_processing scoremodel = "WeightedSumofScoresAttempted"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "15" cutvalue = "10"/> </outcomes> <map_output varname="SCORE">SCORE_WSOSA</map_output> </outcomes_processing> <outcomes_processing scoremodel = "ParameterWeightedSumofScores"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "20" cutvalue = "10"/> </outcomes> <objects_condition> <objects_parameter pname="qmd_weighting">2<objects_parameter> </objects_condition> <map_output varname="SCORE">SCORE_PWSOS</map_output> </outcomes_processing> <outcomes_processing scoremodel = "ParameterWeightedSumofScoresAttempted"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "20" cutvalue = "10"/> </outcomes> <objects_condition> <objects_parameter pname="qmd_weighting">2<objects_parameter> </objects_condition> <map_output varname="SCORE">SCORE_PWSOSA</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>3</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing>
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110	<outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> <itemmetadata>
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170	<qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>6</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes>
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 201 203 204 205 206 207 208 209 210 211 212	<decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'WeightedSumOfScores' scoring algorithm that is supported for the Section is:

  SCORE_WSOS = 0
  SCORE_WSOS.min = 0
  SCORE_WSOS.max = 0
  SCORE_WSOS.normalized = 0.0
  for ALL_PRESENTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
      begin
        SCORE_WSOS.min = SCORE_WSOS.min + (item.SCORE.minvalue * item(qmd_weighting))
        SCORE_WSOS.max = SCORE_WSOS.max + (item.SCORE.maxvalue * item(qmd_weighting))
        SCORE_WSOS = SCORE_WSOS + (item.SCORE * item.qmd_weighting)
      endif
    enddo
  SCORE_WSOS.normalized = (SCORE_WSOS - SCORE_WSOS.min) / (SCORE_WSOS.max - SCORE_WSOS.min)

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

SCORE_WSOS = 16

SCORE_WSOS.min = 0

SCORE_WSOS.max = 19

SCORE_WSOS.normalized = 0.842

The 'WeightedSumOfScoresAttempted' scoring algorithm that is supported for the Section is:

  SCORE_WSOSA = 0
  SCORE_WSOSA.min = 0
  SCORE_WSOSA.max = 0
  SCORE_WSOSA.normalized = 0.0
  for ALL_ATTEMPTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
      begin
        SCORE_WSOSA.min = SCORE_WSOSA.min + (item.SCORE.minvalue * item(qmd_weighting))
        SCORE_WSOSA.max = SCORE_WSOSA.max + (item.SCORE.maxvalue * item(qmd_weighting))
        SCORE_WSOSA = SCORE_WSOSA + (item.SCORE * item(qmd_weighting))
      endif
    enddo
  SCORE_WSOSA.normalized = (SCORE_WSOSA-SCORE_WSOSA.min)/(SCORE_WSOSA.max-SCORE_WSOSA.min)

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

SCORE_WSOSA = 16

SCORE_WSOSA.min = 0

SCORE_WSOSA.max = 17

SCORE_WSOSA.normalized = 0.941

The 'ParameterWeightedSumOfScores' scoring algorithm that is supported for the Section is:

  SCORE_PWSOS = 0
  SCORE_PWSOS.min = 0
  SCORE_PWSOS.max = 0
  SCORE_PWSOS.normalized = 0.0
  for ALL_PRESENTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
      begin
        SCORE_PWSOS.min = SCORE_PWSOS.min + 
                  (item.SCORE.minvalue * objects_parameter(qmd_weighting))
        SCORE_PWSOS.max = SCORE_PWSOS.max + 
                  (item.SCORE.maxvalue * objects_parameter(qmd_weighting))
        SCORE_PWSOS = SCORE_PWSOS + (item.SCORE * objects_parameter(qmd_weighting))
      endif
    enddo
  SCORE_PWSOS.normalized = (SCORE_PWSOS-SCORE_PWSOS.min)/(SCORE_PWSOS.max-SCORE_PWSOS.min)

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

SCORE_PWSOS = 14

SCORE_PWSOS.min = 0

SCORE_PWSOS.max = 20

SCORE_PWSOS.normalized = 0.7

The 'ParameterWeightedSumOfScoresAttempted' scoring algorithm that is supported for the Section is:

  SCORE_PWSOSA = 0
  SCORE_PWSOSA.min = 0
  SCORE_PWSOSA.max = 0
  SCORE_PWSOSA.normalized = 0.0
  for ALL_PRESENTED_ITEMS do
    if ITEM_HAS_VARIABLE(SCORE) = True
      begin
        SCORE_PWSOSA.min = SCORE_PWSOSA.min + 
                  (item.SCORE.minvalue * objects_parameter(qmd_weighting))
        SCORE_PWSOSA.max= SCORE_PWSOSA.max + 
                  (item.SCORE.maxvalue * objects_parameter(qmd_weighting))
        SCORE_PWSOSA = SCORE_PWSOSA + (item.SCORE * objects_parameter(qmd_weighting))
      endif
    enddo
  SCORE_PWSOSA.normalized =               (SCORE_PWSOSA-SCORE_PWSOSA.min) /
                  (SCORE_PWSOSA.max-SCORE_PWSOSA.min)

In the case where the first eight Items have been attempted and the first seven have been answered correctly then the final set of Section variable states will be:

SCORE_PWSOSA = 14

SCORE_PWSOSA.min = 0

SCORE_PWSOSA.max = 16

SCORE_PWSOSA.normalized = 0.875

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample04.xml'. All more complete description of the 'Sum of Scores' scoring algorithm is given in Appendices B9, B10, B11 and B12.

4.3.5 Example ('Best K from N')

A Section consists of 10 Items and each has a score assigned to its default scoring integer variable (SCORE). The 'maxvalue' score is given for a correct answer and the 'minvalue' for an incorrect answer. The sum of these scores is assigned to the default score variable for the Section (SCORE). The aggregated score is set as the best seven scores.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S05" ident = "IMS_QTIV1p2_S_OUT_05"> <outcomes_processing scoremodel = "BestKofN"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <processing_parameter pname = "BestK">7</processing_parameter> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2">
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83	... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2">
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114	... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'BestKfromN' scoring algorithm that is supported for the Section is:

  SCORE = 0
  SCORE.min = 0
  SCORE.max = 0
  SCORE.normalized = 0.0
  For HIGHEST_SCORES_FROM_PRESENTED_ITEMS(7) and ITEM_HAS_VARIABLE(SCORE) do
    begin
      SCORE.min = SCORE.min + item.SCORE.minvalue
      SCORE.max = SCORE.max + item.SCORE.maxvalue
      SCORE = SCORE + item.SCORE
    Enddo
  SCORE.normalized = (SCORE - SCORE.min) / (SCORE.max - SCORE.min)

In the case where all ten Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

SCORE = 7

SCORE.min = 0

SCORE.max = 7

SCORE.normalized = 1

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample05.xml'. A more detailed description of these scoring algorithms are presented in Appendices B13.

4.3.6 Example ('Guessing Penalty' and 'WeightedGuessingPenalty')

A Section consists of 10 Items each of which has a 'score' assigned to its default scoring boolean variable (CORRECT) that is used to denote if the Item has been answered correctly i.e. 'True'. The number of correct responses is assigned to the default score variable for the Section (COUNT). The outcomes processing algorithms for calculating the number of correct responses both in terms of the Items presented and the Items presented and attempted are given below.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S01" ident = "IMS_QTIV1p2_S_OUT_01"> <outcomes_processing scoremodel = "GuessingPenalty"> <outcomes> <decvar defaultval = "0" varname = "COUNT" vartype = "Decimal" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> </outcomes_processing> <outcomes_processing scoremodel = "WeightedGuessingPenalty"> <outcomes> <decvar defaultval = "0" varname = "COUNT_Weighted" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <map_output varname="COUNT">COUNT_Weighted</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry>
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112	</qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel>
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 142 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171	<fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing>
172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230	... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_penaltyvalue</fieldlabel> <fieldentry>0.2</fieldentry> </qtimetadata> </itemmetadata>
231 232 233 234 235 236 237 238 239 240 241	... <resprocessing> <outcomes> <decvar defaultval="False" vartype="Boolean" varname="CORRECT"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'GuessingPenalty' scoring algorithm that is supported for the Section is:

  COUNT = 0
  COUNT.correct = 0
  COUNT.incorrect = 0
  COUNT.unattempted = 0
  for ALL_PRESENTED(childobjects)and (HAS_VARIABLE(CORRECT)  do
    begin
      if child.object(ATTEMPTED) = True 
      then
        begin
          if childobject.CORRECT = True
          then 
            begin
              parentobject.COUNT.correct = parentobject.COUNT.correct + 1
              parentobject.COUNT = parentobject.COUNT + 1
            end
          else 
            begin
              parentobject.COUNT.incorrect = parentobject.COUNT.incorrect + 1
              parentobject.COUNT = parentobject.COUNT - item(qmd_penaltyvalue)
            end
        endif
      else childobject.COUNT.unattempted = childobject.COUNT.unattempted + 1
    enddo

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

COUNT = 6.8

COUNT.correct = 7

COUNT.correct = 7

COUNT.incorrect = 1

COUNT.unattempted = 2

The 'WeightedGuessingPenalty' scoring algorithm that is supported for the Section is:

  COUNT_Weighted = 0
  COUNT_Weighted.correct = 0
  COUNT_Weighted.incorrect = 0
  COUNT_Weighted.unattempted = 0
  for ALL_PRESENTED(childobjects)and (HAS_VARIABLE(CORRECT)  do
    begin
      if child.object(ATTEMPTED) = True 
      then
        begin
          if childobject.CORRECT = True
          then 
            begin
              parentobject.COUNT_Weighted.correct = 
                parentobject.COUNT_Weighted.correct + 1
              parentobject.COUNT_Weighted = parentobject.COUNT_Weighted + 1
            end
          else 
            begin
              parentobject.COUNT_Weighted.incorrect = 
                parentobject.COUNT_Weighted.incorrect + 1
              parentobject.COUNT_Weighted = 
              (parentobject.COUNT_Weighted * item(qmd_weighting) - 
              item(qmd_penaltyvalue)
            end
        endif
      else childobject.COUNT_Weighted.unattempted = 
        childobject.COUNT_Weighted.unattempted + 1
    enddo

In the case where eight Items have been attempted and seven have been answered correctly then the final set of Section variable states will be:

COUNT = 13.8

COUNT.correct = 7

COUNT.correct = 7

COUNT.incorrect = 1

COUNT.unattempted = 2

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample06.xml'. All more complete description of these scoring algorithm is given in Appendices B14 and B15.

4.3.7 Example (Remapped Output 'Sum of Scores')

A Section consists of 5 Items (a mixture of implicit and referenced) and each Item has a score assigned to its default scoring integer variable (SCORE). A score of 1 is given for a correct answer and -1 for an incorrect answer. The sum of these scores is assigned to the new score variable for the Section (Section_06_SCORE).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S07" ident = "IMS_QTIV1p2_S_OUT_07"> <outcomes_processing scoremodel = "SumofScores"> <outcomes> <decvar defaultval= "0" varname= "Section_07_SCORE" vartype= "Integer" minvalue= "0" maxvalue= "10" cutvalue= "6"/> </outcomes> <map_output varname = "SCORE">Section_07_SCORE</map_output> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes>

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115

... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The output remapped 'SumofScores' scoring algorithm that is supported for the Section is:

  Section_07_SCORE = 0
  Section_07_SCORE.min = 0
  Section_07_SCORE.max = 0
  Section_07_SCORE.normalized = 0.0
  For ALL_ATTEMPTED_ITEMS and ITEM_HAS_VARIABLE(SCORE) do
    begin
      Section_07_SCORE.min = Section_07_SCORE.min + item.SCORE.minvalue
      Section_07_SCORE.max = Section_07_SCORE.max + item.SCORE.maxvalue
      Section_07_SCORE = Section_07_SCORE + item.SCORE
    Enddo
  Section_07_SCORE.normalized = 
  (Section_07_SCORE - Section_07_SCORE.min)/(Section_07_SCORE.max - Section_07_SCORE.min)

In the case where all ten Items have been attempted and eight have been answered correctly then the final set of output remapped Section variable states will be:

Section_07_SCORE = 8

Section_07_SCORE.min = 0

Section_07_SCORE.max = 10

Section_07_SCORE.normalized = 0.8

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample07.xml'.

4.3.8 Example (Remapped Input 'Sum of Scores')

A Section consists of 5 Items (a mixture of implicit and referenced) and each Item has a score assigned to its default scoring integer variable (SCORE). A score of 1 is given for a correct answer and -1 for an incorrect answer. The sum of these scores is assigned to the default score variable for the Section (SCORE).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S06" ident = "IMS_QTIV1p2_S_OUT_06"> <outcomes_processing scoremodel = "SumofScores"> <outcomes> <decvar defaultval= "0" varname= "SCORE" vartype= "Integer" minvalue= "0" maxvalue= "10" cutvalue= "6"/> </outcomes> <objects_condition> <map_input varname = "SCORE">item_SCORE</map_input> </objects_condition> </outcomes_processing> <selection_ordering> <selection seltype/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/>
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89	</outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> ... <resprocessing> <outcomes>
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117	<decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1" varname="item_SCORE"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The input remapped 'SumofScores' scoring algorithm that is supported for the Section is:

  SCORE = 0
  SCORE.min = 0
  SCORE.max = 0
  SCORE.normalized = 0.0
  For ALL_ATTEMPTED_ITEMS and ITEM_HAS_VARIABLE(item_SCORE) do
    begin
      SCORE.min = SCORE.min + item.item_SCORE.minvalue
      SCORE.max = SCORE.max + item.item_SCORE.maxvalue
      SCORE = SCORE + item.item_Score
    Enddo
  SCORE.normalized = (SCORE - SCORE.min)/(SCORE.max - SCORE.min)

In the case where all ten Items have been attempted and eight have been answered correctly then the final set of Section variable states will be:

SCORE = 8

SCORE.min = 0

SCORE.max = 10

SCORE.normalized = 0.8

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample08.xml'.

4.3.9 Example (Parameterized 'Sum of Scores')

A Section consists of 10 Items and each has a score assigned to its default scoring integer variable (SCORE). A score of 1 is given for a correct answer and -1 for an incorrect answer. The sum of these scores is assigned to the default score variable for the Section (SCORE). The intention is to produce a result score for all those Items on the topic that are 'Advanced' and 'Basic'.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S08" ident = "IMS_QTIV1p2_S_OUT_08"> <outcomes_processing scoremodel = "SumOfScores"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "6"/> </outcomes> <objects_condition> <or_objects> <outcomes_metadata mdname="qmd_levelofdifficulty" mdoperator= "EQ">basic</outcomes_metadata> <outcomes_metadata mdname="qmd_levelofdifficulty" mdoperator="EQ">advanced</outcomes_metadata> </or_objects> </objects_condition> </outcomes_processing> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>3</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>advanced</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ...
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112	<resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>advanced</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ...
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172	<resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_06" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>6</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_07" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>2</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_08" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>basic</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata>
173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223	... <resprocessing> <outcomes> <decvar defaultval = "0" minvalue = "0" maxvalue = "1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_09" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>advanced</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> <item ident = "IMS_QTIV1p2_I_OUT_10" maxattempts = "2"> <itemmetadata> <qtimetadata> <qtimetadatafield> <fieldlabel>qmd_weighting</fieldlabel> <fieldentry>1</fieldentry> </qtimetadatafield> </qtimetadata> <qmd_levelofdifficulty>advanced</qmd_levelofdifficulty> <qmd_topic>qtiv1p2test</qmd_topic> </itemmetadata> ... <resprocessing> <outcomes> <decvar defaultval="0" minvalue="0" maxvalue="1"/> </outcomes> ... </resprocessing> ... </item> </section> </questestinterop>

The 'SumOfScores' scoring algorithm that is supported for the Section is:

  SCORE = 0
  SCORE.min = 0
  SCORE.max = 0
  SCORE.normalized = 0.0
  For ALL_PRESENTED_ITEMS            and ITEM_HAS_VARIABLE(SCORE)
                and 
    (item.metadata.difficulty = basic or item.metadata.difficulty = advanced) 
  do
    begin
      SCORE.min = SCORE.min + item.SCORE.minvalue
      SCORE.max = SCORE.max + item.SCORE.maxvalue
      SCORE = SCORE + item.SCORE
    Enddo
  SCORE.normalized = (SCORE - SCORE.min) / (SCORE.max - SCORE.min)

In the case where all ten Items have been attempted and seven have been answered correctly then the final set of Section variable states will be (the meta-data selections ensure that all Items are selected to contribute to the score aggregation):

SCORE = 7

SCORE.min = 0

SCORE.max = 10

SCORE.normalized = 0.7

This example is available in the file: 'ims_qtiasiv1p2/outcome/basicoutexample09.xml'.

4.4 Advanced Example XML Instances

The following set of advanced examples demonstrates how the different use-cases are supported.

4.4.1 Example (Multiple-choice Quiz)

Multiple-choice quiz - this is a simple "end of chapter" quiz given by an LMS. The scores are reported back to the student for self-evaluation, so there is no particular need for high reliability. All of the questions are discrete (single response) questions whose outcomes are a dichotomous variable e.g. multiple-choice questions. For the sake of being definite we assume there are 5 Items and that we have a defined outcome, "CORRECT" for each Item. This takes on the value "True" if the response was correct and "False' if the response was not correct. All items are given equal weight. We wish to know, (1) the number of correct items, (2) the percentage of total score obtained by a candidate and (3) the percentage of items actually answered which were correct.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S01" ident = "IMS_QTIV1p2_S_OUTADV_01"> <outcomes_processing scoremodel = "NumberCorrect"> <outcomes> <decvar defaultval = "0" varname = "COUNT" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing> <outcomes_processing scoremodel = "SumofScores"> <outcomes>
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82	<decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "SCORE">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "SCORE">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing> <outcomes_processing scoremodel = "NumberCorrectAttempted"> <outcomes> <decvar defaultval = "0" varname = "COUNT" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <map_output varname="COUNT">COUNT_Attempted</map_output> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing> <sectionfeedback ident = "SectionMastery" view = "All"> <material> <mattext>Well done. You have passed this Topic.</mattext> </material> </sectionfeedback> <sectionfeedback ident = "SectionFail" view = "All"> <material> <mattext>Sorry. You have failed this topic.</mattext> </material> </sectionfeedback> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> <presentation> <flow> <material> <mattext>Which one of the following statements is true about the QTI specification ?</mattext> </material> <response_lid ident = "LID01" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes" minnumber = "1" maxnumber = "1"> <flow_label> <response_label ident = "A">
83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 11 112 113 114 115 116 117 118 119 120 121 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142	<material> <mattext>It is independent of pedagogy</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "B"> <material> <mattext>It cannot be extended to support proprietary features</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "C"> <material> <mattext>It contains many unnecesary features </mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "D"> <material> <mattext>It provides only one view for different types of participants</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "0" vartype = "Integer" minvalue = "0" maxvalue = "1" varname = "SCORE"/> </outcomes> <respcondition continue = "No"> <qticomment>To detect the correct answer.</qticomment> <conditionvar> <varequal respident = "LID01">A</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback feedbacktype = "Response" linkrefid = "Correct"/> </respcondition> <respcondition continue = "No"> <qticomment>To detect if the incorrect answer is supplied.</qticomment> <conditionvar> <not> <varequal respident = "LID01" case = "No">A</varequal> </not> </conditionvar> <displayfeedback feedbacktype = "Response" linkrefid = "InCorrect"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct">
143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202	<material> <mattext>Correct answer is A.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect" view = "All"> <material> <mattext xml:lang="en-US" xml:space="default" texttype= "" charset= ""> Answers B, C and D are incorrect. </mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <presentation> <flow> <material> <mattext>The QTI specification supports the exchange of which data structure ?</mattext> </material> <response_lid ident = "LID02" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes" minnumber = "1" maxnumber = "1"> <flow_label> <response_label ident = "A"> <material> <mattext>Response</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "B"> <material> <mattext>Question</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "C"> <material> <mattext>Item</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "D"> <material> <mattext>Test</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <outcomes> <decvar varname = "CORRECT" vartype = "Boolean" defaultval = "False"/> <decvar defaultval = "0" vartype = "Integer" minvalue = "0" maxvalue = "1" varname = "SCORE"/> </outcomes> <respcondition continue = "No">
203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 281 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262	<qticomment>Correct response detection</qticomment> <conditionvar> <varequal respident = "LID02">C</varequal> </conditionvar> <setvar varname = "CORRECT" action = "Set">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback linkrefid = "Correct" feedbacktype = "Response"/> </respcondition> <respcondition continue = "No"> <qticomment>Processing for the incorrect answer.</qticomment> <conditionvar> <not> <varequal respident = "LID02">C</varequal> </not> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct" view = "All"> <material> <mattext>Correct. Items.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect" view = "All"> <material> <mattext>No. The correct answer is Item.</mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <presentation> <flow> <material> <mattext>What is the maximum number of Assessments can be contained in a single QTI ASI XML instance?</mattext> </material> <response_lid ident = "TF01" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "A"> <material> <mattext>0</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "B"> <material> <mattext>1</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "C"> <material> <mattext>More than one</mattext> </material> </response_label> </flow_label> </render_choice>
263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322	</response_lid> </flow> </presentation> <resprocessing> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "0" vartype = "Integer" minvalue = "0" maxvalue = "1" varname = "SCORE"/> </outcomes> <respcondition continue = "No"> <qticomment>Processing for the incorrect answer.</qticomment> <conditionvar> <not> <varequal respident = "TF01">B</varequal> </not> </conditionvar> <displayfeedback feedbacktype = "Response" linkrefid = "Incorrect"/> </respcondition> <respcondition continue = "No"> <qticomment>Processing for the correct answer.</qticomment> <conditionvar> <varequal respident = "TF01" case = "No">B</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback linkrefid = "Correct" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect"> <material> <mattext>Sorry, you are wrong. A QTI-XML instance CAN contain, at most, one Asssessment.</mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "1"> <presentation> <flow> <material> <mattext>Which one of the following data structures is used to provide hierarchically structured assessments ?</mattext> </material> <response_lid ident = "LID03" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "A"> <material> <mattext>Assessment</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "B"> <material> <mattext>Section</mattext>
323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382	</material> </response_label> </flow_label> <flow_label> <response_label ident = "C"> <material> <mattext>Item</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "D"> <material> <mattext>Question</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <outcomes> <decvar varname = "CORRECT" vartype = "Boolean" defaultval = "False"/> <decvar defaultval = "0" vartype = "Integer" minvalue = "0" maxvalue = "1" varname = "SCORE"/> </outcomes> <respcondition continue = "No"> <qticomment>Correct response check</qticomment> <conditionvar> <varequal respident = "LID03" case = "No">B</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback linkrefid = "Correct" feedbacktype = "Response"/> </respcondition> <respcondition> <conditionvar> <not> <varequal respident = "LID03" case = "No">B</varequal> </not> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct" view = "All"> <material> <mattext>Correct.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect" view = "All"> <material> <mattext>No. Only Sections support hierachical Assessments.</mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "1"> <presentation> <flow> <material>
383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 403 403 404 405 406 407 408 409 410 411 412 313 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442	<mattext>Which question types are supported by the QTILite specification ?</mattext> </material> <response_lid ident = "LID03" rcardinality = "Multiple" rtiming = "No"> <render_choice shuffle = "Yes" minnumber = "1" maxnumber = "4"> <flow_label> <response_label ident = "A"> <material> <mattext>Multiple choice</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "B"> <material> <mattext>Multiple response</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "C"> <material> <mattext>Fill-in-the-blank</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "D"> <material> <mattext>Drag-and-drop</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <outcomes> <decvar varname = "CORRECT" vartype = "Boolean" defaultval = "False"/> <decvar defaultval = "0" vartype = "Integer" minvalue = "0" maxvalue = "1" varname = "SCORE"/> </outcomes> <respcondition> <qticomment>Correct answer response processing.</qticomment> <conditionvar> <varequal respident = "LID03">A</varequal> </conditionvar> <setvar varname = "CORRECT" action = "Set">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback linkrefid = "Correct" feedbacktype = "Response"/> </respcondition> <respcondition> <qticomment>Incorrect answer response processing.</qticomment> <conditionvar> <not> <varequal respident = "LID03">A</varequal> </not> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/>
443 444 445 446 447 448 448 459 460 461 462 463 464 465 466	</respcondition> </resprocessing> <itemfeedback ident = "Correct" view = "All"> <material> <mattext>Correct. Multple-choice only.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect" view = "All"> <material> <mattext>No. Only Multiple-choice questions are supported.</mattext> </material> </itemfeedback> </item> </section> </questestinterop>

The actual percentage total score and percentage of items actually answered will be derived values from the SCORE.normalized (SumofScores) and COUNT_Attempted.normalized (NumberCorrectAttempted) variables. If it is assumed that the candidate attempts the first four questions and gets the first three of those correct, then the set of variables are:

SCORE = 3

SCORE.min = 0

SCORE.max = 5

SCORE.normalized = 0.6

PercentageTotalScore = 60%

COUNT = 3

COUNT.min = 0

COUNT.max = 5

COUNT.normalized = 0.6

COUNT_Attempted = 3

COUNT_Attempted.min = 0

COUNT_Attempted.max = 4

COUNT_Attempted.normalized = 0.75

PercentageCorrectAttempted = 75%

This example is given as 'ims_qtiasiv1p2/outcome/advoutexample01.xml'.

4.4.2 Example (True/false Quiz)

True/false quiz - this is a simple "end of chapter" quiz given by an LMS. The scores are reported back to the student for self-evaluation, so there is no particular need for high reliability. All of the questions are discrete true/false. For the sake of being definite we assume there are 10 Items and that we have a defined outcome, "isCorrect" for each Item. This takes on the value '1' if the response was correct and "-1' if the response was not correct. All items are given equal weight. We wish to know, (1) the number of correct items, (2) the total score and (3) the weighted total score.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60	<questestinterop> <section title = "IMSQTI_Outcomes_Processing_S02" ident = "IMS_QTIV1p2_S_OUTADV_02"> <outcomes_processing scoremodel = "NumberCorrect"> <outcomes> <decvar defaultval = "0" varname = "COUNT" vartype = "Integer" minvalue = "0" maxvalue = "10" cutvalue = "5"/> </outcomes> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "COUNT">5</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing> <outcomes_processing scoremodel = "SumofScores"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "-10" maxvalue = "10" cutvalue = "0"/> </outcomes> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "COUNT">0</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "COUNT">0</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing> <outcomes_processing scoremodel = "ParameterSumofScoresWeighted"> <outcomes> <decvar defaultval = "0" varname = "SCORE" vartype = "Integer" minvalue = "-20" maxvalue = "20" cutvalue = "0"/> </outcomes> <objects_condition> <objects_parameter pname="qmd_weighting">2</objects_parameter> </objects_condition> <map_output varname="SCORE">SCORE_PWSOS</map_output> <outcomes_feedback_test title = "Section Mastery Test"> <test_variable> <variable_test testoperator = "GTE" varname = "COUNT">0</variable_test> </test_variable> <displayfeedback linkrefid = "SectionMastery" feedbacktype = "Response"/> </outcomes_feedback_test> <outcomes_feedback_test title = "Section Fail Test"> <test_variable> <variable_test testoperator = "LT" varname = "COUNT">0</variable_test> </test_variable> <displayfeedback linkrefid = "SectionFail" feedbacktype = "Response"/> </outcomes_feedback_test> </outcomes_processing>
61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 11 112 113 114 115 116 117 118 119 120	<sectionfeedback ident = "SectionMastery" view = "All"> <material> <mattext>Well done. You have passed this Topic.</mattext> </material> </sectionfeedback> <sectionfeedback ident = "SectionFail" view = "All"> <material> <mattext>Sorry. You have failed this topic.</mattext> </material> </sectionfeedback> <selection_ordering> <selection/> <order order_type = "Sequential"/> </selection_ordering> <item ident = "IMS_QTIV1p2_I_OUT_01" maxattempts = "2"> <presentation> <flow> <material> <mattext>Multiple Assessments cannot be contained in a single QTI ASI XML instance. </mattext> </material> <response_lid ident = "TF01" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "T"> <material> <mattext>True</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "F"> <material> <mattext>False</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <qticomment>Processing for the incorrect answer.</qticomment> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "-1" vartype = "Integer" minvalue="-1" maxvalue="1" varname = "SCORE"/> </outcomes> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF01">T</varequal> </conditionvar> <displayfeedback feedbacktype = "Response" linkrefid = "Incorrect"/> </respcondition> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF01" case = "No">F</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar>
121 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180	<setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback linkrefid = "Correct" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect"> <material> <mattext>Sorry, you are wrong. A QTI-XML instance CAN contain multiple Assessments. </mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_02" maxattempts = "2"> <presentation> <flow> <material> <mattext>Time-based responses are supported by the QTI specification. </mattext> </material> <response_lid ident = "TF02" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "T"> <material> <mattext>True</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "F"> <material> <mattext>False</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <qticomment>Processing for the correct answer.</qticomment> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "-1" vartype = "Integer" minvalue="-1" maxvalue="1" varname = "SCORE"/> </outcomes> <respcondition> <conditionvar> <varequal respident = "TF02">T</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback feedbacktype = "Response" linkrefid = "Correct"/>
181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 281 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240	</respcondition> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF02" case = "No">F</varequal> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect"> <material> <mattext>Sorry, you are wrong. Time-based reponses ARE supported by the specification. </mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_03" maxattempts = "2"> <presentation> <flow> <material> <mattext>Items defined according to QTI V1.0 are not compatible with Items from QTI V1.2. </mattext> </material> <response_lid ident = "TF03" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "T"> <material> <mattext>True</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "F"> <material> <mattext>False</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <qticomment>Processing for the correct answer.</qticomment> <outcomes> <decvar defaultval = "True" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "-1" vartype = "Integer" minvalue="-1" maxvalue="1" varname = "SCORE"/> </outcomes> <respcondition> <conditionvar> <varequal respident = "TF03">F</varequal> </conditionvar>
241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300	<setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback feedbacktype = "Response" linkrefid = "Correct"/> </respcondition> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF03" case = "No">T</varequal> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect"> <material> <mattext>Sorry, you are wrong. V1.0 Items ARE compatible with the V1.2 specification. </mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_04" maxattempts = "2"> <presentation> <flow> <material> <mattext>The QTI specification is incompatible with the IMS Content Packaging specification. </mattext> </material> <response_lid ident = "TF04" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label> <response_label ident = "T"> <material> <mattext>True</mattext> </material> </response_label> </flow_label> <flow_label> <response_label ident = "F"> <material> <mattext>False</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <qticomment>Processing for the correct answer.</qticomment> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "-1" vartype = "Integer" minvalue="-1" maxvalue="1" varname = "SCORE"/> </outcomes> <respcondition>
301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360	<conditionvar> <varequal respident = "TF04">F</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback feedbacktype = "Response" linkrefid = "Correct"/> </respcondition> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF04" case = "No">T</varequal> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect"> <material> <mattext>Sorry, you are wrong. The IMS QTI and Content Packaging specifications are compatible. </mattext> </material> </itemfeedback> </item> <item ident = "IMS_QTIV1p2_I_OUT_05" maxattempts = "2"> <presentation> <flow class = "Block"> <material> <mattext texttype = "text/plain" charset = "ascii-us">The QTI specification is used within the ADL SCORM. </mattext> </material> <response_lid ident = "TF05" rcardinality = "Single" rtiming = "No"> <render_choice shuffle = "Yes"> <flow_label class = "Block"> <response_label ident = "T"> <material> <mattext>True</mattext> </material> </response_label> </flow_label> <flow_label class = "Block"> <response_label ident = "F"> <material> <mattext>False</mattext> </material> </response_label> </flow_label> </render_choice> </response_lid> </flow> </presentation> <resprocessing> <qticomment>Processing for the correct answer.</qticomment> <outcomes> <decvar defaultval = "False" vartype = "Boolean" varname = "CORRECT"/> <decvar defaultval = "-1" vartype = "Integer"
361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392	minvalue="-1" maxvalue="1" varname = "SCORE"/> </outcomes> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF05" case = "No">F</varequal> </conditionvar> <setvar action = "Set" varname = "CORRECT">True</setvar> <setvar action = "Set" varname = "SCORE">1</setvar> <displayfeedback feedbacktype = "Response" linkrefid = "Correct"/> </respcondition> <respcondition continue = "No"> <conditionvar> <varequal respident = "TF05" case = "No">T</varequal> </conditionvar> <displayfeedback linkrefid = "Incorrect" feedbacktype = "Response"/> </respcondition> </resprocessing> <itemfeedback ident = "Correct" view = "All"> <material> <mattext>Correct answer.</mattext> </material> </itemfeedback> <itemfeedback ident = "Incorrect" view = "All"> <material> <mattext>Sorry, you are wrong. The QTI specification is NOT used within ADL SCORM. </mattext> </material> </itemfeedback> </item> </section> </questestinterop>