IMS Accessible Portable Item Protocol (APIP) Conformance and Certification

Version 1.0

 

Candidate Final
Version 1.0

 

Date Issued:            26 March  2012

Latest version:         http://www.imsglobal.org/apip/

IPR and Distribution Notices

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.

IMS takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on IMS’s procedures with respect to rights in IMS specifications can be found at the IMS Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.

Copyright © 2012 IMS Global Learning Consortium. All Rights Reserved.

Use of this specification to develop products or services is governed by the license with IMS found on the IMS website: http://www.imsglobal.org/speclicense.html.

Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.

The limited permissions granted above are perpetual and will not be revoked by IMS or its successors or assigns.

THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.

Public contributions, comments and questions can be posted here: http://www.imsglobal.org/community/forum/categories.cfm?catid=110

Executive Summary

IMS APIP – Accessible Portable Item Protocol – is a technical standard for digital interchange of assessment content (items and tests). APIP also describes digital delivery options for a range of common accessibility needs. Use of APIP enables assessment materials to be exchanged digitally among a wide variety of products, such as item/test authoring products, item banks, and test delivery systems.

APIP addresses two long-standing needs in assessments. First, it allows for the transfer of content between vendors using a standardized XML format. Second, it supplies the necessary accessibility information in that content to support the needs of all students. Using IMS Access For All Personal Needs and Preferences (PNP) profile information, the content can be tailored to meet the access needs of each student.

The APIP standard defines an assessment file exchange that serves all students, not only students who have disabilities or are English language learners. In fact, the default content of the question (item) is based on the well established IMS Question and Test Interoperability (QTI) standard that has been used internationally for over a decade. APIP uses a new profile of QTI v2.1.

Figure 1 illustrates the basic idea of APIP. Authors create content, providing extra information when needed to make that content accessible to all students. An assessment rostering system can provide the specific needs of individual students; these are documented in the PNP profile. An APIP certified delivery system takes the needs of a specific student, and ensures that the content is provided to the student with all their needed access supports.

 

Description: BasicConcept.tif

Figure 1 The Basic Concept of APIP.

The three main areas of APIP are:

·         Content: The accessible content, which has supporting information for different kinds of accessibility needs

·         PNP: The user Personal Needs & Preferences profile, which tells the delivery system what the needs are for specific examinees

·         Delivery: The delivery system, which combines the user needs with the content to ensure the item is accessible for the examinee

Table of Contents

Executive Summary.. 2

1        Introduction.. 4

1.1      Scope and Context 4

1.2      Status of this Document 4

1.3      Structure of this Document 4

2        Conformance and Certification Overview... 6

2.1      Relationship between IMS APIP and IMS QTI. 6

2.2      Conformance Certifications. 6

2.3      How Conformance Certification is Achieved and Recorded. 6

2.4      Achieving Conformance Certification. 6

3        Assessment Conformance Statements. 8

3.1      Conformance Profiles. 8

4        Achieving Certification.. 16

4.1      Automated Validation. 16

4.2      Test Case Based Validation. 17

4.3      Documented or Demonstrable Validation. 17

Appendix A – Accessibility Features. 18

Appendix B – RFP Language for APIP.. 24

B1 – Accessible Portable Item Protocol Standard. 24

B2 – Compliant Content Format 25

B3 – Learner Information Profiles. 26

B4 – Digital Test Delivery System.. 27

B5 – APIP Conformance Certification and Evolution. 27

About this Document.. 29

List of Contributors. 29

Revision History.. 30

 

1                  Introduction

1.1            Scope and Context

The purpose of this document is to define the different areas of conformance available to Accessible Portable Item Protocol (APIP) and to describe the procedures for achieving certification for version 1.0.  APIP is a standard created and governed by IMS Global Learning Consortium, a non-profit learning standards organization. This document is intended to be helpful, for example, to vendors who wish to certify their systems as officially conforming to APIP v1.0 as supported by IMS. Ultimately, certification of systems will help ensure that diverse end-users (students) receive access to computer-based assessments.

1.2            Status of this Document

This document is Candidate Final, meaning the technical solution described in the specification is as definitive as possible in the absence of numerous systems and content that have implemented the specification and achieved official conformance. IMS strongly encourages its members and the community in general to continue/begin developing to the specification now. When a sufficient number of systems have proven the solution works through successful conformance achievement, the Candidate Final specification will be submitted to the IMS membership for Final approval. IMS expects only minor refinements based on implementation experience during this Candidate Final period.

As Candidate Final, the IMS Assessment APMG (Accredited Profile Management Group) takes ownership of the specification and the conformance program. The APMG manages the evolution and promotion of conformance and certification. This document also provides the structure and suggested language for requests for vendor bids (see Appendix B). By defining conformance and certification expectations, the APMG will assist vendors to plan development of solutions that use the APIP standard.

Revisions to the APIP specification will be posted here: http://www.imsglobal.org/APIP/

To join the IMS developer and conformance certification community focused on APIP and QTI v2.1 please visit the IMS QTI2/APIP Alliance online here: http://www.imsglobal.org/developers/apipalliance/index.cfm

Public contributions, comments and questions can be posted here: http://www.imsglobal.org/community/forum/categories.cfm?catid=110

1.3            Structure of this Document

The structure of the rest of this document is:

2.  Conformance and Certification Overview

Describes how the conformance program works and the process for achieving certification;

3.  Assessment Conformance Statements

Contains details of the conformance options available for certification to the APIP v1.0 and the QTI v2.1 specifications;

4.  Achieving Certification

Describes the mechanics of the conformance program;

Appendix A – Accessibility Features

Lists the complete accessibility features available in APIP v1.0 organized by categories;

Appendix B – RFP Language for APIP

Provides information about how to properly describe APIP requirements to be included in Requests for Proposals.

 

1.4            Related Documents

[APIP, 12a]     Accessible Portable Item Protocol (APIP) v1.0 Best Practices and Implementation Guide, Candidate Final Release, G.Driscoll, T.Hoffmann, W.Ostler, M.Russell, M.McKell and C.Smythe, IMS Global Inc., March 2012.

[APIP, 12b]     Accessible Portable Item Protocol (APIP) v1.0 Terms and Definitions, Candidate Final Release, G.Driscoll, T.Hoffmann, W.Ostler, M.Russell, M.McKell and C.Smythe, IMS Global Inc., March 2012.

[CC, 11]          IMS Common Cartridge Profile v1.2, J.Kahn, IMS Global, Inc., October 2011.

[PNP, 10]        IMS Access For All Personal Needs & Preferences Information Model v2.0, R.Schwerdtfeger, M.Rothberg and C.Smythe, Final Release, IMS Global Inc., April 2010.

[QTI, 06]         IMS Question & Test Interoperability Assessment Test, Section and Item Information Model v2.1, S.Lay and P.Gorissen, Public Draft Revision 2, IMS Global Inc., June 2006.

 

2                  Conformance and Certification Overview

2.1            Relationship between IMS APIP and IMS QTI

APIP uses a precisely defined subset of IMS QTI v2.1 with additional features included in some other IMS specifications. The first step in implementing APIP is to implement the APIP defined subset of QTI v2.1. This will allow assessment suppliers to utilize QTI when accessibility is not a requirement and to receive a QTI conformance certification.

APIP and the profiles of QTI used in APIP will stay synchronized by IMS Global in conjunction with IMS member organizations through the QTI/APIP Accredited Profile Management Group (APMG). The QTI subset within APIP, such as interaction types (question types), is expected to grow over time as the marketplace for digital assessment evolves.

The additional features of APIP beyond QTI are accessibility metadata (based on IMS Access For All), accessible delivery options (developed on the original APIP grant program) and curriculum standards metadata (based on IMS Content Packaging/Common Cartridge). These features are defined in the APIP specification.

2.2            Conformance Certifications

IMS is establishing well-defined conformance certifications for QTI v2.1 and APIP v1.0 to ensure high levels of interoperability.  The conformance certification process involves testing for the ability to support different levels of features. The conformance details are described in Section 3.

2.3            How Conformance Certification is Achieved and Recorded

Conformance certification is granted by passing a series of tests, submitting the results to IMS and agreeing to become part of the support community (the QTI/APIP Alliance http://www.imsglobal.org/apip/index.html). IMS records all certifications on a public website: http://imscert.org. At the time conformance is certified, IMS issues a unique registration number designator for the product name and version for use in proposals. Buyers and users are encouraged to visit the certification website to find certified products and to log any issues with certified products.

The community aspect of conformance is critical to address any issues that may arise in the marketplace as well as the evolution to new versions.  IMS requires suppliers to participate in resolving issues that may arise, as well as helping to guide and implement each incremental revision to the conformance certifications.

2.4            Achieving Conformance Certification

The tests required for certification are different depending on the particular certification sought. The tests are also different for content/authoring tools versus delivery platforms. This is because content/authoring tools are tested to ensure that the content produced and exported is conformant within the scope of item types and tests that a particular tool can generate, while delivery systems are tested to ensure they can appropriately process a range of content alternatives. Conformant delivery platforms must import conformant content and provide conformant results. Conformant authoring tools must export conformant content and may or may not import conformant content, depending on the purpose of the tool.

3                  Assessment Conformance Statements

The following conformance matrix lists the current profiles available for QTI and APIP certification as they relate to these categories:

QTI Delivery System & Authoring System

The QTI conformance criteria for Delivery Systems and Authoring Systems.

APIP Authoring System

The APIP conformance criteria for Authoring Systems and APIP packages.

APIP Delivery System

The APIP conformance criteria for Delivery Systems.

APIP Personal Needs & Preferences (PNP) Systems

The APIP PNP conformance criteria for systems supplying or processing PNP instances.

3.1            Conformance Profiles

Named profiles currently defined for conformance are described here as they appear in the matrix below.

QTIv2.1 Summative Entry

Conformance for Delivery Systems that process Tests used for 'Summative' assessment (this is equivalent to APIP Entry minus the accessibility features).

QTIv2.1 Formative Entry

Conformance for Delivery Systems that process Tests based upon QTIv2.1 (this is equivalent to the Common Cartridge profile for QTIv2.1).

QTIv2.1 Core

Conformance for Delivery Systems that process Tests based upon QTIv2.1 (this is equivalent to the Common Cartridge profile but with all of the prohibited features made Elective).

APIP Entry

The entry level APIP conformance (this is equivalent to the Common Cartridge profile). Entry level conformance also permits the set of implementation Electives.

APIP Core

This is the APIP Entry level plus the set of implementation Electives. It is to this level that IMS recommends vendors aspire.

APIP PNP Entry

This is the APIP PNP Entry level requirements for a Preferences System producing or processing PNP instances.

APIP PNP Core

This is the APIP PNP conformance level that IMS recommends vendors aspire for a Preferences System producing or processing PNP instances.

 

Legend

[ X ]       Denotes that an implementation IS NOT expected to support the feature and an instance is PROBHIBITED from containing the feature.

[ R ]       Denotes that an implementation MUST support the feature.

[ e ]        Denotes that an implementation MAY ELECT to support the feature.

[ N/A ]   Denotes that the feature is unavailable to this profile, i.e., it is either inappropriate for the profile or the functionality is addressed by a profile described in another column in the Table.

[ e* ]      Denotes that it is recommended that the Custom Interaction is launched using LTI.

 

[ ** ]      Delivery of Braille is somewhat non-standard at this time. Content and user profiles are required at the Core level to support Braille in anticipation of future delivery implementations.

All cells that are shaded (light red) denote functionality that will NOT be subjected to IMS conformance testing in the initial release of the IMS Assessment Conformance System.  Conformance testing of content instances will be undertaken using the corresponding Online IMS Validator and will ONLY address the validity of the XML with respect to the APIP specification: semantic integrity between the accessibility data and the original content will NOT be addressed. Conformance test of systems will be through self-testing using an IMS provided set of APIP packages.

 

ID

FEATURE

QTI DELIVERY SYSTEM & AUTHORING SYSTEM

APIP AUTHORING SYSTEM

APIP DELIVERY SYSTEM

APIP PNP SYSTEM

QTIv2.1 Summative Entry

QTIv2.1 Formative Entry

QTIv2.1 Core

APIP Entry

APIP Core

APIP Entry

APIP Core

APIP PNP Entry

APIP PNP Core

Tests

T1

Outcome Declaration

R

R

R

R

R

R

R

N/A

N/A

T2

Time Limits

R

R

R

R

R

R

R

N/A

N/A

T2

Stylesheet

R

R

R

R

R

R

R

N/A

N/A

T4

Test Parts

R

R

R

R

R

R

R

N/A

N/A

T5

    Item Session Control

R

R

R

R

R

R

R

N/A

N/A

T6

    Time Limits

e

R

R

R

R

R

R

N/A

N/A

T7

    Sections

R

R

R

R

R

R

R

N/A

N/A

T8

    Test Feedback

X

X

e

X

e

X

e

N/A

N/A

T9a

Outcomes Processing - Common Cartridge

R

R

R

R

R

R

R

N/A

N/A

T9b

Outcomes Processing - Full

e

X

e

e

e

N/A

N/A

T10

Test Feedback

e

e

e

e

e

e

e

N/A

N/A

Sections

S1

Item Session Control

e

X

e

e

e

e

e

N/A

N/A

S2

Time Limits

e

X

e

e

e

e

e

N/A

N/A

S3

Selection

X

X

e

X

e

X

e

N/A

N/A

S4

Ordering

X

X

e

X

e

X

e

N/A

N/A

S5

Rubric Block

R

R

R

R

R

R

R

N/A

N/A

S6

Include

e

X

e

e

e

e

e

N/A

N/A

S7

Items

R

R

R

R

R

R

R

N/A

N/A

S8

Sections

R

R

R

R

R

R

R

N/A

N/A

Interactions

Q1

Associate Interaction

e

X

e

e

e

e

e

N/A

N/A

Q2

Choice Interaction

R

R

R

R

R

R

R

N/A

N/A

Q3

Custom Interaction

e

e

e

e*

e*

e*

e*

N/A

N/A

Q4

Drawing Interaction

e

X

e

e

e

e

e

N/A

N/A

Q5

End Attempt Interaction

e

X

e

e

e

e

e

N/A

N/A

Q6

Extended Text Interaction

R

R

R

R

R

R

R

N/A

N/A

Q7

Gap Match Interaction

e

X

e

e

e

e

e

N/A

N/A

Q8

Graphic Associate Interaction

e

X

e

e

e

e

e

N/A

N/A

Q9

Graphic Gap Match Interaction

e

X

e

e

e

e

e

N/A

N/A

Q10

Graphic Order Interaction

e

X

e

e

e

e

e

N/A

N/A

Q11

Hot-spot Interaction

e

X

e

e

R

e

R

N/A

N/A

Q12

Hot-text Interaction

e

X

e

e

e

e

e

N/A

N/A

Q13

Inline Choice Interaction

e

X

e

e

e

e

e

N/A

N/A

Q14

Match Interaction

e

X

e

e

R

e

R

N/A

N/A

Q15

Media Interaction

e

X

e

e

e

e

e

N/A

N/A

Q16

Order Interaction

e

X

e

e

e

e

e

N/A

N/A

Q17

Position Object Interaction

e

X

e

e

e

e

e

N/A

N/A

Q18

Selection Point Interaction

e

X

e

e

e

e

e

N/A

N/A

Q19

Slider Interaction

e

X

e

e

e

e

e

N/A

N/A

Q20

Text Entry Interaction

R

R

R

R

R

R

R

N/A

N/A

Q21

Upload Interaction

e

X

e

e

e

e

e

N/A

N/A

Items

I1

responseDeclaration

R

R

R

R

R

R

R

N/A

N/A

I2

outcomeDeclaration

R

R

R

R

R

R

R

N/A

N/A

I3

templateDeclaration

e

X

e

e

e

e

e

N/A

N/A

I4

templateProcessing

e

X

e

e

e

e

e

N/A

N/A

I5

stylesheet

R

R

R

R

R

R

R

N/A

N/A

I6

Item Body

R

R

R

R

R

R

R

N/A

N/A

I7

    HTML (Block)

R

R

R

R

R

R

R

N/A

N/A

I8a

Response Processing-Fixed Template

R

R

R

R

R

R

R

N/A

N/A

I8b

Response Processing-Full

e

X

e

e

e

e

e

N/A

N/A

I9

Modal Feedback

e

R

R

e

e

e

e

N/A

N/A

I10

Inline Feedback

e

X

e

e

e

e

e

N/A

N/A

I11

APIP Accessibility

N/A

X

e

R

R

R

R

N/A

N/A

Accessibility

A1

Spoken, Text Only

N/A

N/A

e

e

R

e

R

e

R

A2

Spoken, Text & Graphics

N/A

N/A

e

R

R

R

R

R

R

A3

Spoken, Non-Visual

N/A

N/A

e

e

R

e

R

e

R

A4

Spoken, Graphics Only

N/A

N/A

e

e

e

e

e

e

e

A5

Spoken, Directions Only

N/A

N/A

e

e

e

e

e

e

e

A6

Spoken, user preferences, including recording speed (slow, medium, fast) and voice type (human or synthetic)

N/A

N/A

e

e

e

e

e

e

e

A7

Screen Reader preferences (usage, speech rate, pitch, etc., for synthetic text-to-speech)

N/A

N/A

e

e

e

e

e

e

e

A8

Braille**

N/A

N/A

e

e

R

e

e

e

R

A9

Braille user preferences (grade, dots, cells, etc.)

N/A

N/A

e

e

e

e

e

e

e

A10

Tactile

N/A

N/A

e

e

R

e

e

e

R

A11

Signing: ASL

N/A

N/A

e

e

e

e

e

e

e

A12

Signing: Signed English

N/A

N/A

e

e

e

e

e

e

e

A13

Item Translation

N/A

N/A

e

e

e

e

e

e

e

A14

Keyword Translation

N/A

N/A

e

e

e

e

e

e

e

A15

Simplified Language

N/A

N/A

e

e

e

e

e

e

e

A16

Alternate Representation

N/A

N/A

e

e

e

e

e

e

e

B1

Magnification

N/A

N/A

N/A

N/A

N/A

R

R

R

R

B2

Magnification, Amount (user preference)

N/A

N/A

N/A

N/A

N/A

e

e

e

e

B3

Reverse Contrast

N/A

N/A

N/A

N/A

N/A

e

e

e

e

B4

Alternate Text and Background Colors

N/A

N/A

N/A

N/A

N/A

R

R

R

R

B5

Alternate Text and Background Colors, Color Preferences

N/A

N/A

N/A

N/A

N/A

e

e

e

e

B6

Color Overlay

N/A

N/A

N/A

N/A

N/A

e

e

e

e

B7

Color Overlay, Color Preferences

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C1

Answer Masking

N/A

N/A

N/A

N/A

N/A

e

R

e

R

C2

Masking

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C3

Auditory Calming (background music)

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C4

Additional Testing Time (including time & additional testing time tags)

N/A

N/A

N/A

N/A

N/A

e

R

e

R

C5

Breaks

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C6

Keyword Emphasis

N/A

N/A

e

e

e

e

e

e

e

C7

Line Reader

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C8

Line Reader, Color Preferences

N/A

N/A

N/A

N/A

N/A

e

e

e

e

C9

Language Learner Guidance

N/A

N/A

e

e

e

e

e

e

e

C10

Cognitive Guidance

N/A

N/A

e

e

e

e

e

e

e

D1

Calculator Companion Material

N/A

N/A

e

e

e

e

e

N/A

N/A

D2

Rule Companion Material

N/A

N/A

e

e

e

e

e

N/A

N/A

D3

Protractor Companion Material

N/A

N/A

e

e

e

e

e

N/A

N/A

D4

Reading Passage Companion Material

N/A

N/A

e

e

e

e

e

N/A

N/A

D5

Digital Companion Material

N/A

N/A

e

e

R

e

R

N/A

N/A

D6

Physical Companion Material

N/A

N/A

e

e

e

e

e

N/A

N/A

Results

R1

Context

N/A

X

e

N/A

N/A

N/A

N/A

N/A

N/A

R2

Test result

N/A

X

e

N/A

N/A

N/A

N/A

N/A

N/A

R3

Item Result

N/A

X

e

N/A

N/A

N/A

N/A

N/A

N/A

QTI Metadata

M1

Item Template

e

R

R

e

e

e

e

N/A

N/A

M2

Time Dependent

e

R

R

e

e

e

e

N/A

N/A

M3

Composite

e

R

R

e

e

e

e

N/A

N/A

M4

Interaction Type

e

R

R

e

e

e

e

N/A

N/A

M5

Feedback Type

e

R

R

e

e

e

e

N/A

N/A

M6

Solutions Available

e

R

R

e

e

e

e

N/A

N/A

M7

Tool Name

e

X

e

e

e

e

e

N/A

N/A

M8

Tool Version

e

X

e

e

e

e

e

N/A

N/A

M9

Tool Vendor

e

X

e

e

e

e

e

N/A

N/A

Packaging

P1

Test Instances

R

R

R

R

R

R

R

N/A

N/A

P2

Section Instances

R

R

R

R

R

R

R

N/A

N/A

P3

Item Banks

e

R

R

e

R

e

R

N/A

N/A

P4

Item Instances

R

R

R

R

R

R

R

N/A

N/A

P5

Manifest Metadata (LOM)

R

R

R

R

R

R

R

N/A

N/A

P6

Resource Metadata (LOM)

R

R

R

R

R

R

R

N/A

N/A

P7

Resource Metadata (QTI)

e

R

R

e

e

e

e

N/A

N/A

P8

Curriculum Standards Metadata

e

e

e

e

e

e

e

N/A

N/A

P9

Learning Tools Interop v1.1

e

e

e

e

e

e

e

N/A

N/A

 

Elective access features are those features that could be supported by systems, but are not required for conformance. Elective features will be individually certified, and systems that use elective features must list the specific APIP Elective access features they support. Either Entry or Core certified systems are eligible to be certified in any elective feature. There is no Elective certification without Entry or Core certification.

The certification categories include a differentiation between the ability to import and export APIP Content or PNP files. Since APIP is primarily a transfer format, it will not certify how systems manage or modify data within the system itself. Delivery Systems are expected to combine the information provided by APIP content packages and PNP files. Although they do not need to natively use the APIP format during delivery, certified delivery systems must make use of the data/information supplied by APIP.

Systems that import APIP information must distinguish between valid APIP formats and invalid formats, and provide information about which packages are invalid. Systems will minimally support APIP Entry access features, and possibly support APIP Entry access features, and possibly supporting APIP Core and APIP Elective access features.

Systems that export APIP content or PNP files must be able to supply valid APIP formatted instances, minimally supporting APIP Entry access features, and possibly supporting APIP Core and APIP Elective access features.

4                  Achieving Certification

Validation for APIP conformance will consist of the following strategies:

 

Because the APIP standard is primarily a data transfer format, conformance and certification does not govern how applications manage or modify data within the system itself. For example, delivery systems must be able to combine the information provided by imported content and PNP files, but need not natively use the APIP format during delivery. There are limits to what can be verified in an automated fashion. In the categories listed in the prior section as an example, it makes a clear distinction between import and export. The reason for this is that export functions are much more easily verified using automation. The output from an export process must adhere to the APIP XML specifications and can be verified using automation. Import, on the other hand, is much more difficult to verify because the import process results in a proprietary system consuming the information and representing it in an internal format that does not need to adhere to the XML specification and cannot be verified using automation.  Each of these approaches is discussed in detail below.

4.1            Automated Validation

Capabilities to be validated using automated tools:

·         APIP Content Package export

·         APIP PNP file export

For the export functional context, the APIPv1 conformance approach focuses on determining if the system can generate an object (PNP file or item) that is non-problematic. IMS provides an online tool available for individuals or organization to submit an object (PNP file, item) and then receive information about the existence (and sometimes nature) of problems with syntax and semantics of the object.

It is expected that the automated validation tool will:

1)      Accept Entry, Core, and Elective APIP conformance content elements identified in the conformance matrix.

2)      Accept custom (and profiled) extensions that are agreed to between exchanging parties (interdependent vendors)

3)      Identify which Core and Elective access features as well as the QTI v2.1 features are implemented in the supplied instance

4.2            Test Case Based Validation

Capabilities to be validated using a standard set of test cases:

·         APIP Content Package import

·         APIP PNP file import

The APIP Alliance makes available sets of objects (PNP files, items) — some that are problematic and others that are non-problematic — for conducting import validation. Import functionality focuses on determining if the system can (a) flag as problematic objects that are known to have problems in syntax or semantics and (b) recognize as non-problematic the objects that are known to be without problems.

4.3            Documented or Demonstrable Validation

Capabilities to be documented and/or demonstrated by the solution provider:

As each test delivery system may provide a unique user experience and use different technologies to deliver to content, providing automated validation of each implementation is virtually impossible. Therefore, the IMS APMG will rely on the solution provider to document that the required APIP content and PNP features have been implemented and can document and/or demonstrate them. The documents and demo systems can be submitted to the APMG for consideration and review but the APMG will not be able verify the exact system behaviors.

 

Appendix A – Accessibility Features

 Online Accessibility can be thought of as consisting of four major categories for supporting accessibility:

A.    Accessibility through Alternate Representations

B.     Accessibility through Adapted Presentations

C.     Accessibility through Adapted Interactions

D.    Accessibility through Adapted Response Modes

Future versions of the APIP standard may (a) include access features for category D (Adapted Response Modes), (b) add new access features to approaches A, B, and C, and (c) change the core/elective designation of a feature (e.g., for the delivery area, change Braille from elective to core).

The complete list of APIP v1 access features are listed below, organized by stated categories above.

A: Alternate Representations

Spoken Support (sometimes called read aloud or verbal): Text presented to the user is spoken aloud. Graphics (tables/diagrams/pictures) would have alternate text that could be spoken aloud.

A1    Spoken, Text Only: Only text (alpha-numeric) content will have spoken support. At the discretion of the author, some text may have enhanced spoken information, or may not be included (like reading numbers in a math assessment). Graphics would NOT be described to this user.

A2    Spoken, Text and Graphics: Text content as well as graphic content will be described to the user. Some more contextual information about layout or object orientation may be added for this user, depending on the content.

A3    Spoken, Non-Visual: All content is read and described to the user, as the expectation is that the user cannot see the content. Contextual information about layout or object orientation may be needed for some content. Special consideration may be needed for the order in which the information is presented.

A4    Spoken, Graphics Only: Only graphic descriptions would be read aloud to this examinee. Examinees access this information on-demand only (at the examinee’s request).

A5    Spoken, Directions Only: Only test directions are read aloud to the user.

A6    Recording Speed (Slow, Medium, Fast): the rate of speech that a recorded audio file presents spoken verbal information. Systems that can adjust the rate of speech for pre-recorded audio files at the moment of delivery might adjust the rate based on the Medium (or regular) rate of speech.

         Voice Type (Human or Synthetic): whether a pre-recorded audio file was created by a human reading text, or generated by computer speech emulation (Synthetic).

               For Content compliance, content would provide specific information about the sound files provided within the content, allowing user preferences about the type of audio representation to be considered for use during presentation. For PNP compliance, the system would store the user’s spoken preferences. For Delivery compliance, the system would provide the user’s spoken preferences (using their Personal Needs Profile information) to the delivery system when the testing session is initiated.

A7    Screen Reader Preferences (usage, speech rate, pitch, etc.): preferred characteristics of computer generated voice for the examinee. For PNP compliance, the system would store the user’s synthetic spoken preferences. For Delivery compliance, the system provides specific synthetic spoken preferences (using their PNP information) to the delivery system when the testing session is initiated.

Braille: Content would have specific text strings to be used in a refreshable Braille display device. The device is expected to convert the text into Braille.

A8    Braille: All content is presented as Braille characters and described to the user, as the expectation is that the user cannot see the content. Contextual information about layout or object orientation may be needed for some item or test content.

A9    Braille user preferences (grade, dots, cells, etc.): Allows the system to provide the user’s preferred Braille preferences (using their Personal Needs Profile information) to the delivery system for use with a refreshable Braille display.

Tactile: An external, physical representation of parts of the item content, or the item’s companion materials.

A10  Tactile: A tactile representation of the graphic information is made available outside of the computer testing system. The access element tags should include descriptions of how to locate the specific tactile sheet (using referencing numbers/letters/symbols) needed to respond to the item. This support is simply a reference to tactile materials for the examinee. It does not provide a description of how to form/create the physical, tactile representation.

Sign Language: Animated or live-action movie recordings are provided to the user.

A11  Signing Type: ASL: Additional American Sign Language content is provided to the user. Unlike other translated content, ASL content is presented in addition (or in parallel) to the default content.

A12  Signing Type: Signed English: Additional Signed English content is provided to the user. Unlike other translated content, Signed English is presented in addition (or in parallel) to the default content.

Item Translation: A different language variant of the same item.

A13  Item Translation: An alternate item is made, and the user is exposed to the alternate language version. The content package would contain which specific language it is providing.

Keyword Translation: parts of item or test content are translated and available to specific examinees.

A14  Keyword Translation: Certain specific words or phrases would have translations available to users who need some assistance with difficult or important words in the content. The examinee profile would specify the language requested, and the content would supply the translation for the program-required languages.

Simplified Language: An entirely different variant of the item is given to the user, using simpler language.

A15  Simplified Language: the examinee is presented with the Simplified Language variant of the item (if available). Not at items in a particular assessment may need simplification. The default content may be considered accessible by examinee’s requiring Simplified Language support.

Alternate Representation: Specifies an alternate way of displaying content to facilitate the users access to the content. As an example, a text-based description of a figure displaying a life cycle might be provided, or an animation that represents a series of events described in text might be provided.

A16  Alternate Representation: present a text string alternative of the original representation.

 

B: Adapted Presentations

Magnification: All content is magnified by the amount specified by the user.

B1    Magnification: Provides the capability to magnify content

B2    Magnification, Amount: For PNP compliance, the system stores the user’s preferred magnification amount. For Delivery compliance, the system provides the user’s preferred magnification amount (using the PNP information) when the testing session is initiated.

Color Adjustments: specific changes are made to the interface at the request of the user that allows them to visually access the content or interface.

B3    Reverse Contrast: inverts all the color values in the assessment environment. Black is white, white is black, Red is Green, Blue is Orange, etc.

B4    Alternate Text and Background Colors: The interface allows the user to select alternate colors for the text (foreground) and background colors.

B5    Color Preferences for Alternate Text and Background Colors: the stored, specific color preferences of the examinee. For PNP compliance, the system stores the user’s color preferences for the text and background.  For Delivery compliance, the system provides the user’s preferred color choices (using their PNP information) to the assessment delivery system when the testing session is initiated.

B6    Color Overlay: A color tint is layered over the content to aid in reading the item and test content. It emulates the classroom practice of using color Acetate (transparent plastic) sheets over paper.

B7    Color Preferences for Color Overlay: the stored, specific color preferences of the examinee. For PNP compliance, the system stores the user’s color preferences.  For Delivery compliance, the system provides the user’s preferred color choice (using their PNP information) for the overlay to the assessment delivery system when the testing session is initiated.

 

C: Adapted Interactions

Masking: Portions of the content and interface may be covered to reduce the amount of stimulus provided to the student during testing.

C1    Answer Masking: By default, answer choices for multiple-choice items are covered when the item is first presented. The user has the ability to remove the masks at a time of their choosing.

C2    Masking: The user is able to create their own masks to cover portions of the item until needed.

Auditory Calming: The examinee can listen to music or sounds in the background as they respond to questions during the testing session.

C3    Auditory Calming: Background music or sounds

Additional Testing Time: If a test has a time limit, the user will be allowed additional time (specific additional amounts or indefinite) to complete the test.

C4    Additional Testing Time: time & additional testing time are provided by the examinee’s user profile. This support is provided without the ability for the examinee to alter or remove the support.

Breaks: User is allowed to take breaks, at their request, during the testing session.

C5    Breaks

Keyword Emphasis: Certain words are designated in the content as key words. Contracting programs would designate how they are to be emphasized (bold, italic, etc.)

C6    Keyword Emphasis

Line Reader: Examinee has an on-screen tool available that assists them in moving a reading tool (line highlighter or underscore) down (or up) the block of text, line by line, to assist in reading the content.

C7    Line Reader: Examinee has access to a Line Reader Tool.

C8    Color Preferences for Line Reader: the stored, specific color preferences of the examinee. For PNP compliance, the system stores the user’s color preferences.  For Delivery compliance, the system provides the user’s preferred color choice (using their PNP information) for the color overlay in the Line Reader Tool to the assessment delivery system when the testing session is initiated. Note: not all Line Reader Tools use color to help highlight or guide reading.

Guidance: The examinee is provided with supplemental information about all or portions of the item/test content to aid in their processing of the presented information.

C9    Language Learner: Additional information is provided in the test language about words or phrases that is intended to assist the Language Learner process those words or phrases.

C10  Cognitive Guidance: Additional information is provided to assist an examinee process or understand all or part of an item. Guidance may include how they are expected to respond to an item (e.g., additional instructions that emphasize that the student should use the information presented in a graph to answer the question).

 

D: Companion Materials

D1    Calculator: All Calculators would have access to all number digits, decimal key, equals button, and Clear button. Read aloud capability should be something that is configurable to either allow or not allow during testing. Additionally, some institutions allow for reading the numbers or functions as you use them, but do not allow reading the number as a whole. This is usually for Math related content. The four possible calculators that can be specified are Basic, Standard, Scientific, and Graphing.

D1a        Basic Calculator: Assumed functions: Add, Subtract, Multiply, Divide.

D1b        Standard Calculator: Assumed functions: all basic calculator functions, Square root, Percentage (%) , Plus/Minus, a.k.a. Sign Change, Memory Functions

D1c        Scientific Calculator: Assumed functions may include, but are not limited to: all standard calculator functions, a π key, sign change (+/-), square (x2) , cube (x3),  x to the y (xy), square root (√) , cube root , xth root , logarithm keys, log, ln, base 10, base e, Trigonometry function keys with an INVERSE key for the inverse functions, sin, cos, tan, hsin (hyperbolic sin), hcos, (hyperbolic cos), htan (hyperbolic tan), DEG, RAD, GRAD conversion, a capacity to work in both degree and radian mode, a reciprocal key (1/x) – calculate the inverse of the displayed value, permutation and/or combination keys (nPr , nCr), parentheses keys, metric conversion, permutation and combination keys, nPr,  cPr, x!

D1d        Graphing Calculator: A Graphing calculator includes many of the same functions of a scientific calculator, plus the ability to display equations graphically.

D2    Rule: Allows for the presentation of a measuring device for use on the computer with the supplied content. The ruling system can be specified as metric (SI) or US measurement systems.

D3    Protractor: The examinee will be supplied with an on-screen protractor while responding to the item. Increments can be specified as metric (radians) or US (degrees) systems of angular measurement.

D4    Reading Passage: Allows a reference to a reading passage that needs to be provided to the examinee while responding to an item.

D5    Digital Materials: Content or reference materials that relate to the item content. Examples could be a map, a table of information, a sheet of math formulas, an interactive periodic table of elements, or even graphic creation tools.

D6    Physical Materials: External materials needed to work with, or respond with, when the examinee responds to the item.

Appendix B – RFP Language for APIP

A successful accessible assessment program requires the coordination of assessment content, learner preferences and the supporting assessment delivery technology platforms. When considering how to specify the requirements for your programs, all three must be considered together. When choosing specific accessibility features for your assessment program, you will need to consider how each feature will impact these three areas.

Every assessment program is unique. What requirements that you identify for your program will take precedence over any requirements provided in this sample text. APIP compliance, as defined by IMS Global, pertains to the standard implementation itself. The contract you have with your service provider will ultimately dictate the obligations your service provider must achieve.

Close consideration should also be given to paper-based, online, and mixed mode assessment programs and how each may impact accessibility requirements. Some accessibility options, required or elective, will have cost implications for the development and implementation of content and processing systems. Special consideration should be given for programs that plan to repurpose existing content repositories that currently do not provide the necessary accessibility information. In addition, an existing repository may not be formatted in the QTI v2.1 format. You may wish to provide pricing options so that your vendors can identify those items separately.

[NOTE] – For text below that is enclosed in square brackets, it is assumed that the requesting organization will modify or insert text as appropriate for your program.

The suggested RFP language follows.

B1 – Accessible Portable Item Protocol Standard

The Accessible Portable Item Protocol (APIP) Standard is a freely available specification for standardizing the exchange file format of digital test items for enabling accessible assessments. Version 1.0 of APIP is scheduled for release by Fall 2012. By extending the QTI v2.1 standard, APIP allows accessibility information to be included in a digital test item definition so that items can be transferred between APIP compliant item banks.

When combined with an individual learner’s accessibility needs and preferences, an assessment delivery system can implement interfaces required to make assessments accessible to learners with a variety of disabilities and special needs. APIP offers a tagging map to the IMS Personal Needs and Preference (PNP) standard that can seamlessly link a learner’s accessibility information to assessment content.

In summary, the response must describe how the assessment solution will approach [please remove any that are not included in this RFP/RFI/other]:

The APIP Conformance documentation provides for a table of QTI, PNP, and APIP elements (tags) with a designation of the element being an Entry level or a Core level element. The Entry level indicates those elements that are expected, at a minimum, to be in place for any APIP compliant application. The Core level entries provide for “implementation electives” that can be provided by the application or can be added as required elements for any specific program. The conformance documentation can be found at http://www.imsglobal.org/apip.

[The requesting organization should review the list of elements in the conformance documentation and note any program specific variations that may apply to your program. This may include removing unnecessary, unwanted, or not-applicable features for your program or designating elective features as required for your program.]

The response should include the conformance matrix from this APIP Conformance document with a description of how each feature (excluding any features noted above) is addressed by the solution being proposed.

B2 – Compliant Content Format

The APIP item standard includes provisions for accessibility information that extends the default representation of item content (QTI v2.1) to include a large variety of access features that support the default representation.

The content should support specific inclusion orders and associated access elements in the content for the following accessibility categories [remove any that do not apply or note as optional].

The following list identifies areas where full accessibility compliance may be excluded by an issuer of an RFP. The responder should articulate how such content will be identified in each category [remove any that do not apply or note as optional; note: while contract exceptions may be allowed by an issuer of an RFP, compliance for systems or instances will not be permitted in the certification process].

B3 – Learner Information Profiles

The assessment administration system must be able to accept learner information files containing the required accessibility components as defined in the matrices above. The [state/districts] will provide the learner’s accessibility profile information in the following ways [remove any that do not apply or note as optional]:

The [vendor or state] will provide instructions and additional resources to the local districts and schools needing to generate the necessary profile data or files as part of the assessment registration or supplemental process.

B4 – Digital Test Delivery System

The digital test delivery system employed to deliver test items to learners must have all the required features and functions for those elements identified in this [RFP/RFI/other]. The test delivery system must be able to present the specified accessibility features based on the learner’s preferences. Specifically, the test delivery system must be able to provide the accessibility features as defined in the tables above. In addition, the test delivery system must allow learners to interact with the system using a standard compliant point device (mouse), keyboard (standard and alternate), touch screen, and/or a tab-enter enabled device. If the delivery system requires additional hardware components or is incompatible with specific devices, the responder should specify those requirements in the response. Responders must identify the supported technology delivery platforms (hardware requirements and software with versions) and any limitations or considerations that must be given for accessibility features.

[Note 1: You may wish to also state expectations. Possible examples are one or more of the functional performance criteria of the section 508 standards (subpart C; http://www.access-board.gov/sec508/standards.htm), which might be adapted to an assessment delivery setting.]

[Note 2: Braille support may be delivered using refreshable Braille technology or with paper-based Braille supports. If you want capabilities beyond those required by the standard (i.e., for online, paper-based, or hardware for refreshable Braille) they should be specified here.]

B5 – APIP Conformance Certification and Evolution

The IMS Global Learning Consortium is establishing a conformance testing and certification program for APIP. Conformance testing and certifications will be available in Fall 2012 for content import and export from assessment item banks. In addition, IMS manages the ongoing evolution of the underlying specifications, most notably Question & Test Interoperability (QTI) and the Personal Needs and Preferences (PNP).

All supported versions of QTI and APIP should be identified. Plans should also be defined that will address the processes, costs, and timelines for upgrading assessment content, learner profiles, and assessment technology platforms as newer version of QTI and APIP are released to the public.

About this Document

Title:                                       IMS APIP Conformance and Certification

Editors:                                  Colin Smythe (IMS Global) and Mark McKell (IMS Global)

Co-chairs:                              Gary Driscoll   ETS (USA), Thomas Hoffmann (Measured Progress) and Wayne Ostler (Pearson)

Version:                                  1.0

Version Date:                         26 March 2012

Release:                                  Draft 1.0

Status:                                    Candidate Final

Summary:                              The aim of the APIP Project is to use well established e-learning interoperability standards to enable the exchange of accessible assessment content between computer-based assessment systems, tools and applications.  Users of systems, tools and applications that adopt the APIP are able to use their accessible assessment content on a wide range of systems.

Revision Information:           First release.

Purpose:                                 This document is made available for adoption by the public community at large.

Document Location:             http://www.imsglobal.org/apip/

 

List of Contributors

The following individuals contributed to the development of this document:

Rob Abel            IMS Global (USA)

Devin Loftis       McGraw-Hill/CTB (USA)

Gary Driscoll      ETS (USA)

Mark McKell      IMS Global (USA)

Eric Hansen        ETS (USA)

Wayne Ostler      Pearson (USA)

Casey Hill           ETS (USA)

Zack Pierce         Measured Progress (USA)

Regina Hoag       ETS (USA)

Mike Russell       Measured Progress (USA)

Thomas Hoffmann      Measured Progress (USA)

Colin Smythe      IMS Global (UK)

Revision History

 

Version No.

Release Date

Comments

Candidate Final v1.0

26 March 2012

The first formal release of the Candidate Final version of this document.

 

 

 

 

 

 

 

 

 

IMS Global Learning Consortium, Inc. (“IMS Global”) is publishing the information contained in this document (“Specification”) for purposes of scientific, experimental, and scholarly collaboration only.

IMS Global makes no warranty or representation regarding the accuracy or completeness of the Specification.

This material is provided on an “As Is” and “As Available” basis.

The Specification is at all times subject to change and revision without notice. 

It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.

IMS Global would appreciate receiving your comments and suggestions.

Please contact IMS Global through our website at http://www.imsglobal.org.

Please refer to Document Name: IMS APIP Conformance and Certification v1.0
Candidate Final v1.0

Date: 26 March 2012