IPR and Distribution Notices
Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.
IMS takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on IMS’s procedures with respect to rights in IMS specifications can be found at the IMS Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.
Copyright © 2018 IMS Global Learning Consortium. All Rights Reserved.
Use of this guide to develop products or services is governed by the license with IMS found on the IMS website: http://www.imsglobal.org/speclicense.html.
Permission is granted to all parties to use excerpts from this document as needed in producing requests for proposals.
The limited permissions granted above are perpetual and will not be revoked by IMS or its successors or assigns.
THIS GUIDE IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NON INFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS GUIDE SHALL BE MADE ENTIRELY AT THE IMPLEMENTER’S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS GUIDE.
The Caliper Analytics® Specification, version 1.1, provides a structured approach to describing, collecting and exchanging learning activity data at scale. Establishing a common vocabulary for describing learning interactions is a central objective. Promoting data interoperability, data sharing and data-informed decision making are also important goals.
Caliper also defines an application programming interface (the Sensor API™) for marshalling and transmitting event data from instrumented applications to target endpoints for storage, analysis and use. Industry-wide adoption of Caliper offers the tantalizing prospect of a more unified learning data environment in which to build new and innovative services designed to measure, infer, predict, report and visualize.
This document is the certification guide for Caliper Sensors. In this release of the Caliper specification, certification services are not provided for Caliper endpoints. Endpoint implementors should take note that it is the intent of the Caliper Working Group to add endpoint certification in forthcoming releases of the specification. Implementors should also note that behavioral requirements for Caliper Endpoints are provided in the Caliper Analytics® Specification, version 1.1, Section 6.0.
This document is considered the Final Release. This means that the Caliper Analytics® Sensor Certification Guide, version 1.1, is now made available as a public document following acceptance by IMS Global member organizations, a number of whom have successfully achieved conformance certification at the time of the release of this document.
IMS Global strongly encourages its members and the greater public to provide feedback that focuses on improving the Caliper specification. To join the IMS developer and conformance certification community focused on Caliper please visit https://www.imsglobal.org/activity/caliper.
Public comments and questions can be posted at the Caliper Analytics® public forum.
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119. A Sensor implementation that fails to implement a MUST/REQUIRED/SHALL requirement or fails to abide by a MUST NOT/SHALL NOT prohibition is considered nonconformant. SHOULD/SHOULD NOT/RECOMMENDED statements constitute a best practice. Ignoring a best practice does not violate conformance but a decision to disregard such guidance should be carefully considered by implementers. MAY/OPTIONAL statements indicate that implementers are entirely free to choose whether or not to implement the option.
Blank Node Identifier: a string that begins with “_:” that is used to identify an Entity for which an IRI is not provided. An Entity provisioned with a blank node identifier is neither dereferenceable nor has meaning outside the scope of the JSON-LD document within which it resides.
Action: something performed or done to accomplish a purpose. Caliper Event subtypes define a controlled vocabulary of one or more actions relevant to the activity domain. A Caliper Event includes an
action attribute for expressing the associated action.
Context: a special JSON-LD keyword that maps the terms employed in a JSON document to IRIs that link to one or more published vocabularies. Inclusion of a JSON-LD context provides an economical way of communicating document semantics to services interested in consuming Caliper event data.
Describe: a Caliper message containing an Entity that is not directly associated with an Event. Entities can be sent asynchronously from events using
Describe messages in order to reduce verbosity (e.g. sending a Person entity as a
Describe avoids having to repeat the Person object in each Event that includes it).
Entity: an object or a thing that participates in learning-related activity. Caliper Entity types provide coarse-grained representations of applications, people, groups and resources that constitute the “stuff” of a Caliper Event. Each Entity corresponds to a node in a directed graph.
Event: describes a relationship established between an Agent (the
actor) and an Entity (the
object) formed as a result of a purposeful
action undertaken by the
actor in connection to the
object at a particular moment in time.
JSON-LD: a specification providing a JSON-based data serialization and messaging format, processing algorithms and API for working with Linked Data. The messages described in this guide are intended to be used in programming environments that support JSON-LD.
IRI: The Internationalized Resource Identifier (IRI) extends the Uniform Resource Identifier (URI) scheme by using characters drawn from the Universal character set rather than US-ASCII per RFC 3987. Linked Data rely on IRIs to refer to most nodes and properties.
ISO 8601: Caliper data and time values are formatted per ISO 8601 with the addition of millisecond precision. The format is yyyy-MM-ddTHH:mm:ss.SSSZ where ‘T’ separates the date from the time while ‘Z’ indicates that the time is set to UTC.
Linked Data: A set of design principles first articulated by Tim Berners-Lee for discovering, connecting, and sharing structured data over the Web. The principles can be summarized as follows: use IRIs/URIs as names for things; use HTTP IRIs/URIs so that information about things (e.g., people, objects, concepts) can be retrieved using a standard format; link out to other relevant things by way of their IRIs/URIs in order to promote discovery of new relationships between things.
Metric Profile: models a learning activity or a supporting activity that helps facilitate learning. Each profile provides a domain-specific set of terms and concepts that application designers and developers can draw upon to describe common user interactions in a consistent manner using a shared vocabulary.
Sensor: Software assets deployed within a learning application that implement the Sensor API™ for marshalling and transmitting Caliper data to the certification service endpoint.
Sensor API™: The standard set of methods and supported parameters that a Sensor implements according to the Caliper Analytics® Specification, version 1.1, Section 5.0 in order to transmit Caliper data in an interoperable way.
Term: a word or short expression that expands to an IRI when mapped to a JSON-LD context document. Terms are employed by Caliper as
type property string values in order to distinguish between various JSON representations of entities and events defined by the Caliper information model.
URI: the Uniform Resource Identifier (URI) utilizes the US-ASCII character set to identify a resource. Per RFC 2396 a URI “can be further classified as a locator, a name or both.” Both the Uniform Resource Locator (URL) and the Uniform Resource Name (URN) are considered subspaces of the more general URI space.
URL: A Uniform Resource Locator (URL) is a type of URI that provides a reference to resource that specifies both its location and a means of retrieving a representation of it. An HTTP URI is a URL and using HTTP IRIs/URIs to identify things is fundamental to Linked Data.
URN: A Uniform Resource Name (URN) is a type of URI that provides a persistent identifier for a resource that is bound to a defined namespace. Unlike a URL a URN is location-independent and provides no means of accessing a representation of the named resource.
UUID: a 128-bit identifier that does not require a registration authority to assure uniqueness. However, absolute uniqueness is not guaranteed although the collision probability is considered extremely low. Caliper recommends use of randomly or pseudo-randomly generated version 4 UUIDs. Each Caliper Event MUST be assigned a UUID that is expressed as a URN using the form
urn:uuid:<UUID> as described in RFC 4122.
Certain prerequisites MUST be met before you can certify your platform, application or service as Caliper compliant.
As described more fully in the Caliper Analytics® Specification, version 1.1, the Caliper information model defines a number of metric profiles, each of which models a learning activity or a supporting activity that helps facilitate learning. Each profile provides a domain-specific set of terms for describing common user interactions.
Each Caliper profile is also a unit of certification for Caliper Sensor implementations. Any given Sensor may apply for certification for one or more of the Caliper Metric Profiles. In the subsections below, the Minimum Conformance and Restrictions sections specified for each Profile defines the corresponding conformance criteria in detail.
@contextkeyword and value as required.
The Caliper Annotation Profile models activities related to the annotation of a DigitalResource.
See Caliper Analytics® Specification, version 1.1, Section 3.1.
Create and send a “Bookmarked” AnnotationEvent to the certification service endpoint. All other actions included in the profile are considered optional for certification purposes.
DigitalResource or subtype
The Caliper Assessment Profile models assessment-related activities including interactions with individual assessment items.
See Caliper Analytics® Specification, version 1.1, Section 3.2.
Create and send a “Started” AssessmentEvent followed by a “Submitted” AssessmentEvent to the certification service endpoint. All other event types and actions included in the profile are considered optional for certification purposes.
The Assignable Profile models activities associated with the assignment of digital content to a learner for completion according to specific criteria.
See Caliper Analytics® Specification, version 1.1, Section 3.3.
Create and send a “Started” AssignableEvent followed by a “Submitted” AssignableEvent to the certification service endpoint. All other event types and actions included in the profile are considered optional for certification purposes.
The Caliper Forum Profile models learners and others participating in online forum communities.
See Caliper Analytics® Specification, version 1.1, Section 3.4.
Create and send a “Posted” MessageEvent to the certification service endpoint. All other event types and actions included in the profile are considered optional for certification purposes.
See Caliper Analytics® Specification, version 1.1, Section 3.5.
Create and send a “Graded” GradeEvent to the certification service endpoint. All other actions included in the profile are considered optional for certification purposes.
Agent or subtype
The Caliper Media Profile models interactions between learners and rich content such as audio, images and video.
See Caliper Analytics® Specification, version 1.1, Section 3.6.
Create and send a “Started” MediaEvent followed by an “Ended” MediaEvent to the certification service endpoint. All other event types and actions included in the profile are considered optional for certification purposes.
The Caliper Reading Profile models activities associated with navigating to and viewing digital textual content.
See Caliper Analytics® Specification, version 1.1, Section 3.7.
See Caliper Analytics® Specification, version 1.1, Section 3.8.
Create and send a “LoggedIn” SessionEvent to the certification service endpoint. All other actions included in the profile are considered optional for certification purposes.
The Caliper Tool Use Profile models an intended interaction between a user and a tool.
See Caliper Analytics® Specification, version 1.1, Section 3.9.
Create and send a “Used” ToolUseEvent to the certification service endpoint.
The Caliper Basic Profile provides a generic Event for describing learning or supporting activities that have yet to be modeled by Caliper.
See Caliper Analytics® Specification, version 1.1, Section 3.10.
Create and send a generic Caliper Event to the certification service endpoint.
Agent or subtype
Any Caliper defined action can be used to describe the interaction.
Entity or subtype
Use of the Basic Profile is limited to describing interactions not modeled in other profiles. Any events described MUST be expressed using only the Event supertype.
Caliper events and entities are serialized as JSON-LD, a JSON-based data interchange format that encourages use of shared vocabularies and discoverable key:value identifiers when constructing JSON documents.
JSON-LD documents require inclusion of a context, denoted by the
@context keyword, a property employed to map document Terms to IRIs. JSON-LD contexts can be embedded inline or referenced externally in a document. Inclusion of a JSON-LD context provides an economical way for Caliper to communicate document semantics to services interested in consuming Caliper event data.
IMS Global provides a remote Caliper 1.1 JSON-LD context document (http://purl.imsglobal.org/ctx/caliper/v1p1) for mapping Caliper Terms to IRIs. Implementers are encouraged to familiarize themselves with the term definitions described therein.
@contextdefined as a property of the top-level object.
@contextproperty type MUST be defined as a string or an array.
@contextvalue is defined as a string it MUST be set to the Caliper remote context URL “http://purl.imsglobal.org/ctx/caliper/v1p1”.
@contextvalue is defined as an array of multiple contexts, the remote Caliper JSON-LD context MUST be listed last in order to ensure that Caliper terms retain their primacy given that JSON-LD parsers rely on a “most-recently-defined-wins” approach when evaluating duplicate terms.
@contextis mandatory. The terms it defines MUST NOT be defined inline as an object.
Caliper specifies the use of IRIs for identifying nodes (i.e., the things being described) and their attributes. IRI values MUST be absolute containing a scheme, path and optional query and fragment segments. A URI employing the URN scheme MAY be used as an identifier although care should be taken when employing a location-independent identifier since it precludes the possibility of utilizing it in future to retrieve machine-readable data over HTTP. If an IRI is deemed inappropriate for the resource a blank node identifier may be assigned.
For additional information regarding identifiers see Caliper Analytics® Specification, version 1.1, Section 4.2.
Caliper permits Entity values to be expressed either as a JSON object or as a string corresponding to its IRI. JSON-LD also supports the coercion of data values to specified types based on value type mappings defined in a JSON-LD context. For a given
@type the keywords
@vocab may be assigned as a value in order to signal to a JSON-LD parser that if a term’s instance value is set to a string it is to be interpreted as an IRI. Type coercion of this sort provides representational flexibility that implementers are encouraged to leverage.
For examples of coerced Caliper values see Caliper Analytics® Specification, version 1.1, Section 4.3.
A Caliper Event is a generic type that describes the relationship established between an
actor and an
object, formed as a result of a purposeful action undertaken by the
actor at a particular moment in time and within a given learning context. Caliper defines a number of Event subtypes, each scoped to a particular activity domain and distinguishable by a
type attribute. Considered as a JSON data structure an Event constitutes an unordered set of key:value pairs that is semi-structured by design.
eventTime properties are required and MUST be specified; all other properties are optional and MAY be omitted when describing an Event. Adherence to the rules associated with each property specified is mandatory. Custom attributes not described by the model MAY be included but MUST be added to the
extensions property as a map of key:value pairs. Properties with a value of null or empty SHOULD be excluded prior to serialization.
|@context||A top-level JSON-LD context MUST be specified as described above in Section 4.1.|
|id||Set the value to a 128-bit long universally unique identifier (UUID) formatted as a URN per RFC 4122, which describes a URN namespace for UUIDs.|
|type||Set the string value to the relevant Caliper term (e.g., “AssessmentEvent”).|
|actor||Set to Agent or one of its subtypes (e.g., Person). The
|action||Set the string value to the relevant action term (e.g., “Started”) specified by the governing Metric Profile.|
|object||Set the value to the relevant Entity (e.g., Assessment) specified by the governing Metric Profile. The
|eventTime||Set the date and time value expressed with millisecond precision using the ISO 8601 format YYYY-MM-DDTHH:mm:ss.SSSZ set to UTC with no offset specified.|
A Caliper Entity is a generic type that represents objects that participate in learning-related activities. A variety of Entity subtypes have been defined in order to better describe people, groups, organizations, digital content, courses, software applications, and other objects that constitute the “stuff” of a Caliper Event. Like an Event, an Entity is considered semi-structured data consisting of an unordered set of key:value pairs.
If an Entity is expressed as a JSON object the
type properties are required and MUST be specified. If transmitted as a describe a top-level
@context MUST also be specified. Otherwise, omit the
@context when the Entity is specified as a value in an Event except in cases where custom terms are specified. See Section 4.1 above for more details regarding JSON-LD context handling.
All other properties are optional and MAY be omitted when describing an Entity. Adherence to the rules associated with each property referenced is mandatory. Custom attributes not described by the model MAY be included but MUST be added to the
extensions property as a map of key:value pairs. Properties with a value of null or empty SHOULD be excluded prior to serialization.
|@context||If the Entity is transmitted as a describe a top-level
|id||Set the string value to a valid IRI or a blank node identifier. The IRI MUST be unique and persistent. The IRI SHOULD be dereferenceable; i.e., capable of returning a representation of the Entity. A URI employing the URN scheme MAY also be utilized.|
|type||Set the string value to the relevant Caliper term (e.g., “DigitalResource”).|
A Caliper Sensor MUST demonstrate that it is capable of transmitting Caliper data successfully to the certification service Endpoint. Certification is limited to message exchanges using the Hypertext Transport Protocol (HTTP) with the connection encrypted with Transport Layer Security (TLS). Messages MUST be sent using the POST request method. For certification purposes, the Sensor MUST also support message authentication using the
Authorization request header, setting the value to the provided bearer token.
Caliper Event and Entity data are transmitted inside an Envelope, a JSON data structure that includes metadata about the emitting Sensor and the data payload. Each Event and Entity describe included in an envelope’s
data array MUST be expressed as a JSON-LD document.
data properties MUST be specified. No custom properties are permitted.
|sensor||Set the string value to a unique identifier assigned either to the Sensor or to the instrumented platform, application or service utilizing the Sensor. The identifier SHOULD be in the form of an IRI.|
|eventTime||Set the date and time value expressed with millisecond precision using the ISO 8601 format YYYY-MM-DDTHH:mm:ss.SSSZ set to UTC with no offset specified that indicates the time at which the Sensor issued the message.|
|dataVersion||Set the string value to the Caliper JSON-LD remote context URL “http://purl.imsglobal.org/ctx/caliper/v1p1”. This indicates that the Caliper Analytics® Specification, version 1.1, governs the form of the Caliper entities and events contained in the
|data||An ordered collection of one or more Caliper Event and/or Entity describe objects. The Sensor MAY mix Event and Entity describe data in the same Envelope.|
Each HTTP request message sent to the certification service MUST consist of a single serialized JSON representation of a Caliper Envelope. Messages MUST be sent using the POST request method.
The following standard HTTP request headers MUST be set for each message sent to the certification service Endpoint:
|Authorization||Set the string value to the bearer token provided by the certification service and associated with the test endpoint (e.g., Authorization: Bearer <token value>).|
|Content-Type||Set the string value to the IANA media type “application/json”.|
|Host||Set the string value to the test endpoint URL provided by the certification service.|
Following receipt of a Sensor request message the certification service will reply with a response message. The response will include a three-digit status code indicating whether or not the certification service was able to understand and satisfy the request as defined by RFC 7231.
2xxclass status code. The body of a successful response will be empty.
400 Bad Requestresponse.
dataproperties of the required form), the certification service will reply with a
400 Bad Requestresponse.
Authorizationrequest header of the RECOMMENDED form or sends a token credential that the certification service is unable to either validate or determine has sufficient privileges to submit Caliper data, the certification service will reply with a
Content-Typeother than “application/json”, the certification service will reply with a
415 Unsupported Media Typeresponse.
dataVersionvalue that the endpoint cannot support the certification service will reply with a
422 Unprocessable Entityresponse.
The certification service MAY respond to Sensor messages with other standard HTTP status codes to indicate result dispositions that vary from the cases described above. The certification service MAY also communicate more detailed information about problem states, using the standard method for reporting problem details described in RFC 7807.
Caliper Analytics® Specification, version 1.1 defines the use of a single transport protocol (HTTP/HTTPS). However, IMS Global is interested in specifying the use of other transport protocols that can support the exchange of Caliper data. Organizations wishing to work with IMS Global to add other transport protocols to the Caliper specification should contact the Caliper Working Group directly or indicate interest via the public forum.
Visit the Caliper Certification service at https://www.imsglobal.org/sso/launch.php/caliper. You MUST be logged in to the IMS Global website to access the Caliper certification service. If you do not have an account, please register at https://www.imsglobal.org/user/register.
The certification service provides a playground for testing your Caliper messages. Click the “Start Testing” link under Test Your Product to access the playground.
Once you are ready to commence certification testing, click the “Certify Your Product” link under Certify Your Product to commence testing. The following steps will guide you through the process. A screencast of the certification workflow is available for review.
Complete the online form by providing the following information:
Click the green “Start Certification” button. To terminate testing click the white “Cancel” button.
An “Instructions” page provides both a test endpoint URL and a bearer token. Configure your software to send Caliper messages to the test endpoint URL. For each request, set the HTTP
Authorization header field value to the provided bearer token and the HTTP
Host header field value to the provided endpoint URL.
Initiate the product test, sending messages to the certification service endpoint.
When the first message is received by the Certification service the “Instructions” page will be replaced with a view displaying conformance progress. A list of available Caliper metric profiles is displayed on the left side of the page. A log of submitted messages or “items” is displayed on the right side of the page. As messages are processed metric profile certifications that have been attained will be indicated by check marks. Clicking a profile’s plus (+) link will display a list of profile events and actions processed successfully. Clicking the log “items” link provides additional information regarding events processed, errors encountered and individual message JSON-LD documents received.
Once testing is complete click the “Complete Certification” button.
Read the confirmation statement and then click the “Submit Certification Results” button.
Repeat the test as necessary in order to certify against additional metric profiles or address previous failed tests.
After submitting your successful conformance information and receiving confirmation and a registration number from IMS Global you may then apply the appropriate conformance mark. The IMS Global conformance chart will list your conformance details. If you have any questions, please feel free to contact us at any point. Products without an IMS conformance registration number are not considered compliant by IMS Global.
Caliper certification covers individual metric profiles only and is scoped to the specific version of the Caliper specification tested. Major or minor releases of the Caliper specification and/or associated metric profiles will require recertification of your upgraded platform, application or service. All IMS Certifications require that you renew and retest your certification after one year.
The following Caliper Working Group participants contributed to the writing of this guide:
|Anthony Whyte||University of Michigan|
|Markus Gylling||IMS Global|
|Lisa Mattson||IMS Global|
Caliper Spec. IMS Global. Anthony Whyte, Viktor Haag, Linda Feng, Markus Gylling, Matt Ashbourne, Wes LaMarche and Etienne Pelaprat. Caliper Analytics® Specification, version 1.1. 12 January 2018. URL: http://www.imsglobal.org/caliper-spec-v1p1
JSON-LD Syntax. W3C. M. Sporny, D. Longley, G. Kellog, M. Lanthaler and N. Lindström. JSON-LD 1.1. A JSON-based Serialization for Linked Data. 15 February 2017. URL: http://json-ld.org/spec/latest/json-ld/
JSON-LD Processing. D. Longley, G. Kellog, M. Lanthaler, M. Sporny. JSON-LD 1.1 Processing Algorithms and API. 15 February 2017. URL: http://json-ld.org/spec/latest/json-ld-api/
Linked Data. Tim Berners-Lee. “Linked Data.” W3C internal document. July 2006, rev. June 2009. URL: https://www.w3.org/DesignIssues/LinkedData.html
RFC 2119. IETF. S. Bradner. “Key words for use in RFCs to Indicate Requirement Levels.” March 1997. URL: https://tools.ietf.org/html/rfc2119
RFC 2396. IETF. T. Berners-Lee, R. Fielding, L. Masinter. “Uniform Resource Identifiers (URI): Generic Syntax.” August 1998. URL: https://www.ietf.org/rfc/rfc2396.txt
RFC 3987. IETF. M. Duerst and M. Suignard. "Internationalized Resource Identifiers (IRIs). January 2005. URL: https://www.ietf.org/rfc/rfc3987.txt
RFC 4122. IETF. P. Leach, M. Mealling and R. Salz. “A Universally Unique Identifier (UUID) URN Namespace.” July 2005. URL: https://tools.ietf.org/html/rfc4122
RFC 6750. IETF. M. Jones and D. Hardt. “The OAuth 2.0 Authorization Framework: Bearer Token Usage.” October 2012. URL: https://tools.ietf.org/html/rfc6750
RFC 7231. IETF. R. Fielding and J. Reschke, eds. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. June 2014. URL: https://tools.ietf.org/html/rfc7231.
RFC 7807. IETF. M. Nottingham, E. Wilde. “Problem Details for HTTP APIs.” March 2017. URL: https://tools.ietf.org/html/rfc7807
IMS Global Learning Consortium, Inc. (“IMS Global”) is publishing the information contained in this document (“Guide”) for purposes of scientific, experimental, and scholarly collaboration only.
IMS Global makes no warranty or representation regarding the accuracy or completeness of the Guide.
This material is provided on an “As Is” and “As Available” basis.
The Guide is at all times subject to change and revision without notice.
It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Guide as it relates to you.
IMS Global would appreciate receiving your comments and suggestions.
Please contact IMS Global through our website at http://www.imsglobal.org.
Please refer to Document Name: IMS Caliper Analytics® Certification Guide, version 1.1
Date: 12 January 2018
For more information on the IMS trademark usage policy see trademark policy page - https://www.imsglobal.org/trademarks