Learning Impact Blog
FRAMEWORK FOR EXTENDED TRANSCRIPTS: PILOT IMPLEMENTATION & RESULTS (APRIL 2017)
Authors: Joellen Evernham Shendy, Associate Vice Provost & Registrar, University of Maryland University College, and Insiya Bream, Assistant Vice Provost, Registrar Strategic Operations, University of Maryland University College
At University of Maryland University College (UMUC), a New Learning Model project is underway that focuses on building a student’s competencies over time through project-based learning—providing closer connections between the learning and expectations of employers in the field. In the fall of 2016, five of UMUC's Graduate School programs, including the Master of Business Administration and some Cybersecurity programs, implemented the new learning model. As part of innovating and developing a new learning model, UMUC realized that the documentation and evidence given to students regarding their learning would need to change as well. Telling a student story, via a new learning model, requires retooling in order to share the story effectively. For years institutions have heard from employers and others that the universally issued student transcript does a poor job of conveying what a student knows and can do, and is insufficient in providing evidence that helps an employer and others determine skills, competencies, and potential fit for an organization.
UMUC was one institution among 12 that received grant funding provided by the Lumina Foundation through the American Association of Collegiate Registrars and Admissions Officers (AACRAO) and National Association of Student Personnel Administrators (NASPA) Comprehensive Student Records Project. The project focused on piloting new comprehensive record models that display a student's educational and/or experiential background. These records would be in addition to, and not a replacement for, the original official transcript. The goal of the project was to identify models that could then be adopted by more schools—all helping to share the students' story in new and creative ways that speak to a more holistic view of who the student is and what they know. The launch of UMUC’s new learning model and the start of IMS Global’s Extended Transcript (eT) pilot were aligned and provided a natural segment of students that could give feedback on the first iteration of the Extended Transcript (eT) achievements record standard.
How We Did It
UMUC worked closely with Learning Objects through IMS Global’s CBE and Extended Transcript Workgroup, a part of the IMS Global Digital Credentials and Badges portfolio, to establish the initial data model that will allow institutions to manage and exchange competency and achievement data between systems, and ultimately, publish this data in an extended transcript achievements record store—for consumption by an employer, student, or other educational institution or organization. Through multiple iterations, Learning Objects, IMS Global members and staff, and UMUC’s IT team worked together to pull and display student competency data from the D2L learning management system (LMS). Learning Objects was able to digitally represent the competencies in a user interface (UI) that had a more modern look and feel, allowing the student to see, and interact with, evidence of their mastered learning.
Watch Learning Objects lightning talk from the 2016 Learning Impact Leadership Institute.
An overarching goal of the eT pilot was to solicit student feedback on the concept of an eT achievements record and their initial impressions of the utility of an extended transcript. Through the pilot, we were hoping to discover specifics on the following:
- What did students think of the virtual document itself?
- Did students find the content useful?
- How would students use the information?
- Did they understand the purpose?
- How could a digital document like this help carry them towards their next professional goal?
Over the past five months, UMUC’s eT pilot students were able to access an eT achievements record displaying mastered competencies through an icon link in their D2L classroom. Through a "soft rollout," the link was activated and a week later students were notified via email of the pilot. The email included information on the overall purpose and asked them for their feedback via a survey by a link on the eT itself and in an email. One of the survey questions included whether or not the student would be willing to participate in a focus group in early spring 2017 to provide more detailed feedback on the eT. A “help” icon on the eT itself explained what UMUC hoped to achieve and learn through the pilot. The internal marketing team supplemented the email and eT with a web page that included general pilot information and an FAQ for the wider campus community. Out of approximately 2,000 pilot students that had access to the eT, over 800 students accessed the document.
What do Students Think of the eT Achievements Record?!
UMUC students responded throughout the pilot period to the survey, sharing their thoughts and recommendations. The majority of survey respondents made it clear that they liked the graphical look and feel of the eT. Working with a third-party vendor, Learning Objects, allowed UMUC to help create the design and feel of the eT by utilizing a variety of input from various University stakeholders and by modifying the document until we were satisfied with the appearance and ease of use for students. Leveraging Learning Objects' technology brought a 21st Century user experience for students to the eT achievements record.
Survey results showed that over 70% of respondents found the eT content useful. Students noted that they could use the information on the eT for resume writing to explain or communicate their competencies and achievements in the academic program. Students felt they would share this mostly with future employers, but also with other admissions offices, their current employers, family, and friends. We asked survey respondents if they recommend we open this up to the larger UMUC community and the answer was an overwhelming YES.
UMUC also conducted a focus group to drill down into some of the survey questions to get additional information to help inform the next iteration of the eT. In general, the focus group students were positive about the eT and 84% recommended UMUC implement an eT-type achievement record for all students. They shared additional details about the appearance and content of the eT. Students felt that the eT would give them more confidence when pursuing job opportunities—helping them to articulate with clarity and confidence their competencies. Several students remarked that their own understanding of their achievements was influenced by the content they viewed in the eT record.
The team of academic and technology colleagues working through IMS Global have been integral to the success of this effort.
"We extend hearty congratulations to UMUC for this critical first step connecting their students to employers through digital credentials. The Extended Transcript Achievements Record forms the foundation of an institution’s digital credentials strategy. UMUC and their peer leaders Capella University and the University of Wisconsin-Extension are paving the way to tomorrow’s marketplace of verified credentials."
—Mark Leuba, vice president,product management, IMS Global Learning Consortium
What areas of improvement did the students identify for the next version of the eT? Although most students did not feel the digital document was too lengthy, some felt the length would cause employers to gloss over the information. One idea shared was to allow students to curate and create their own “view” of the achievements record that was specific to a particular employer, organization, or industry. This would allow them to share a portion of, but not the entire, eT and highlight accomplishments most relevant to the students’ goal(s). Focus group participants noted that it would be interesting to see the competencies organized from an industry or sector perspective and to break down the “level” of competence based on position requirements such as entry level, middle, or C-Suite management. There was also a student desire for the opportunity to contribute to the document by way of adding their own supporting content, providing them an option to make adjustments for their unique situation.
Overall, the feedback collected through the survey showed that students understood the eT achievements record and would use it in multiple ways to reach their goals. While the respondents felt understanding their own accomplishments and articulating them were a huge benefit, they did feel most strongly that the connection with employers was critical. One particular comment from the survey highlights why we need to continue to build these bridges, via new ways to evidence learning, between institutions and the workplace:
“The document is a great reminder of what a graduate is capable of doing. Instead of saying I walked away from college with an x.xx GPA, I can say explicitly, this is what I am capable of doing; these efforts gained me these competencies.”
Future State of the Extended Transcript
UMUC successfully closed the pilot at the conclusion of the winter 2017 term. Our next steps are to leverage the work we are doing as an initial contributor to Credential Engine with adding our program competencies to the nationwide registry. This will provide additional linking and connection opportunities. UMUC is also working towards finding better ways to “match” a student's competencies with employer needs by continuing to work with innovative and dedicated colleagues through IMS Global in their digital credentials and badges program. As far as a next version of the extended transcript achievements record, we look forward to incorporating our students feedback and reaching out to the employer and recruiting community to build closer ties and feedback loops, ensuring our students will be armed with 21st Century digital evidence of their learning which presents a more holistic picture of their fit to organizational needs.
For more details on the next phase of the Extended Transcript, see Achievements Records Standard and SkillsCenter Search™
“I can see for miles and miles and miles.” – The Who
We’re all impatient and getting more impatient all the time. We’re all looking for time well spent. But, we also want to be prepared for the future. How do you achieve both?
I love the work of the IMS Global Learning Consortium because our members are on exactly that quest—saving time and money now while laying the foundation for the future. If that is what you are looking to do as an institutional leader or as a product leader, you will not be disappointed in the upcoming Learning Impact Leadership Institute (#LILI17).
Every year several executive level or aspiring executive level leaders tell us that the Learning Impact Leadership Institute is the best event they’ve attended, which is a strong statement because there are a lot of great events in the educational technology space.
What’s different about the Learning Impact Leadership Institute is that at the heart is the IMS collaboration, a set of education industry leaders that are inspired and motivated to move the industry forward in a set of very important core areas: digital content, learning apps/tools, e-assessment, student data, academic data, digital micro-credentials, competency-based education, educational badges, learning analytics and digital accessibility.
By far the best thing about IMS is the leadership community that gathers around and collaborates on the major barriers to implementing these key technologies at scale. At IMS we recognized early on that the “standards” needed to remove the barriers cannot be addressed independently from being a catalyst for innovation. We needed to do both and are doing both. Every day.
You are really missing out on something special if you don’t attend #LILI17. IMS holds quite a few meetings around the world and presents at nearly 60 conferences each year but LILI is the only time and place where you can within a pretty short timeframe—a few days—see for miles and miles and miles ahead—all while you are most likely walking away with something that will save you time and money in the near-term, too.
To see what happens at the Learning Impact Leadership Institute watch this 3-minute video of the 2016 NGDLE plenary talks.
discuss the future of Open Badges during the 2017 Summit on Digital Credentials & Badges.
“Cause I’m not like everybody else.” —The Kinks
Standards are the foundation for diversity at scale. Just look around you . . . and then realize that all this diversity came from a common standard that we call DNA. Lack of a foundation in replicable standards means lack of scalability. Think the plethora of grant projects ending up in one-off successes.
Every teacher and every student are not like everybody else. The future of education isn’t massification; it’s personalization. Personalization at scale.
During IMS Global’s recent quarterly meeting in Orlando, we had the pleasure of hosting a Summit on digital micro-credentials, including Open Badges (a standard and community that IMS Global is now facilitating after a recent handoff from the Mozilla Foundation).
Micro-credentials and Open Badges are key foundations for scaling personalized education. They are a way to enable and recognize personalized learning and life pathways. In essence, they are a way to create and recognize each person’s unique story.
Educational systems and educators that embrace and perfect micro-credentials will be the ones leading us to the personalized and scalable future.
The Summit by itself would have made anyone’s week. But there was much more at the quarterly meeting.
Agreement was reached on a breakthrough competency and learning standards framework (Competencies and Academic Standards Exchange) that enables digital resources to be mapped to achievement frameworks and thus enables finding and selecting resources based on those frameworks.
A set of leading educational content providers established a breakthrough open source collaboration to enable ubiquitous availability of OneRoster®.
The emerging field of learning analytics continued its advance thanks to an incredible cadre of leaders in IMS Global who are making learning analytics practical via Caliper Analytics® and IMS Global’s vibrant partnership with ADL/xAPI.
Among the most exciting sessions of the week were the Learning Tools Interoperability® (LTI®) Showcase and Hackathon. Suppliers and institutions are doing amazing things with LTI apps that can work across platforms. And, yes, there were some very tangible signs that the market is beginning to catch up with the more advanced features of LTI and LTI v2.
If you weren’t there and you’re not like everybody else come join us next time! You’ll see and lead the personalized scalable future of education and learning.
On Monday, August 15, at the IMS Global quarterly meeting at Utah Valley University, the leaders of the Advanced Distributed Learning (ADL) initiative, responsible for the Experience API (xAPI), came together with the leaders of the Caliper Learning Analytics Framework (Caliper) from IMS Global. ADL has joined as a Contributing Member of IMS Global and the parties are committed to exploring a unified path for xAPI and Caliper. This may mean alignment at some level or potentially even convergence.
There were roughly 50 experts in the room for an entire day on August 15, and I believe it is fair to say that both ADL and IMS were very pleased with the eagerness of participants to cooperate and depth of discussion. Presentations from the meeting are posted here and there should be a synopsis of the comparison posted soon.
At the next IMS quarterly meeting November 7-10, 2016 at Arizona State University, there will be a follow-up session on Wednesday November 9 open to the public (registration required). I expect it to be another great session that should lead to some tangible prioritization and next steps. On November 8 there will also be a day-long Summit on Creating an Educational Analytics Ecosystem.
IMS and ADL are committed to collecting and achieving public input as the process moves forward. We are likely to create a public forum to encourage the public community to comment and also help crosswalk the information models. Stay tuned to ADL and IMS announcements and twitter feeds (IMS twitter feed is @LearningImpact). Currently there is a survey for those that have views on the potential convergence of the specifications here if you would like to participate.
I attended the entire August 15th session and would like to provide my sense of the big picture. I’m not the technical expert, but I have a long history in the applications of standards, including those that evolved into SCORM, and of course the dramatic growth of IMS standards adoption the last 10 years in the education sector.
I am extremely impressed with both the xAPI and Caliper work. While they are perceived by many as “solving the same problem” my own take is that this is largely because there is still quite a lot of variability in the Learning Analytics field in general. Thus, almost any set of technical work that deals with transcribing, sending, logging and analyzing data during any form of e-learning is considered “learning analytics.” From my perspective, while xAPI and Caliper certainly have some overlap, they are largely complementary at this phase in their respective evolution.
At the risk of over-simplifying things, my impressions of the strengths of each specification are captured in the table above (while there are four “strengths” shown for each specification they are not meant to be compared in four dimensions - the above are just lists of the primary strengths). The strength of xAPI is as a way for a single application to log freeform natural language statements and for that data store to be retrievable by the logging entity. The strength of Caliper is as a way to aggregate agreed upon events across a set of applications to enable processing across the aggregation of data.
It is also very clear that while neither specification “requires” the body of work from which they came (SCORM for xAPI and IMS standards, like LTI and OneRoster for Caliper) that these foundations greatly influence each. For instance, in the case of Caliper the co-existence of the other standards in an implementation means that Caliper can simply complement a slew of outcomes and context data that are already flowing via LTI. Again, because the Learning Analytics field is rather loosely defined at this point with widely varying “use cases” one can easily get confused about whether they even need Caliper or should just be using LTI to it’s fullest potential (for instance, LTI can already send data to an endpoint that is not the LMS).
I will also note that in the IMS Communities of Practice relating to data and analytics in IMS, we are seeing a very high priority need to simply be able to see all the data visualizations in one place. The level of data processing may be fairly minimal – but the humans in the equation need a better way to integrate visualizations that may already be available from various products they are using.
Bottomline, in my humble opinion, is that as we move forward on alignment and/or convergence of xAPI and Caliper I think we need to consider the use cases more carefully and, in essence, better define the “categories” or “scenarios” of Learning Analytics. While that may seem like “a step backwards” well, I think we need more clarity. The good news is that I don’t believe we will have any problem given the wealth of actual implementations we are working with in IMS and ADL.
Our goal in IMS is to enable serious school districts, higher ed institutions, states and nations to implement high impact learning analytics use cases and to enable a plug and play ecosystem of products that provide the data needed to solve those use cases. Either specification is good enough to begin doing some work. But, in order to do what we need to do we need a very large interoperable ecosystem of products, users, and researchers that can all invest together to get to the understanding we wish to collectively achieve. I would see elements of xAPI and elements of Caliper as both being essential in that goal.
In the previous posts of this series I have reviewed the requirements driving next generation digital learning environments, looked at the anatomy of interoperable educational apps and considered the realities of sector trends in learning platforms and products. Episode #6 concluded that the evolution to NGDLE will be driven by a combination of high value use cases enabled by an interoperable product ecosystem.
The Critical Importance of Sector Leadership
First a bit about the sector leadership required in getting to the NGDLE. From where I sit, education sector leaders need to decide if NGDLE (or whatever comes next) is something that is happening to us or something that we are going to make happen?
To explain further, tech innovation in the consumer world tends to “happen to us.” Apple iTunes is not something that consumers invented. If consumers had invented Apple iTunes I’m sure we would have chosen to make sure all those tunes and apps would work on any mobile device we chose, not just Apple devices. Or, why is the Apple library limited to use on 5 devices? That is not very consumer friendly. But, individual consumers don’t have much influence in terms of the platform strategies of high tech companies.
But, the education sector is different. While historically there has been a steady pattern of what I call the "alien invasion" theory of edtech adoption, there is huge potential for institutions that can work together to shape the ecosystem, and therefore the platform strategies of suppliers, going forward. Indeed, we are at a period in time right now where not only institutional leaders, but leading suppliers are largely agreeing that vendor-specific ecosystems are NOT the way forward in the education sector. No single vendor can provide the diversity of digital resources, tools, apps and platforms. And no single vendor can provide the sales channel to dominate education revenue generation.
But, strong collaborative leadership is required to establish an open edtech ecosystem. The “system” of leadership that seems to be working are suppliers that can bring the product ideas and technical expertise while institutions bring the clarity of the high value use cases and the firm requirement for plug and play integration. In IMS we have seen the combination of these two forces achieve some amazing things. Leadership from both institutions and edtech suppliers in creating and evolving the ecosystem the NGDLE (as envisioned in the EDUCAUSE research and this blog series) is essential.
Use Cases and Emerging Product Categories that Will Shape NGDLEs
In IMS Global we definitely believe that necessity (aka compelling need and value proposition) is the mother of invention. Exact timing of the cycle of adoption can highly variable based on a number of factors (see the discussion on filtering ideas based on the four major VC risk areas in the previous post of this series). In IMS we like to determine if the value proposition is strong, then establish a track of work and then let the market feedback help us sort out the risk factors and timing.
High value use cases for the NGDLE are those that enable flexibility, choice and personalization that lead to improved educational outcomes. Here are some obvious winners:
- Seamless user experience for teachers and students: Sorry if this one seems rather uninspiring but the reality today is that the user experience in navigating among apps and platforms is still not very good. Making the user experience seamless is key to enabling the development of NGDLEs that are seen as a step forward, rather than backwards. Indeed, an obvious first and essential step in building further confidence toward the NGDLE concept is to achieve seamless integration of some of the core instructional applications already in use at most institutions (see a list in the previous post).
- Curated digital resource choices with associated assessments: Student outcomes improvement ala Benjamin Bloom’s famous “2 Sigma Problem” is dependent on delivering effective differentiated instruction via a classroom cohort. This infers well thought out instructional strategies and resources, especially assessment resources, that can support and inform those strategies. Also, as the number of educational “apps” and “digital resources” scales, the need for effective curating that combines easy access/integration with effectiveness becomes paramount.
- Digital resource recommendations: Ability to help automate the recommendation of educational resources from among the curated choices will provide high value to instructors, students and parents. This is a very advanced topic that will take many years to develop. However, it is completely feasible that machine learning techniques could be applied to monitoring and capturing the recommendations of instructional experts.
- Understanding the usage of digital resources: As the use of digital materials scales there are obvious high value questions that need to be answered concerning the usage of digital resources. Which ones are being used, when and why? Which ones are not being used? Why not? This information is of great value to suppliers and institutions alike. However, if it is not “easy” to get the data in one place and understand it, the adoption of digital resources will be hampered.
- Optimizing and/or personalizing student pathways to a credential or degree: The ability to understand student pathway options and choices, improve them and recommend them has obvious value. True personalized learning, an umbrella term that can encompass many things, is achieved when the outcomes are defined for each student. The true personalization occurs then across a set of courses or competencies. Educational success and fulfillment are achieved through a balance of personalization and optimization of the path through courses and gaining of competencies.
- Better understanding of student progress and risks: For an NGDLE to deliver on student success it will need to do a better job of helping students understand their level of success and risks of failure. The NGDLE must support real-time communication and dashboards across a variety of activities to help students measure themselves. An NGDLE will help students, advisors and faculty understand when self-service needs to be supplemented with high touch intervention.
We are observing emerging product categories and how they are fitting into the edtech ecosystem all the time in IMS Global. I am going to refrain from mentioning specific product names here. Rather, I will use abstracted definitions of the categories we see growing and emerging. A very important note, that while the learning management system category is not listed, the LMS has the potential to enable or implement many of the categories below. Whether or not it is a good business strategy to do so is another question.
- Modular digital curriculum and assessment: Instructional material or curriculum as a “black box” in which the modular construction and progression are difficult to access and understand are becoming less attractive in the digital age. There is both huge opportunity and risk here for organizations that can structure their curriculum products to help solve the high value use cases above.
- Integrated adaptive assessment: While related to the ability to modularize assessment (previous product category) the category of assessment tools that can be readily integrated with a variety of instructional resources and approaches will be fundamental to improving digital learning outcomes. Assessment is a specialized science and quality assessment is not simply providing some test items. This category will combine ease of use/integration with high quality assessment with real-time feedback.
- Digital curriculum mapping and management: Differentiated instruction, individualized instruction and personalized learning all require management of the pathways and options available to teachers and students. The management of the curriculum needs to take a step up from where it is today. This product category will enable that.
- Learning resource catalogs, app stores, repositories: Perhaps Google or Amazon will be the answer to searching for educational resources. However, the search and integration of resources required to enable the high value use cases above is a lot more complicated than the sort of searching and recommendations that suffice for enabling Internet commerce. This product category addresses the organizational and ecosystem integration features needed to enable better learning outcomes and ease of use.
- Learning analytics processing and messaging: Learning analytics is a hot new product category. It is also something that many LMS platforms and tools offer in some shape or form. As with assessment, learning analytics is a highly specialized field. And, analytics needs to be performed across many apps. Therefore, I think it is likely that new products will emerge that are separate from existing products.
- Outcomes and achievements management: Several new product types are emerging that are helping institutions manage how they structure and assess student progress in terms of competencies and micro-credentials. Some of these products are incorporating the “integrated” assessment category. This category will help institutions structure the relationships among the outcomes they hope to achieve with students. It is different but potentially related to curriculum management. It is a layer above.
- Student pathway management: Products that can help advisors and students understand the path. These products will help students plan and optimize their path to their learning and degree objectives.
- Edtech research data collection and processing: An NGDLE will enable better research of the effect of technology and instructional approaches on student achievement and outcomes. Therefore, I anticipate that products that enable learning analytics correlation of edtech usage to student success will emerge. This could be considered a subcategory of the “learning analytics” category above, but am calling it out separately as a vey important and necessary contribution that may not be covered by other analytics applications.
Want to read about specific products along these lines? The best place is the IMS Learning Impact Report, which we publish annually, and which we consider to be ongoing research into the product categories that are leading the way to next generation digital learning.
Finally I will leave you with a simple idea and figure (shown here) to help illustrate the road we are on in terms of a maturity model of edtech products and the evolution toward next generation digital learning environments. At the base level, an educational institution should be able to support the use of technology to support teaching and learning. This is stuff like BYOD and Google Apps for Education or Microsoft Office 365. The next level up is the enhancing of productivity using technology. Herein lies the success of LMSs and many other tools and technologies. However, one should not think it trivial to get to this 2nd/middle level because there are loads of examples of digital tools, and lack of integration of those tools, that have made life less productive for teachers and students. The “top” tier are emerging applications that have the power to improve student achievement and learning outcomes. Some of these are new product categories and others are existing products that are used in ways that clearly support better outcomes. But the key point is that for an NGDLE, the expectations are evolving from the perspective of the customers – the students, the faculty, the institutions and society in general.
The motivation for collaborating on NGDLEs is to enable a more effective ecosystem of educational platforms, apps, tools and resources that have measureable impact. I hope to see you at the Learning Impact Leadership Institute where a unique collection of sector leaders gathers to further this important collaboration.
In the previous posts of this series I have provided a high level overview of the potential goals of next generation learning environments. In the last several posts I have taken the perspective of what it might mean to be an “interoperable app” in the context of next generation digital learning environments.
Now it’s time to come back to reality and look for clues from the market in terms of how we are likely to evolve to next generation digital learning environments.
Today is the First Day of Your Journey to NGDLE
One of our conundrums in getting to the next generation is dealing with the current generation. Educational institutions are already dealing with “sub-optimal” integrations (I’m being kind here) of systems, platforms and applications that teachers and students touch every day.
Indeed, I would be a much less grumpy person, and more importantly, faculty and students would be much more satisfied with technology, if the following products, typically found in a majority of classrooms today, were better integrated:
- Digital textbook (interactive)
- Library digital resources
- Summative testing
- Formative testing
- LMS with Gradebook (or separate Gradebook/outcomes assessment & tracking)
- Classroom capture
- Classroom response
- Classroom management
- Interactive whiteboard
- Other digital resources and applications
This is a pretty limited set. But if just a subset of these applications were really nicely integrated into a seamless and easy experience for students and faculty, technology would be a lot more appreciated in the education sector. Although in K-12, the “other digital resources and apps” are typically in the hundreds – even today.
The following types of products are also in the mix already.
- Online class conferencing
- Assignment grading system
- Adaptive learning tool(s)
- Course evaluation
And don’t forget basic shared IT infrastructure:
- Student information systems
- File storage and sharing
- Video management
- Content authoring
And then there is the potential for integrating those mobile apps that come to educational institutions courtesy of the Apple, Google, Amazon or Microsoft consumer Internet app platforms.
Learning Platforms Are Evolving
Now let’s consider the evolution of the “learning platforms” themselves. In IMS we use this term rather broadly. In the future I expect we will be characterizing the subcategories of learning platforms better. But for now a learning platform is essentially an integration point for learning resources that may provide as little functionality as single sign-on and 1-click launch.
News alert! If you’ve been following IMS recently you will know that there are more than 400 products that have passed conformance certification. There are over 70 LTI learning platforms and a couple hundred LTI tools in that mix – which probably represent about 1/3 of the actual adoption of the LTI standard in the market.
Indeed in K-12 school districts we are seeing easily 20-50 LTI tools or content series being integrated in districts (1+ million learning objects via thin common cartridge) that have gotten their LTI act together.
One question to ask is, “Are the more sophisticated learning platforms in tune with the stated requirements of the NGDLE?” Back in October of 2015 I did a featured talk at OLC (Online Learning Consortium) in which I highlighted some of the directions toward NGDLE. In preparation for that talk I reviewed the web sites of several of the leading learning management systems looking for messaging that might be consistent with the NGDLE requirements.
The figure here shows what I was able to ascertain. Apologies to all concerned that this is dated now – and therefore may be different today.
While this comparison is a very crude analysis, I do think it reflects several market realities in getting to the NGDLE:
- Analytics (however it might be defined) is clearly the area of the NGDLE that is getting the most attention, interpreted as, “there is strong customer interest and money to be made there.”
- Accessibility as a standalone category is not getting the attention that all of us would hope it would be getting, although personalization (which I have connected to accessibility in this series – see discussion on user preferences here) is getting some play (but I expected more).
- Collaboration gets a fair amount of coverage, but it is not the hot item that the NGDLE research seemed to indicate it should be – perhaps because, as in the world of IMS, collaboration of various types can be supported via a wide range of apps.
- There were a range of other key marketing themes that did not get too much attention in the NGDLE research, including usability, reliability, mobile, content management, course building, grading and competency-based education (CBE).
- Perhaps most interestingly, is that the marketing of the so-called “lynchpin” of the NGDLE, interoperability, was very uneven and, form the perspective of IMS, pretty weak.
While again, this is a very crude analysis, I think the market is giving us some important messages about NGDLE. One could take a “glass half empty” of “glass half full” interpretation. The glass half-empty view is that the product providers are not in tune with those institutional leaders participating in the NGDLE research. The glass half-full view is that the product companies are focused more on marketing to higher value themes – like product usability, student success, competency-based education, etc.
My own view is that both interpretations are correct. In my thirty-five years of product development experience across a variety of sectors, I feel that there is no sector in which the product providers are more out of touch with their customers than the education sector. The senior executives and marketing folks in a large number of product companies are not understanding as well as they could the desires of the end users. On the other hand, I do believe that it will be very specific areas of improvement that will drive progress toward the NGDLE. General ideas, like personalization, are not going to drive the suppliers and customers to work together to get to the NGDLE. More specific ideas, like clearly better usability of technology to support teachers and students needs, will.
What Will Drive the Progression Toward NGDLEs?
Getting to next generation digital learning environments will happen because the high-value use cases require cooperation among a diverse set of digital resources, apps, tools and platforms. Enabling specific high-value use cases is key. Where the “architecture” comes in is to be able to evolve the solutions to those use cases collaboratively across an ecosystem of suppliers and institutions – versus a much more slow and ultimately unreliable approach of big bets on one-vendor solutions.
To put it another way, the evolution to NGDLE will be driven by a combination of high value use cases enabled by an interoperable product ecosystem.
Up next in the series: In the next and final post of this series on next generation digital learning environments I will take my guess at specific use-cases and new product categories that are driving us toward NGDLEs.
In the previous post of the series I provided a high level overview of the interoperable inputs that an educational app might utilize. Now it’s time to look at the outputs.
It is interesting to note that last week I was on a panel at the well-known ASU GSV Summit entitled, “Why Can’t We Be Friends: Vendor Interoperability in Districts.” The salient takeaway for this post is that there was great agreement that while the sector is making some pretty clear progress on the input side of interoperability, the output side is pretty early in its development. So, get ready for the harder part!
As with my discussion on the inputs I am going to refer to the Anatomy of an Interoperable Learning App figure shown here. And, I am going to focus on what makes these outputs “next generation” in comparison to today.
For next gen learning apps the objective is to enable better information outputs to understand progress, further personalize the learning experience and just in general understand the usage of various digital resources. Shown in the figure are some categories of potential outputs (on the right hand side) that are discussed further here:
Activity: Some IMS members call this aspect of digital resources “adoption.” As we evolve into the next generation of scaling the number and types of digital educational resources and applications one of the first questions that needs to be answered is, “Are teachers and students using these resources?” One function of this is for administrators, teachers and suppliers to get an understanding of what is being used, and perhaps be able to infer why or why not? One of the major deficits of research into use of digital resources is the dependence on user surveys. It is much better to have ways to directly measure usage. And, once usage can be measured accurately it enables the potential that use of specific resources or combinations of resources may be statistically correlated with outcomes via various assessment mechanisms. Activity information is also very important to the content creators/publishers because it enables a better understanding of how digital resources are being used and thus provide clues for improvement.
Outcomes: The most effective educational platforms and apps will be those that can indicate to the student, parent and teacher how well the learning is progressing. Of course, the term “outcomes” can have a very broad definition in education. However, the thought here is that the educational application will have some way to report progress, including scores to measure success on a specific activity, “gradebooks” that are compilations of scores, and even comparisons to the progress of other users. The challenge is that different applications will measure progress differently, and thus be difficult to compare and analyze. From the perspective of interoperability, one potential big win is a usable way to collect all the progress information on a single student on a single screen or other usable interface. Today, many tools may have various progress reports or dashboards, but they are resident “inside the application” and thus not readily looked at in conjunction with progress information from other apps. Another big win is agreement on some common summary information across a wide variety of learning applications. The holy grail perhaps, depending on your view of the viability of common learning standards, is an agreed upon approach for all apps and platforms to report on progress by specific learning standards. Finally, another very important type of outcome data in education is assessment “item data,” which is the collected responses to assessment items.
Credentials: There’s a lot of talk these days about the potential advantages of micro-credentials in education. Micro-credentials are ways to recognize granular achievements or cross-curriculum competencies that are highly valuable and typically not inferred by a higher-level construct such as a course grade or GPA. Such achievements or competencies may be extremely valuable in matching a person to a job or career. In the educational context, micro-credentials might do a much better job than grades in determining if a student is “college ready” in terms of meeting specific prerequisites. In the context of next generation digital learning there is the distinct possibility that a learning application may provide evidence toward a defined micro-credential or even a validation of such a credential. The interoperability that will be required will be a way to combine or mash-up such credentials. IMS is working to apply open badge extensions and modular electronic competency and transcripts to enable credential validation and mash-up. The current state of the market is quite early and exploratory. So, we have a long way to go – but this is very important work for the future that may find useful application in those institutions that are doing a better job at micro-credentialing and competencies.
Artifacts: This very important area of next generation learning is potentially related to credentials. It is about how to capture, in an effective manner, what the learner may have created during their interactions with the application. While an artifact is not a credential, it can provide evidence of competency, creativity, agency, etc. The ability to capture meaningful artifacts can help a learner reflect on and assess their own learning, reinforce past learning experiences and display their work to others. An e-portfolio is a type of learning app that is designed for this purpose. But, artifacts can be created in many other types of learning applications and platforms. Next generation learning requires not only interoperable publishing mechanisms, but also interoperable metadata to enable classifying, sorting and searching artifacts.
Ramifications on Next Gen Architectures
From the above, one can see that there are a broad range of possible outputs from learning applications and platforms. As previously noted, interoperability of the outputs of learning applications is a complex topic that needs a lot or work. It is also much more variable across different educational levels and types than the inputs. Highly variable areas that are early in their maturity are usually not good topics to attempt to get industry agreement on. Therefore, we can reasonably make some conclusion regarding interoperability of outputs for next generation learning
- Interoperability of activity, outcomes and credentials will need to be structured to enable affordable collection of data that supports research on potential correlations.
- A very high-level of cooperation will be required among suppliers of a wide range of products to establish the interoperability standards and enable the data exchange to achieve #1.
- For #1 and #2 to accelerate understanding of the impact of learning apps and platforms, industry must set up and sustain funding models for cross-industry research.
- There will also need to be some “quick win” areas for interoperability of outputs. There are some ideas for such quick wins in the descriptions of the categories of outputs above. By “wins” I mean providing value to institutions, users and suppliers. Another way to state this is that “the road of establishing the interoperability for next generation learning must provide current generation benefits to all market participants.”
With the release this week of the IMS Global Annual Report 2015, I wanted to attempt to summarize some ways in which “our little engine of change” is helping to enable some potentially pretty profound stuff.
A little more than two years ago (December 2013) I was asked by Diana Oblinger (CEO of EDUCAUSE), to present, along with Jack Suess (CIO at University of Maryland Baltimore County), a “game changer” webinar discussing the recently published EDUCAUSE Review article, “A New Architecture for Learning.” We decided to entitle the webinar: “Building the Connected Learning Platform – One Brick at a Time.”
Of course, it was flattering to be considered a “game changer” – whatever that term really means. Most people, including technology veterans, would probably think of interoperability standards (the work of IMS Global and important basis for above paper and webinar) as being “part of the game” rather than necessarily “changing the game.” I would include myself in that group when thinking about the large majority of standards-setting activities.
However, I did think it possible that the work of IMS Global could be a “game changer” when I became the CEO a little over 10 years ago. Now I believe in the “game-changing” nature of IMS even more. I’d like to enumerate here some of the specific ways that IMS has been and is changing the “edtech game” (albeit never fast enough for me!).
#1 Providing a foundation for collaboration that fits where the edtech sector wants to go in terms of diversity of platforms, resources, apps and tools. The rules of the game are different when an industry is dominated by proprietary walled-gardens then if there is plug and play interoperability of tools among a wide variety of platforms. As consumers, we live all day long in proprietary walled gardens created by a few dominant platform suppliers with app stores and mobile devices. However, the education sector has figured out that the idea of just a few dominant platforms just does not make sense for the sector. For instance, most K-12 districts are compelled to support both Microsoft and Google. IMS standards, and more importantly the critical mass of IMS members leading the charge, don’t see a few dominate platforms as the future of edtech (to the MBA people out there that will say that every market moves in the direction of a handful of dominate providers, I just want to say that I know those arguments well, but the education sector is very, very different). I’m very pleased to report that the IMS community is leading a better way to build an ecosystem based on what the education sector requires, namely products that can actually work together to improve teaching and learning while keeping lock-in and barriers to entry low to spur innovation.
#2 Providing an interoperability development process that is based on engaging directly with suppliers and institutions. Much of the work on standards in EdTech has been very visionary and developed by some really brilliant people. And, my hats off to the great work. However, the missing link to make this work payoff for all parties concerned is a necessary reality check of compelling sector needs and market adoption. IMS has added this extremely important element in the last 10 years. For those familiar with venture capital or market development approaches, the IMS process is very much a funnel where lots of good ideas go in that are then processed through market validation. So, to get out (become a published IMS standards) there has to be an acceptable combination of market, financial, people and technical risk (which are typically the four dimensions of risk that VC firms look at).
#3 Focusing on enabling the plug and play ecosystem that results in more investment on innovation. IMS is creating a major culture shift in EdTech standards, namely that the measure of a successful standard is that it helps to lift up the entire sector. IMS sees our job as enabling an interoperable ecosystem – what has already become hundreds and in the future thousands of products that work with each other “out of the box” (or as close as possible). Time and cost of integration goes to zero. When that ecosystem happens it focuses investment on the innovation of the individual products themselves and also of what can occur when products work together. To anyone familiar with some of the more mainstream standards in the tech world that were huge successes, like Ethernet, the Worldwide Web, or wireless standards, you would most likely say, “isn’t that the point of most standards?” A good standard reduces cost while increasing connection into a large ecosystem, which means that suppliers are naturally motivated to adopt.
#4 Accelerating interoperability progress through synergies across K-12 and higher education. I’ve gone on record many times that I believe there are many differences between the needs of K-12, higher education and corporate training. I know from actual experience that if you try to build a product for all of them you are unlikely to succeed with any of them. Thus, when the U.S. K-12 sector came knocking on the door of IMS about 5 years back I had an open mind, but was cautious about how it would work out. Well, it has worked out great! What we have found is that one sector can take work coming from the other sector, get use out of it, and then, amazingly, take it further in ways that have then gone back the other way to help the originating sector. It has taken some very good market development and project filtering skills (see #2 above) to encourage the right focus at the right times. But, it is very clear that IMS is now a 50/50 partnership between higher education and K-12 – which of course means that each sector is leveraging the investment and ecosystem of the other. The bigger the ecosystem the more valuable the standards.
#5 Creating an EdTech ecosystem leadership community that will lift the sector in the sort term and is likely to endure in the long term. This is really the most important impact that IMS is having. As I mentioned in a recent EDUCAUSE Review article, “Foundations Past and Future,” IMS is really all about education sector participants willing to lead a critical aspect of cooperation that will accelerate the innovation we need to improve education. There are, of course, many other critically important areas where cooperation and communities are developing with complementary foci. But, I do believe that interoperability is an underlying foundation for much of the rest that requires strong leadership collaboration. The beautiful thing is that we are finding that those organizations, institutions or suppliers, that are investing time in leadership in IMS, are doing very well. It turns out that to be technically interoperable also requires organizational leadership that is interoperable. It’s a big claim to make, but I will make it. Namely that we are seeing IT, curriculum and instructional leadership and strategies getting to the next level when an organization works on and “gets” interoperability. We are seeing a focus that goes way beyond the historical desire to just have the latest technology to being able to leverage technology toward personalized learning and other strategic goals. As indicated in the figure, the net result is a financially strong (while still small compared to many education associations such as EDUCAUSE or ISTE) IMS organization that can play a substantial ongoing role in developing the EdTech ecosystem.
As a result of the above innovations, led by the IMS members, I do think the trajectory of EdTech is changing in a very positive way. I’m very proud of the IMS members stepping up to take on the challenge of working together on building an open and innovative EdTech ecosystem.
In the previous post of the series I laid out a perspective on interoperability for next generation digital learning environments from the perspective of an application being able to fit into a configurable constellation of educational apps. I would now like to elucidate the key features that make said app and environment “next generation.” I will once again refer to the “Anatomy of an Interoperable Learning App” figure introduced in the previous post.
What might next gen learning apps know about the learner? What should they be allowed to know? What can be personalized in the user experience from that knowledge?
For next gen learning apps the objective is to enable better information inputs to allow the app to personalize the learning experience. What is the information and what does the app do with it? Shown in the figure are some categories of potential inputs (on the left hand side) that are discussed further here:
Institutional context: This is the type of information that is typically found in a student information system (students, student grade level, courses, sections or other groupings). There are potential common languages for this from interoperability standards like OneRoster and Learning Information Services, as well as the Rostering (aka Membership) Services of Learning Tools Interoperability. While exchange of this sort of information is not especially “next gen,” enabling this exchange without the need for custom integrations and code writing is. The information is typically used to authorize use of the app as well as group users together (such as in a collaborative app). Information about who the user is can also be used to bring the student back to the place where they left off, store results in progress specific to the user, etc.
User context: This is information about the learning activities that the student is currently engaged in (reading, assessment, discussion, assignment, media, etc) when the app is launched. User context is also information that the app can generate and output in the form of an activity description. User context is an area that requires more work to develop a common language. Standards work applicable to this is Caliper Analytics and Learning Design. User context information could be used by the app to personalize how the app behaves. For instance, an assessment app might act differently if it is launched in conjunction with a reading activity, versus if it is launched as a separate assessment. Again, this is a more advanced topic. It is very “next gen” in that it enables an app to respond to the circumstances (activity sequence) in which it is launched.
User Preferences: We all know that apps typically have some ability to allow the user to specify preferences. For “next gen” digital learning environments to be personalized in a scalable way, there needs to be a way for the user to specify preferences one time and have that information propagated to all relevant apps. These are not “app specific” preferences, but rather preferences that could be pertinent to most apps. For instance, when a user interacts with apps on their smart phone they may have certain preferences for size of fonts, use of media, etc. There is a rich body of interoperability work called Access for All that has developed a rich framework for describing personal needs and preferences (PNP). Access for All is applicable to all users, but also has a rich foundation in accessibility (and has even been published as an ISO/IEC standard). PNP has been applied to high stakes assessment accessibility via the Accessible Portable Item Protocol (APIP) standard. One of the most important preference areas in education is privacy. While the “setting privacy preferences in every app” model of consumer app stores may make sense for that world, it does not make sense for education. Students or parents need to be able to set privacy preferences once in the context of their institutional experience and then have those privacy options set as defaults for every app. Therefore, for “next gen” learning environments, our expectation is that the privacy preferences will be selected from an interoperable privacy framework that is provided as an input to each app for each user.
Learner Profiles: One of the potentially most valuable interoperable inputs for next gen learning apps is a learner profile. This is information that lets the app know where the student is in their learning experience and progression. It can be used to provide a personalized experience for the student. In some ways, the learner profile is the “holy grail” of next gen learning in that the better a learner’s “state” can be described the more personalized the learning experience could be. The problem of course is that no one knows exactly how to describe what a person knows or doesn’t know. A learner “state” must be generated and kept for every adaptive learning or assessment app. Is that state record standardized or interoperable? No, not at this point. In fact, such a description is probably considered to be the “secret sauce” of such products. If the products are tracking progress toward agreed upon competencies or learning standards (like the U.S. K-12 Common Core) then it makes the possibilities of exchanging learning profiles greater. But there are still a lot of nuances. More light will potentially emerge at the end of that tunnel as adaptive summative testing becomes more mature. Such testing will require well-defined “levels” of mastery to agreed upon learning standards. However, even in the short-term there are some interesting possibilities. For instance, in IMS right now we see tow more tractable paths to building and exchanging learner profiles. In higher education the IMS community is developing ways to exchange competency frameworks among cooperating products in the educational enterprise. In K-12 the IMS community is looking at taking small, but potentially powerful steps looking at profile items like reading level. IMS does have some past work on learner profiles and competencies as well that may become applicable: Learner Information Package, ePortfolio and Reusable Definition of Competency or Educational Objective.
Ramifications on Next Gen Architectures
Getting the architecture “right” for next generation digital learning presents numerous opportunities and challenges. Here are some questions that the education community needs to think through:
- Is there a software application that is best suited to be the keeper and provider (in terms of interoperability) of learner profiles? Should the learning management system or student information system do this? Or, perhaps a competency management product? What about the new category that the Gates Foundation has been encouraging, Integrated Planning and Advising Systems (IPAS). Yet another possibility is a competency-oriented assessment product.
- Same question for entering, storing and sharing personal needs and preferences, including the all important privacy preferences?
- What pragmatic steps do we take in moving our institutional or product architecture forward as we build toward this future?