Learning Impact Blog
FRAMEWORK FOR EXTENDED TRANSCRIPTS: PILOT IMPLEMENTATION & RESULTS (APRIL 2017)
Authors: Joellen Evernham Shendy, Associate Vice Provost & Registrar, University of Maryland University College, and Insiya Bream, Assistant Vice Provost, Registrar Strategic Operations, University of Maryland University College
At University of Maryland University College (UMUC), a New Learning Model project is underway that focuses on building a student’s competencies over time through project-based learning—providing closer connections between the learning and expectations of employers in the field. In the fall of 2016, five of UMUC's Graduate School programs, including the Master of Business Administration and some Cybersecurity programs, implemented the new learning model. As part of innovating and developing a new learning model, UMUC realized that the documentation and evidence given to students regarding their learning would need to change as well. Telling a student story, via a new learning model, requires retooling in order to share the story effectively. For years institutions have heard from employers and others that the universally issued student transcript does a poor job of conveying what a student knows and can do, and is insufficient in providing evidence that helps an employer and others determine skills, competencies, and potential fit for an organization.
UMUC was one institution among 12 that received grant funding provided by the Lumina Foundation through the American Association of Collegiate Registrars and Admissions Officers (AACRAO) and National Association of Student Personnel Administrators (NASPA) Comprehensive Student Records Project. The project focused on piloting new comprehensive record models that display a student's educational and/or experiential background. These records would be in addition to, and not a replacement for, the original official transcript. The goal of the project was to identify models that could then be adopted by more schools—all helping to share the students' story in new and creative ways that speak to a more holistic view of who the student is and what they know. The launch of UMUC’s new learning model and the start of IMS Global’s Extended Transcript (eT) pilot were aligned and provided a natural segment of students that could give feedback on the first iteration of the Extended Transcript (eT) achievements record standard.
How We Did It
UMUC worked closely with Learning Objects through IMS Global’s CBE and Extended Transcript Workgroup, a part of the IMS Global Digital Credentials and Badges portfolio, to establish the initial data model that will allow institutions to manage and exchange competency and achievement data between systems, and ultimately, publish this data in an extended transcript achievements record store—for consumption by an employer, student, or other educational institution or organization. Through multiple iterations, Learning Objects, IMS Global members and staff, and UMUC’s IT team worked together to pull and display student competency data from the D2L learning management system (LMS). Learning Objects was able to digitally represent the competencies in a user interface (UI) that had a more modern look and feel, allowing the student to see, and interact with, evidence of their mastered learning.
Watch Learning Objects lightning talk from the 2016 Learning Impact Leadership Institute.
An overarching goal of the eT pilot was to solicit student feedback on the concept of an eT achievements record and their initial impressions of the utility of an extended transcript. Through the pilot, we were hoping to discover specifics on the following:
- What did students think of the virtual document itself?
- Did students find the content useful?
- How would students use the information?
- Did they understand the purpose?
- How could a digital document like this help carry them towards their next professional goal?
Over the past five months, UMUC’s eT pilot students were able to access an eT achievements record displaying mastered competencies through an icon link in their D2L classroom. Through a "soft rollout," the link was activated and a week later students were notified via email of the pilot. The email included information on the overall purpose and asked them for their feedback via a survey by a link on the eT itself and in an email. One of the survey questions included whether or not the student would be willing to participate in a focus group in early spring 2017 to provide more detailed feedback on the eT. A “help” icon on the eT itself explained what UMUC hoped to achieve and learn through the pilot. The internal marketing team supplemented the email and eT with a web page that included general pilot information and an FAQ for the wider campus community. Out of approximately 2,000 pilot students that had access to the eT, over 800 students accessed the document.
What do Students Think of the eT Achievements Record?!
UMUC students responded throughout the pilot period to the survey, sharing their thoughts and recommendations. The majority of survey respondents made it clear that they liked the graphical look and feel of the eT. Working with a third-party vendor, Learning Objects, allowed UMUC to help create the design and feel of the eT by utilizing a variety of input from various University stakeholders and by modifying the document until we were satisfied with the appearance and ease of use for students. Leveraging Learning Objects' technology brought a 21st Century user experience for students to the eT achievements record.
Survey results showed that over 70% of respondents found the eT content useful. Students noted that they could use the information on the eT for resume writing to explain or communicate their competencies and achievements in the academic program. Students felt they would share this mostly with future employers, but also with other admissions offices, their current employers, family, and friends. We asked survey respondents if they recommend we open this up to the larger UMUC community and the answer was an overwhelming YES.
UMUC also conducted a focus group to drill down into some of the survey questions to get additional information to help inform the next iteration of the eT. In general, the focus group students were positive about the eT and 84% recommended UMUC implement an eT-type achievement record for all students. They shared additional details about the appearance and content of the eT. Students felt that the eT would give them more confidence when pursuing job opportunities—helping them to articulate with clarity and confidence their competencies. Several students remarked that their own understanding of their achievements was influenced by the content they viewed in the eT record.
The team of academic and technology colleagues working through IMS Global have been integral to the success of this effort.
"We extend hearty congratulations to UMUC for this critical first step connecting their students to employers through digital credentials. The Extended Transcript Achievements Record forms the foundation of an institution’s digital credentials strategy. UMUC and their peer leaders Capella University and the University of Wisconsin-Extension are paving the way to tomorrow’s marketplace of verified credentials."
—Mark Leuba, vice president,product management, IMS Global Learning Consortium
What areas of improvement did the students identify for the next version of the eT? Although most students did not feel the digital document was too lengthy, some felt the length would cause employers to gloss over the information. One idea shared was to allow students to curate and create their own “view” of the achievements record that was specific to a particular employer, organization, or industry. This would allow them to share a portion of, but not the entire, eT and highlight accomplishments most relevant to the students’ goal(s). Focus group participants noted that it would be interesting to see the competencies organized from an industry or sector perspective and to break down the “level” of competence based on position requirements such as entry level, middle, or C-Suite management. There was also a student desire for the opportunity to contribute to the document by way of adding their own supporting content, providing them an option to make adjustments for their unique situation.
Overall, the feedback collected through the survey showed that students understood the eT achievements record and would use it in multiple ways to reach their goals. While the respondents felt understanding their own accomplishments and articulating them were a huge benefit, they did feel most strongly that the connection with employers was critical. One particular comment from the survey highlights why we need to continue to build these bridges, via new ways to evidence learning, between institutions and the workplace:
“The document is a great reminder of what a graduate is capable of doing. Instead of saying I walked away from college with an x.xx GPA, I can say explicitly, this is what I am capable of doing; these efforts gained me these competencies.”
Future State of the Extended Transcript
UMUC successfully closed the pilot at the conclusion of the winter 2017 term. Our next steps are to leverage the work we are doing as an initial contributor to Credential Engine with adding our program competencies to the nationwide registry. This will provide additional linking and connection opportunities. UMUC is also working towards finding better ways to “match” a student's competencies with employer needs by continuing to work with innovative and dedicated colleagues through IMS Global in their digital credentials and badges program. As far as a next version of the extended transcript achievements record, we look forward to incorporating our students feedback and reaching out to the employer and recruiting community to build closer ties and feedback loops, ensuring our students will be armed with 21st Century digital evidence of their learning which presents a more holistic picture of their fit to organizational needs.
For more details on the next phase of the Extended Transcript, see Achievements Records Standard and SkillsCenter Search™
“I can see for miles and miles and miles.” – The Who
We’re all impatient and getting more impatient all the time. We’re all looking for time well spent. But, we also want to be prepared for the future. How do you achieve both?
I love the work of the IMS Global Learning Consortium because our members are on exactly that quest—saving time and money now while laying the foundation for the future. If that is what you are looking to do as an institutional leader or as a product leader, you will not be disappointed in the upcoming Learning Impact Leadership Institute (#LILI17).
Every year several executive level or aspiring executive level leaders tell us that the Learning Impact Leadership Institute is the best event they’ve attended, which is a strong statement because there are a lot of great events in the educational technology space.
What’s different about the Learning Impact Leadership Institute is that at the heart is the IMS collaboration, a set of education industry leaders that are inspired and motivated to move the industry forward in a set of very important core areas: digital content, learning apps/tools, e-assessment, student data, academic data, digital micro-credentials, competency-based education, educational badges, learning analytics and digital accessibility.
By far the best thing about IMS is the leadership community that gathers around and collaborates on the major barriers to implementing these key technologies at scale. At IMS we recognized early on that the “standards” needed to remove the barriers cannot be addressed independently from being a catalyst for innovation. We needed to do both and are doing both. Every day.
You are really missing out on something special if you don’t attend #LILI17. IMS holds quite a few meetings around the world and presents at nearly 60 conferences each year but LILI is the only time and place where you can within a pretty short timeframe—a few days—see for miles and miles and miles ahead—all while you are most likely walking away with something that will save you time and money in the near-term, too.
To see what happens at the Learning Impact Leadership Institute watch this 3-minute video of the 2016 NGDLE plenary talks.
discuss the future of Open Badges during the 2017 Summit on Digital Credentials & Badges.
“Cause I’m not like everybody else.” —The Kinks
Standards are the foundation for diversity at scale. Just look around you . . . and then realize that all this diversity came from a common standard that we call DNA. Lack of a foundation in replicable standards means lack of scalability. Think the plethora of grant projects ending up in one-off successes.
Every teacher and every student are not like everybody else. The future of education isn’t massification; it’s personalization. Personalization at scale.
During IMS Global’s recent quarterly meeting in Orlando, we had the pleasure of hosting a Summit on digital micro-credentials, including Open Badges (a standard and community that IMS Global is now facilitating after a recent handoff from the Mozilla Foundation).
Micro-credentials and Open Badges are key foundations for scaling personalized education. They are a way to enable and recognize personalized learning and life pathways. In essence, they are a way to create and recognize each person’s unique story.
Educational systems and educators that embrace and perfect micro-credentials will be the ones leading us to the personalized and scalable future.
The Summit by itself would have made anyone’s week. But there was much more at the quarterly meeting.
Agreement was reached on a breakthrough competency and learning standards framework (Competencies and Academic Standards Exchange) that enables digital resources to be mapped to achievement frameworks and thus enables finding and selecting resources based on those frameworks.
A set of leading educational content providers established a breakthrough open source collaboration to enable ubiquitous availability of OneRoster®.
The emerging field of learning analytics continued its advance thanks to an incredible cadre of leaders in IMS Global who are making learning analytics practical via Caliper Analytics® and IMS Global’s vibrant partnership with ADL/xAPI.
Among the most exciting sessions of the week were the Learning Tools Interoperability® (LTI®) Showcase and Hackathon. Suppliers and institutions are doing amazing things with LTI apps that can work across platforms. And, yes, there were some very tangible signs that the market is beginning to catch up with the more advanced features of LTI and LTI v2.
If you weren’t there and you’re not like everybody else come join us next time! You’ll see and lead the personalized scalable future of education and learning.
On Monday, August 15, at the IMS Global quarterly meeting at Utah Valley University, the leaders of the Advanced Distributed Learning (ADL) initiative, responsible for the Experience API (xAPI), came together with the leaders of the Caliper Learning Analytics Framework (Caliper) from IMS Global. ADL has joined as a Contributing Member of IMS Global and the parties are committed to exploring a unified path for xAPI and Caliper. This may mean alignment at some level or potentially even convergence.
There were roughly 50 experts in the room for an entire day on August 15, and I believe it is fair to say that both ADL and IMS were very pleased with the eagerness of participants to cooperate and depth of discussion. Presentations from the meeting are posted here and there should be a synopsis of the comparison posted soon.
At the next IMS quarterly meeting November 7-10, 2016 at Arizona State University, there will be a follow-up session on Wednesday November 9 open to the public (registration required). I expect it to be another great session that should lead to some tangible prioritization and next steps. On November 8 there will also be a day-long Summit on Creating an Educational Analytics Ecosystem.
IMS and ADL are committed to collecting and achieving public input as the process moves forward. We are likely to create a public forum to encourage the public community to comment and also help crosswalk the information models. Stay tuned to ADL and IMS announcements and twitter feeds (IMS twitter feed is @LearningImpact). Currently there is a survey for those that have views on the potential convergence of the specifications here if you would like to participate.
I attended the entire August 15th session and would like to provide my sense of the big picture. I’m not the technical expert, but I have a long history in the applications of standards, including those that evolved into SCORM, and of course the dramatic growth of IMS standards adoption the last 10 years in the education sector.
I am extremely impressed with both the xAPI and Caliper work. While they are perceived by many as “solving the same problem” my own take is that this is largely because there is still quite a lot of variability in the Learning Analytics field in general. Thus, almost any set of technical work that deals with transcribing, sending, logging and analyzing data during any form of e-learning is considered “learning analytics.” From my perspective, while xAPI and Caliper certainly have some overlap, they are largely complementary at this phase in their respective evolution.
At the risk of over-simplifying things, my impressions of the strengths of each specification are captured in the table above (while there are four “strengths” shown for each specification they are not meant to be compared in four dimensions - the above are just lists of the primary strengths). The strength of xAPI is as a way for a single application to log freeform natural language statements and for that data store to be retrievable by the logging entity. The strength of Caliper is as a way to aggregate agreed upon events across a set of applications to enable processing across the aggregation of data.
It is also very clear that while neither specification “requires” the body of work from which they came (SCORM for xAPI and IMS standards, like LTI and OneRoster for Caliper) that these foundations greatly influence each. For instance, in the case of Caliper the co-existence of the other standards in an implementation means that Caliper can simply complement a slew of outcomes and context data that are already flowing via LTI. Again, because the Learning Analytics field is rather loosely defined at this point with widely varying “use cases” one can easily get confused about whether they even need Caliper or should just be using LTI to it’s fullest potential (for instance, LTI can already send data to an endpoint that is not the LMS).
I will also note that in the IMS Communities of Practice relating to data and analytics in IMS, we are seeing a very high priority need to simply be able to see all the data visualizations in one place. The level of data processing may be fairly minimal – but the humans in the equation need a better way to integrate visualizations that may already be available from various products they are using.
Bottomline, in my humble opinion, is that as we move forward on alignment and/or convergence of xAPI and Caliper I think we need to consider the use cases more carefully and, in essence, better define the “categories” or “scenarios” of Learning Analytics. While that may seem like “a step backwards” well, I think we need more clarity. The good news is that I don’t believe we will have any problem given the wealth of actual implementations we are working with in IMS and ADL.
Our goal in IMS is to enable serious school districts, higher ed institutions, states and nations to implement high impact learning analytics use cases and to enable a plug and play ecosystem of products that provide the data needed to solve those use cases. Either specification is good enough to begin doing some work. But, in order to do what we need to do we need a very large interoperable ecosystem of products, users, and researchers that can all invest together to get to the understanding we wish to collectively achieve. I would see elements of xAPI and elements of Caliper as both being essential in that goal.