Asset Management & Analysis of DSO Training
ELS Educational Services
As organizations grow, training programs become ever more critical to their success. Likewise, so does their document management system, which houses vital information and assets needed to ensure consistency across a global organization, access to important resources and guidelines, and a means to evaluate how your people are learning from and using those resources to serve internal and external stakeholders.
​
This case study explores how we, a small team of 3 dedicated trainers and directors, tackled those very issues, realizing the important role our DSOs played in day-to-day customer service and operations.
The Challenge
Immigration advisors (DSOs) at our 40+ locations throughout the US were struggling to find, understand, and use critical immigration compliance resources on our corporate site and SharePoint system.
Scale
In large-scale universities, an International Student Services department may include between 10-15 advisors.
In 2016, ELS had 60 locations nationwide, with an average of 2 advisors per location.
By 2019, ELS had reduced to 40 centers and roughly 40-60 advisors nationwide.
Compliance
In order to maintain compliance with the Student Exchange & Visitor Program (SEVP), a department of Homeland Security (DHS), immigration advisors must:
-
Comply with timely actions & updates to student records.
-
Manage student records within the Student & Exchange Visitor (SEVIS) database.
-
Advise students on regulations and policy as it relates to the visa classification.
The Team
Operations -
-
US Director of Ops
-
Director of SEVIS Compliance
-
2 Trainers/Center Directors (myself included)
​
Information Technology Team-
-
Director of IT
-
Manager of Database Administration
-
IT Support
​
Impact
If SEVP approved institutions fail to meet compliance requirements, their ability to issue I-20s needed for student visa issuance can be revoked.
ELS’ largest customer segment is comprised of F1 students who come to the US to improve language skills or become degree-seeking students.
Without the ability to issue I-20s, ELS would not be able to sustain itself as an institution.
My Role/Activities
-
UX Research
-
Training & Compliance Support
-
Asset Design
-
System Design Propositions
​
Research Methodology
-
Surveys with common advising scenarios and document location tasks were created to assess DSOs knowledge & ability to retrieve resources.
​
-
Interviews were casually conducted with 10 DSOs whose survey results placed them in the highest & lowest percentiles.
​
-
Contextual Inquiries took place during resource, policy, and systems training to understand DSO frustrations.
Research Goals
-
Use a variety of methodologies to capture feedback on needs, expectations, and frustrations.
​
-
Use surveys to assess DSO knowledge and identify patterns in learning gaps.
​
-
Discuss, one-to-one, key issues and expectations of DSOs.
​
-
Incorporate contextual inquiry into training and observe for areas of frustration or confusion during systems training and while accessing our document management systems.
Research Insights - Needs
-
90% of DSOs wanted one place where they could quickly find and retrieve advising documents & forms.
​
-
7 out of 10 DSOs needed clearer, scenario-based advising docs.
​
-
100% of DSOs expected that complex policy and regulation would be simplified for use when interaction with non-native speakers.
Research Insights - Pain Points
-
5 out of 10 DSOs felt ill-prepared to advise students on critical issues and provide real-time solutions.
​
-
100% felt that they lacked time to read extensive policy, but the information was presented this way.
​
-
4 out of 10 DSOs disliked advising students due to language barriers, information complexity, and recurring status issues.
"First, Seek to Understand..."
Research Goals
-
Use a variety of methodologies to capture feedback on needs, expectations, and frustrations.
​
-
Use surveys to assess DSO knowledge and identify patterns in learning gaps.
​
-
Discuss, one-to-one, key issues and expectations of DSOs.
​
-
Incorporate contextual inquiry into training and observe for areas of frustration or confusion during advising training, systems training, and while accessing our document management systems.
Research Methodology
-
Surveys with common advising scenarios and document location tasks were created to assess knowledge & ability to retrieve resources of 60 DSOs.
​
-
Interviews were casually conducted with 10 DSOs whose survey results placed them in the highest & lowest percentiles.
​
-
Contextual Inquiries took place during resource, policy, and systems training to understand DSO frustrations.
Research Insights - Needs
-
90% of DSOs wanted one place where they could quickly find and retrieve advising documents & forms.
​
-
7 out of 10 DSOs needed clearer, scenario-based advising docs.
​
-
100% of DSOs expected that complex policy and regulation would be simplified for use when interaction with non-native speakers.
Research Insights - Pain Points
-
5 out of 10 DSOs felt ill-prepared to advise students on critical issues and provide real-time solutions.
​
-
100% felt that they lacked time to read extensive policy, but the information was presented this way.
​
-
4 out of 10 DSOs disliked advising students due to language barriers, information complexity, and recurring status issues.
Rethinking Asset Management & Design
Goals
-
Use feedback from DSOs to drive improvements to doc management, presentation of advising guidance, and improvement of training experience.
​
-
Assess DSOs in survey format- providing instant feedback (scores) and a ranking tool for our use identifying DSOs that needed additional support.
​
-
Evaluate how centers are managing digital student records, student files and paperwork, and using training assets to present information consistently across our 40+ locations.
Improving Document Management
Issue: Resource retrieval of critical forms and advising information was one of the most common complaints among DSOs.
Background: Training and advising documents were housed in two separate places:
-
Manuals & Forms in our corporate site, which included key documents for center compliance, corrections and form letters, and the regulatory manual that DSOs were to use in advising and training.
​
-
DSO Portal in SharePoint: this portal housed important policy-based guidelines, training webinars, and status-critical procedures and reminders for DSOs.
Applying Research: Observations & Expectations
Research and feedback we gathered from surveys and interviews showed that DSOs felt very confused when it came to finding the information they needed since it was spread across two sites.
Contextual Inquiry revealed that DSOs would often go to the wrong place when asked to retrieve a specific document. We also saw that many DSOs, even after completing training that included walkthroughs of the two portals, would use our SEVIS Team email to ask where they could locate or retrieve important documents.
​
This was a major point of frustration and confusion among DSOs across our 40+ centers.
​
It was evident that we needed to find a way to streamline and simplify asset management to help DSOs feel confident and prepared in their daily advising duties.
Simplify
Taking an incremental approach, we started with a simple reorganization of documents and looking at ways we could simplify and visualize them.
Streamline
The second phase of our plan was to streamline and merge documents into one portal, eliminating the need for the other and reducing the overall number of assets (getting rid of irrelevant or legacy stuff) available.
Model
The third and final step of the proposition included modeling the Study in the States linked index to reduce friction when searching by topic and provide continuity between our internal document management system and external sites that are critical for DSOs to know and understand.
Simplifying Advising Assets & Training Materials
Solving through Simplicity
Issue: DSOs didn’t have time to read lengthy regulatory or policy information, they needed the information synthesized in a way that they could easily help students understand responsibilities and options.
​
Context: Advising documents included regulatory language that was vital for DSOs to understand but were also text-heavy and difficult to disseminate to non-native speakers.
​
Goals: As DSO and Directors, our goal was to find a way to simplify complex advising information into a more visual format that allowed for scannability, reduced cognitive load, and aided in quick comprehension of required steps and actions.
​
An Emphathy-first Approach
As DSOs and Directors, we understood first-hand the challenges of each day, balancing the needs of the students/customers, against the needs of managing a center. We were constantly juggling priorities with walk-in counseling sessions (we did not use appointment systems) for panicked students who most often were coming to us after things went awry.
​
As trainers who did the very job we were teaching, we were extremely emphatic to the needs and pain points of our DSOs.
​
This ability to relate, to listen, and to really put ourselves in the place of our trainee users, allowed us to approach this problem using the insights we gained and observed in research paired with a good dose of empathy.
​
Designing for the User & for Usefullness
Our research and discussions with DSOs offered insight into some of the most common and more complex advising scenarios they were encountering on a recurring basis.
​
They expressed feeling confused as to which actions were best suited for patterned situations, often feeling that the ambiguity of each case made the steps they should take less clear.
​
Added complexity came from the fact that our program ran monthly rather than most traditional programs that use a block, quarter, or semester system. This meant that all the regulations were written with traditional programs in mind, and our job was to make them applicable to our schedule and policy.
​
Students would also read information online that was only applicable to students in semester-based programs or get advice from a friend prior to meeting with an advisor, which often led the student to distrust what they were being told.
​
For us, this meant we needed to provide simple work and advising flows to visually walk the DSO through their process, so they could confidently advise, while also being easily understood by students of varying language levels.
Before
After
Grace Periods simplified into the three flows that both the regulations and our policy supported, while also accounting for variables like Program End Date.
Why this Matters ...
If students are misadvised or misinterpret their departure date and overstay their grace period it can negatively impact their ability to get a visa for future return or/and lead to detainment and deportation.
Before
After
Similar to Grace Periods, these critical and timely actions are simplified into the most common scenarios and steps so that DSOs and students can easily understand their options and any possible limitations of their choice.
Why this Matters ...
These are very different actions to take on a student's record (I-20). Shortening effectively ends the I-20 so it is no longer valid for use, while Termination for Authorized Early Withdrawal puts the I-20 into a state of pause, allowing it to be reactivated for the student's return within 5 months. For the student/customer, this is the difference between being able to return more easily and resume studies and having to start the process over with a new I-20.
Before
After
We simplified "What If" scenarios by topic rather than presenting information in clusters. The Excessive Absence Flowchart shown above is a guideline for center-wide communication, so both academic and administrative staff could proactively help students through communication, counseling, and awareness. Simplifying by topic also meant that DSOs could more easily search for, scan, and apply the guidelines.
Why this Matters ...
Advising students who were at or nearing the category of excessive absences was a major pain point for DSOs. This was a complex issue, that often involved mental and physical health issues, lack of transit options, or lack of awareness.
​
For DSOs, it was critical to get to the root of the issues through proper counseling, but also make students aware of how their status may be impacted and options they have to maintain that status lawfully.
Implementing & Evaluating
Present & Gather Feedback
Share findings and test solutions with DSOs.
Assess Growth & Knowledge
Implement ways to teach and support DSOs in asynchronous learning and assessments.
Evaluate Change
Evaluate the change over time through analytics, audits, and training feedback.
Initial Feedback
As a team, we rolled out the new assets over a two to three-month period, presenting our findings and proposed solutions to our family of DSO in monthly webinars and as part of ongoing training.
​
DSOs were delighted with the changes and very vocal in their support of how these work and advising flows would ease the complications they often felt when advising, making their daily work a bit easier.
​
But we wanted to also assess how the use of these assets would help DSOs lessen knowledge gaps, understand information more succinctly, and advise more proactively to avoid issues, alerts, and negative terminations.
Adding Data Analysis to the Assessment Model
Issue: There was no system in place to assess DSO knowledge, so while we saw issues coming from specific groups of centers, we had no way to identify what advisors were struggling with.
​
Context: DSOs, who most often were also Center Directors and Student Advisors, changed frequently, namely at the advisor level. This meant we were training new DSO constantly and managed ongoing training with 60+ DSOs (on average) at varying levels of knowledge throughout the year. There were only 3 of us on the SEVIS Training Team- and up to 60 DSOs.
​
Goals: The purpose of adding data to bi-monthly assessments was to help us easily identify DSOs who were struggling and needed additional training support, while also providing correct answers for scenarios we tested to keep DSOs informed and aware of their own knowledge gaps.
​
Analytics: I took our bi-monthly DSO raffle and turned it into an online assessment tool using Google Forms and Survey Monkey. Using their analytics, we were able to see:
-
the number of DSOs completing the assessments
-
how DSOs (new and seasoned) were understanding and applying ongoing training and information
-
patterns in knowledge gaps
-
progress of individual DSOs over time
-
whether struggling DSOs were participating
​
Simplifying Compliance Audits
In an effort to evaluate how DSOs were using required forms, documenting student files, and managing records, we performed audits of centers at random.
Problem: Audit requirements were unclear to centers, often resulting in centers sending the entire student file rather than necessary docs. Audits were taking over 2 hours per center.
Solution: I compiled all the requirements and created a simple checklist and email template with explicit instructions to centers.
Result: The average audit time went from 2 hours to 45 minutes per center, and many centers used this checklist as an insert in the student file to maintain consistency in their records keeping habits and local training.
4-2019
Improving Consistency in Training
As we were testing our solutions, a pattern of concern that emerged was that DSOs felt there were some inconsistencies in the materials covered during training. After discussion with DSOs and the SEVIS team, I proposed simplifying our training steps into checklists to create consistency, clarify as a guideline for both the trainer and trainee and ensure any materials that weren't covered (due to time constraint) or needed follow-up could be easily accounted for in the form.
​
This was especially critical for Systems Training, as shown in the example provided, as it was more complex when working between our Batch and RTI system and the SEVIS Database.
​
Problem: Training of 40-60 DSOs at various stages of their program was leading to inconsistencies in the material covered. This left DSOs feeling ill-prepared to advise and manage student records.
Solution: To create more consistency in our training program, I updated and compiled all the requirements into checklists that both trainers and trainees could follow.
Result: DSOs called and wrote to express gratitude for increased transparency in the training materials and clear guidelines on what they are expected to learn, know, and apply.
Improving our Document Management System
Proposition: Hyperlinked Index
-
Modeled around SITS (Study in the States) SEVIS Help Page so the two feel congruent
​
-
Keep visual layout simple so not to overwhelm
​
-
Keep text in subtopics bulleted and contextual
​
-
Hyperlink topics to PDF versions of training materials (PP, Raffle Q&A, etc.)
​
-
Meant to save DSOs time in problem resolution and required actions in Corp/SEVIS.
​
-
Merging of Manuals & Forms with DSO Portal
Current Systems
Manuals & Forms
-
Poor visibility and unclear hierarchy with layered links.
-
Poor discoverability of key documents (out of site, out of mind).
SharePoint
-
Difficult search functionality.
-
Unclear organization of materials and files.
Proposition
Hicks Law & Heuristic Evaluation
When a user visits the DSO Portal they immediately see the this hyperlinked index of key topics to select from.
​
Hicks Law: Regulations related to student visas are vast and dense, so in an effort to not overwhelm the user with too much information or too many choices, common advising scenarios and their related guidelines and processes are simplified into topical categories.
​
Heuristics:
Terminology used is identical to terms used while training, so users are immediately familiar with the topics and subtopics.
​
Cognitive Load is reduced with the removal of unnecessary language, guidance, and actions that are unrelated to a DSO's work within a non-degree program.
​
Consistency & Learnability is considered when modeling our internal system after Study in the States, which is one of the official DHS sites that advisors should be accessing for advising.
​
Context for Recognition is offered by categorizing like actions under their related topics.
​
Navigation is simplified by topic making search options clear and contextual and adding to discoverability.
​
Help & Documentation are provided in the form of linked documents directly related to the selected topic or subtopic, allowing for immediate access to required docs in the moment the user needs them. Recorded Webinars are also available for review of topics and training.
Lessons & Reflections
Asset Management is Never Ending
When it comes to building, designing, organizing, updating, and maintaining an Asset Management System, it is important to understand early that this is never 'one and done.' This living, breathing, evolving ecosystem of materials and resources will always need to be reexamined and iterated on to meet the changing needs of how your users learn, what they need to learn, what resources are most relevant, and how those resources and assets are shaped by outside forces, be it trends or regulatory changes.
I Love to Make Things Learnable
I have been told by many students, trainees, and managers that I am particularly good at taking complex information and making it understandable. This comes from my days as a course designer and instructor.
​
I recall professors and instructors bragging about how many students were failing their classes. They found it comical. I was of the mind that if a large percent of your students were failing your class, it was in fact, the instructor, who was failing.
​
The role of any good instructor, trainer, or UX designer is to understand the needs of the users they are creating for and then find ways to make even the most complex information or process simple enough that users can feel a sense of satisfaction and success when they are able to easily do something new.
Documentation is My Jam
Alright, if you made it this far you might as well meet my inner documentation-nerd. My name is Gwendolyn, and I love checklists, workflows, and more recently, design specs. I love the clarity and consistency that comes from well documented, easy to understand and apply processes. And, I thoroughly enjoy being part of the teams that build-out and maintain these systems.
​
I am uber-organized in my professional life, but more than that, I like keeping things organized for others. If users are able to easily learn, do, or apply something based on my documentation, flowchart, or UI, then I feel a sense of achievement.