Building Digital pathways to academic success
How User-Centered Design (UCD) helped college students graduate on time avoid risks + costs
EXPERIENCE DESIGN | USER RESEARCH | IXD | USABILITY TESTING | DESIGN OPS
ROLE
Lead UX Designer at BNED Inc.
TIMELINE
Oct 2017 - Dec 2019
TOOLS
Axure RP Team, Sketch, Invision, JIRA, Confluence, UserVoice, GoogleForms
CLIENT
PXU, a public university in Portland, Oregon*
*To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study.
Overview
Portland X University (PXU) is an urban school in the metro's busy business district.
As part of the university’s commitment to “improve the student experience through digital services innovation”, PXU uncovered the need to improve their Degree Planning experience to help students map out their 4-year academic plan.
Barnes and Noble Education was a perfect partner to co-create this solution. As the UX Design Lead on this project, I helped them introduce the Interactive Degree Planning tool through innovative design thinking, research and technology. The result was a robust, useful solution that delighted students and drove forward-thinking within the university.
“more than 40 percent of first-time, full-time bachelor's degree-seeking students at four year postsecondary institutions
are dropping out before finishing a certificate or degree within six years”
3000+ students / 12 advisors
Across pilot Study and MVP release (20000+ users post-MVP)
57 NPS
Likely to recommend based on the Net Promoter Score
Top 15%
of most usable applications based on the System Usability Scale
75% of users
Returned within a month
Problem Statement(s)
From our discovery and secondary research we found the negative experiences/outcomes below that systemically affect the future generation and their trust in the educational system.
“too many choices and too
little guidance”
STUDENTS AND ADVISORS ROUTINELY FACE THE PAIN AND INACCURACY OF TEDIOUS, INACCURATE PAPER-BASED DEGREE PLANS. A POORLY EXECUTED DEGREE PLAN COULD JEOPARDIZE OR EVEN ELIMINATE CHANCES OF A STUDENT GRADUATING ON TIME.
STUDENTS AREN'T ABLE TO RECEIVE THE PERSONALIZED HELP THEY NEED TO MAKE THE RIGHT CHOICES AT THE RIGHT TIME WHICH MUST TAKE INTO CONSIDERATION THEIR VERY UNIQUE CIRCUMSTANCES AND PREFERENCES.
ADVISORS ARE SWAMPED WITH THE NUMBER OF STUDENTS THEY HAVE TO ATTEND TO (~200 PER ADVISOR), AND HAVE TO ACCESS UP TO 4 SEPARATE APPLICATIONS TO GET A 360 DEGREE VIEW OF A STUDENT BEFORE THEY CAN CHART OUT A NEW STATIC DEGREE PLAN EACH SEMESTER.
THE BUSINESS PROBLEM FACED BY THE UNIVERSITY WAS THE NEED TO BOTH IMPROVE GRADUATION AND RETENTION RATES WHILE AT THE SAME TIME PREDICT COURSE SEAT DEMAND FOR THE FUTURE.
The Interactive Degree Planner is a platform that saves time for college students and academic advisers. It removes the pain of previously tedious, inaccurate paper degree plans with a clear, easy to use digital interface.
The app's planning engine uses powerful machine learning and AI driven algorithms to reduce redundant effort and provide choices, information and interventions at the right time to help students and advisors collaborate effectively.
Users & Audience
The main users for this platform were college students and academic advisors. We identified the most critical candidates to be students who transfer over from a community college and have previously earned college credits. They typically want to know which degree choice will be the quickest and cheapest to pursue.
We also had to keep in mind the academic advisors who know the ideal pathways by memory and help students create plans and pick the right courses.
“students are lost in the maze of choices and are making costly mistakes”
Roles & Responsibilities
My role was to conduct research, lead design efforts, test the experience for the entire product as well as provide a seamless design-development handoff and benchmark success metrics.
I collaborated closely with visual designers, product managers and engineering/QA teams to launch this project. I was determined to know the efficacy of the design solutions we put out. As it was something the company hadn't ventured into before, I went to work to pick simple yet effective methods to test users and track success metrics. Insights from early usability testing helped the product managers avoid unnecessary features and validated our custom built UI patterns. I ensured these went into a modular design system that streamlined development efforts.
“ Foresight, insight and oversight - Bringing it all together ”
“Today's new student majority—
demand a learning environment
that is more personalized;
learning that is more specific to
their individual needs
and goals. ”
Scope & Constraints
The challenging scope was to design a digital solution that could fuse student records with complex course structure relationships spread across multiple data stores – and yet be approachable and easy-to-understand for students.
These digital academic plans would reduce friction during the advising process and allow students and advisors to co-create and visualize 4-year degree plans. Insights generated from this information would help forecast future course seat demand for the university’s academic planning, staffing and administration purposes to efficiently adapt the teacher to student ratio.
As we broke down the high-level requirements into Epics and more granular user-stories, all teams were jolted by the scope changes and feature creep that threatened to derail the project.
The Process
There was a clear need to regularly validate our designs while delivering incremental functionality based on the initially scoped requirements. In the first two months, I managed to implement a cycle of user testing using hi-fidelity interactive design prototypes. Through this practice, we eliminated much ambiguity and struck a fine balance between user experience, technical effort and aggressive delivery time-lines.
“Step by Step..Term-by-Term”
Insights and user feedback were effectively fed into every decision along the way. We sought further to measure overall product success and satisfaction levels by choosing the two KPIs (Net Promoter Score and System Usability Scale) that strongly correlate to end-user acceptance and satisfaction. I convinced the Product Director to make this standard practice along the rest of the project time-line.
We also sourced qualitative data from feedback surveys to get a pulse of what students enjoyed and what needs may have yet been unaddressed. This information helped prioritise requirements and focus efforts on the highest value needs and pivot nimbly when required.
Based on the nature of the product, the discovery findings and my own secondary research we outlined a set of UX values that would define the product:
UX Values
[ Play ] Allow students to experiment and personalise their paths to graduation as suited their needs and interests.
[ Optimise ] Dynamically update degree plans based on a student’s records and remove manual effort where possible.
[ Compare ] Allow a student or advisor to clearly understand the pros and cons against the degree plans they create.
[ Prepare ] Provide adequate information and inform of anything the student needs to be aware of.
[ Nimble ] Reduce load/wait times and present the student with the information they need to make a decision such as changing majors/swapping classes etc.
For each incremental "feature set" I employed an effective quick turnaround lean UX design process which sort of looked like...
01 Discover
User Needs Research
Concept Walkthroughs
Problem Definition
Analyze
Synthesize
We had to identify the few segments of Student users. One of our biggest focus were transfer students who need to know how their previous earned credits contribute to degree completion.
We also had to keep in mind the certain types of advisors or university staff who would rely on the platform.
02 Design
Ideation
Interaction Design
Information Architecture
Wireframe & Prototype
Usability Testing
Validate + Track Success KPIs
Feature Feedback Loop
Cross-team collaboration
Feature Definition
Early on in the project, there was much miscommunication between the teams due to the nature of the project requirements; Four disparate systems at the university contributed to delicate complexities in the data - I had to ensure every technical feature we designed, struck a perfect balance of visual simplicity and user freedom. Many new team members simply didn't grasp the system from a holistic angle seeing only a small part that involved their piece of work. The design team brought order to the chaos by producing Interaction Flows, User Scenarios, Task models and Wireframes which significantly sped up project velocity. The goal was to ensure the distributed team could quickly understand and tackle possible and probable User Flows.
Our efficiency and execution brought design activities two whole sprints ahead of the development team.
I supervised visual design and set up a Team Axure project to rapidly produce prototypes for usability testing and actively set up and managed a dedicated space in Confluence to get the entire team's perspective and approval on our proposed designs solutions. We ran prototype demos on a regular basis to ensure that business objectives and technical feasibility were addressed, alongside my key responsibility of crafting a cohesive, intuitive and delightful experience. At this stage I was also responsible for scheduling and assigning design work using JIRA + Trello and also flagged roadblocks for my team on a weekly basis.
In collaboration with the on-site client engagement team, I wrote up test scripts and tasks for student usability testing. We also ran early design concepts and incremental product builds with academic advisors and university admin staff to validate our ideas and guide product decisions. The continuous cycle of testing helped the design organically evolve into a fine balance of form and function that scored an exceptional 82.5 System Usability Score. The design team brought back insights and findings from these tests to help define product features and UX improvements along the way.
03 Deliver
Visual Design Specs
UI Pattern Library
Design Guidelines
UI/IA Reference Catalogs
Frontend Review
Bug Resolution
While simultaneously designing for the next sprint or two, the design team was also expected to support ongoing development efforts - answering team queries, guiding front-end implementation, monitoring ADA (accessibility) compliance and helping to resolve UI & Interaction Bugs etc.
I proactively set up a bare-bones design system of modular UI Patterns (due to time constraints) and also documented and posted reference material to serve up a single-source-of-truth for the other teams. The visual designer was tasked with inspecting that the designs were faithfully reproduced according to the guidelines that I laid out.
Final UI and Experience*
A fun, vibrant theme with coded colours for category recognition. Familiar UI patterns sped up learnability and ease-of-use. Loading indicators, wait states and subtle micro-interactions made the platform feel fast and energetic for multi-tasking, Gen-Z students.
04 Measure
Survey Data
Monitor Usage Patterns + Analytics
Measure + Track Success KPIs (NPS, SUS and Satisfaction)
Product Feedback Loop
I paired up with the on-site business analyst to formulate a strategy on how best to establish our success metric benchmarks. We evaluated several user feedback and survey tools, but eventually had to settle for finely tuned excel sheets, due to shifting project management priorities.
We did manage to integrate the UserVoice platform to gather feedback/feature requests which continued to help improve and prioritise future product capabilities.
At this stage we were gathering user feedback and testing for both implemented product features as well as design prototypes. I pitched in to help with critical Product and Project Management duties like - prioritising and scheduling feature backlog, Usability/UX enhancements and bug resolution. Some of the implemented high value/low effort enhancements we had proposed and designed were:
STANDARDISED APPLICATION LOADING INDICATORS AND EMPTY STATE PLACEHOLDERS
A PLAN ACTION CENTER TO CONSOLIDATE 4 EPIC-LEVEL FEATURE SETS, DE-CLUTTER THE INTERFACE AND BE EXTENSIBLE FOR FUTURE REQUIREMENTS
ADA (Americans with Disabilities Act) COMPLIANT MESSAGING AND USER FEEDBACK HANDLING
Over the many tests and interviews we conducted, we correlated patterns and best practices. The immense amount of data was distilled into snapshots and snippets that were shared with the project team for much needed approval to the months of effort. They were also circulated across other teams and management levels to demonstrate the value, returns and success of a user-centered design approach - which transformed the standing process for all our other products in the organization.
RESULTS
Prior to the releasing the pilot, we captured student impressions & System Usability Scale (SUS) scores after each Usability Test. The median SUS score across 24 usability tests conducted over 7 months was 82.5; a leading indication that the prototype designs were accepted by student users. High SUS scores correlate very strongly with positive NPS scores and hence predict potential adoption and product growth.
“Tell me and I forget; teach me and I remember; involve me and I learn.”
- Benjamin Franklin
The user response to the Degree Planner Pilot was phenomenal. 28 pre-selected students and 5 advisors were asked to complete various tasks and openly explore the platform over 6 weeks.
In the first week, NPS & median SUS was +25 promoters and 68.75 respectively. Put into perspective, the average NPS across the software industry is about +41. According to comprehensive studies an SUS of 68 is considered to be an acceptable usability score for first time users. By the end of Week 6, the KPIs had jumped significantly; NPS & median SUS after the 6th week was +57 and 81.25 respectively! What's interesting here is that the early SUS scores measured using dummy design prototypes during usability testing mirrored scores against the actual developed product released 7 months later.
Testimonials from students captured in video interviews and survey responses were also overwhelmingly positive and encouraging for the team. Upon final roll-out, we also tracked Adoption Rates and Return usage patterns which showed a steady increase month after month.
Advisors were impressed with the speed and accuracy in which they could create and configure a complete 4-year degree plan for a student - an average of 8 minutes per plan effectively cutting 60% of the time it would usually take to do it on paper. They also loved the option to compare up to 3 degree plans at a time and the ability to share and co-author a plan with their students.
The following release added features, fixes and resolved bugs we identified from UserVoice feedback, survey data and user interviews. The platform was then offered to 3000 users on an open-access basis. The built-in analytics showed that 88% of open access users visited the app, 65% made at least 1 plan and 79% of students returned in a month.
what we learned
Probabilities over possibilities
Focus attention on high usage scenarios rather than try to solve every edge case.
Quick, iterative, user-centric designs based on actionable insights from user research and Usability testing
Ensure that a proposed design solution works (and works well!) before it is committed to development.
Minimum Lovable Product (MLP) over Minimum Viable Product (MVP)
A great user experience is not just about features and functionality, it also has to look and feel good while solving user needs.
Make UX measurable by tracking success KPIs like NPS, SUS as well as qualitative feedback often
Success metrics show whether or not we were able to effectively solve a user problem. Aim to continuously improve these scores.
