PALMs Authoring Tools

A web-based platform for educators to create and share perceptual and adaptive learning modules

Figure 1: A selection of experimental apps made by the internal version of the platform.

Metadata

Customer
Advanced Distributed Learning,
US Government

End User
DoD Educators

Objective
Enable educators to create adaptive training content

Solution
Responsive web-based authoring platform

Timeline
2016—18

Status
Live on ADL’s warehouse (Demo available)

Team
Domain Experts x 2
Project Manager x 1
Engineer x 4
Design Engineer x 1 👋

My Role
Product design
Interaction design
Code templates
QA (design & usability)

Impact
The product’s timely, high-quality delivery unlocked major revenue through multiple awarded contracts.

Figure 2: Introductory shots as a quick visual overview of the project.

Quick Overview

PALMs address gaps in traditional education by focusing on pattern recognition, factual, and procedural learning. ADL commissioned Insight to develop a platform that allows DoD educators to convert course content into PALMs.

Problem Statement

Mastery-based education demands more than didactic teaching. Given PALMs’ effectiveness, how can we eliminate the technical barrier for educators? How do we build a system around proprietary tech, under a high-profile client, while unlocking revenue potential?

Strategy, Planning, and Ideation

We knew how to build PALMs but not a system to replace ourselves. I led research, strategy, and roadmap development, driving group discussions and whiteboarding exercises. I spearheaded reverse-engineering efforts to define interaction models, UI, and workflows.

Figure 3: The preliminary components backlog (top left), the initial proposal (top right), the development schedule spanning years one and two (bottom left), and the features and activities saved for the 3rd year (bottom right).

Based on ideated concepts involving assumptions and hypotheses, I built a click-through wireframe system (over 50 screens). The goal was to have a visual guide for designing a cohesive and well-designed experience for educators.

Figure 4: Various maps drawn to aid the decision-making and development processes. The map on the right reflects the latest iteration of the platform.

Based on ideated concepts—which involved assumptions and hypotheses—I built a click-through wireframe system (over 50 screens). The goal was to have a visual guide for designing a cohesive and well-designed experience for educators.

Figure 5: The systems modeling of The Main Editor, early iterations.

Then, through internal critique sessions, pilot educator interviews, and direct observation of test users, I noticed a few flaws in the system that led to revisions.

Example Hypotheses

✅ The building process should be top-down.
✅ Structural elements should come before attributes or relationships.

❌ We should have two creation paths: assisted and expert

Prototyping & Building

After aligning stakeholders through wireframes, we entered development. I:

Built responsive UI templates, later converted into React components.
Rigorous testing across devices, browsers, and screen sizes.
Gathered feedback from team members (5), pilot educators (2), and novice users (3) in both unmoderated and task-primed sessions.

Key learnings:

Behavioral Insight: Users optimize workflows iteratively, refining modules and assets over time.
Usability Insights: Small inefficiencies compound, significantly impacting UX.
Invalidated Hypothesis: Wizards were slow for high-volume materials, leading us to refine the main editor instead.

Example Design Challenge
A Search Compatible Tree

Figure 6: The tree model for browsing PALM structure in its initial state features indentation, entity-type cueing symbols, and collapsibility. In search mode, the view changes to a flat list where search results feature their bread-crumb-like addresses, cueing where each entity belongs and how to navigate to that location. This view is helpful in cases where various problems from different categories share the same name.

Example Design Challenge
An Efficient Interaction Model for Managing Images and Tags

Figure 7: The images tab is the entry point to the image management system. The tags tab is the entry point to the tags management system. Through a familiar presentation, the learning curve for utilizing them becomes shallow, and the effort for reaching them is equal.

Example Design Challenge
Templatization of The Creation Process

Figure 8: Defining problem attributes (e.g., prompt question, distractors, feedback text, etc.) in a template and assigning that template to a problem saves users from taking redundant steps in similar problems.

Example Design Challenge
Automating Distractor Insertion Using Lists

Figure 9: Each problem comes with correct answer choices and multiple distractors. Defining multiple distractors for numerous problems can be a cumbersome task. Instead, defining a list of distractors and using the list name in the distractor fields can automate the process. A behind-the-scenes mechanism excludes the correct answer at each fetch.

Example Design Challenge
Embedding Progress Reports Into Essential Screens to Avoid Additional Screens

Figure 10: Module launcher page features the progress and mastery metrics. Progress reports, triggered after each definable block of problems, allow users to take a break and reflect on their performance.

Refinement

I matured the product's visual language in phase. In this phase, we focused on increasing the components' quality by making them efficient and performant. Our further usability tests surfaced several edge cases that we had to address. In this phase, we also received several delayed—or post-delivery—feedback that we needed to address. While working on those, I also started polishing the UI and paying extra attention to the visual language of the product.

Figure 11: Various modals requesting user input for basic and advanced functionalities.

Outcomes

The timely and quality delivery of the product positioned Insight Learning Technology for significant revenue growth.

The PALMs Authoring Tools system is now available for government use on desktop and mobile devices. This powerful authoring system enables easy learning module creation and leverages perceptual and adaptive pedagogical methods. Content authors can upload images and text, create and prioritize questions, set question timers, etc.*

Throughout 2017 and 2018, the U.S. Marine Corps (USMC) conducted usability testing of the PALMs platform and its associated authoring tool. This work led to a more robust instrument evaluation to support USMC education and training requirements**.

* Perceptual and adaptive learning modules (palms). ADL Initiative. (n.d.). Retrieved January 12, 2022, from https://www.adlnet.gov/projects/palms/
** Freed, M., Folsom-Kovarik, J. T., & Schatz, S. (2017). More than the sum of their parts: case study and general approach for integrating learning applications. In Proceedings of the 2017 Modeling and Simulation Conference.