Modality Translation
Modality translation is the practice of converting information from one form of perception or interaction to another while preserving its essential meaning and function. This is a cornerstone of accessibility work and inclusive design.
Common Modality Translations
Sensory Translations
- Visual → Auditory: Screen readers converting text and interface elements to speech
- Visual → Tactile: Braille displays, haptic feedback
- Auditory → Visual: Captions, visual alerts for sounds
- Auditory → Tactile: Vibration patterns for notifications
Interaction Translations
- Mouse → Keyboard: Keyboard navigation for all interactive elements
- Touch → Voice: Voice commands replacing touch gestures
- Gesture → Button: Providing button alternatives for swipe actions
Beyond Disability
While modality translation is essential for accessibility, it benefits everyone:
- Situational Needs: Captions in noisy environments, voice commands while driving
- Learning Styles: Visual learners benefiting from diagrams, auditory learners from podcasts
- Cultural Preferences: Different cultures preferring different modes of communication
- Task Context: Choosing the most efficient modality for the current task
The Art of Equivalent Experience
Good modality translation doesn’t just convert information—it creates an equivalent experience. This means:
- Understanding the intent behind the original modality
- Preserving emotional tone and emphasis
- Maintaining information hierarchy and relationships
- Ensuring the translated version is as efficient and pleasant to use
AI’s Role
Large Language Models excel at modality translation because they can understand context and intent across different representations. They can extract the “why” behind information presentation and recreate that purpose in a different modality, making them powerful tools for creating more inclusive experiences.
See Also
- Epistemological Translation - Translating between different ways of knowing
- Building Ladders - The philosophy of creating tools that perform modality translation
- Alt Text Generation - A practical example of visual to textual modality translation
Related Terms
Building Ladders
The practice of creating technology that helps people climb to expertise and capabilities previously out of reach. Unlike bridges that assume you can already walk their path, ladders meet you at your current understanding and let you ascend, rung by rung, adapting to your unique way of knowing rather than forcing you to adapt to them.
Epistemological Translation
The process of converting knowledge from one way of understanding (epistemology) to another, making information accessible across different cognitive frameworks and modes of perception.
Glass Box
An approach to AI systems that prioritizes transparency, auditability, and human oversight. Unlike black box systems that hide their workings, glass box AI reveals its reasoning through structured outputs, scratch pads, and explicit decision steps that humans can inspect, understand, and modify.
Ladders
Tools, patterns, and systems that help people climb to expertise and capabilities previously out of reach. Unlike traditional tools that require extensive prerequisites, ladders meet you at your current understanding and let you ascend, rung by rung, to heights that were inaccessible before.