My learning 0 of 20 lessons complete
Start lesson 1 →
15 documented entries

Cross-Device And Physical Interaction

Touch, haptics, sensors, camera, permissions, offline states, and device adaptation.

Browse all problem families
Learn the family

How does the task survive a change in device, place, or input mode?

The same outcome may cross touch, camera, voice, keyboard, sensors, and offline moments. Design each transition as part of the task rather than a separate surface.

Use this method

  1. Map the task and the environments where it occurs.
  2. Identify the device capabilities and permissions each step uses.
  3. Provide a visible alternative and recovery route for interruption.
  4. Test continuity across input modes and connection states.

Compare the outcome

Strong: A visitor starts a room reservation on a phone, scans a lobby code, and sees the same reservation status at the kiosk.

Weak: A visitor completes a phone reservation, then the lobby kiosk requests a new booking because the two surfaces show different status.

Try it: Trace one task across two devices and identify the shared state, permission request, interruption, and recovery point.

Read these patterns first

  1. Responsive navigation adaptationA product supports multiple viewport widths, device classes, orientations, split-screen sizes, or foldable postures.
  2. Offline mobile retryA mobile user action can be saved locally and retried after network or app lifecycle conditions improve.
  3. QR scanUsers need to decode a QR or barcode to pair a device, open a known destination, redeem a ticket, start payment, enroll an authenticator, join Wi-Fi, identify inventory, or import a short payload.

Build the foundation

Preserve the task across screens and input modes →

Follow the evidence

Complete study route · 15 entries

Work through the family in order

Begin with the core behavior, apply it in different contexts, then review cases that reveal the quality boundary. Open each entry for its decision rule, states, example, and source trail.

0 of 15 studies complete

Start this family →
6 entries

Build the foundation

Learn the shared task and the core interaction choices.

Lesson 01 / 15

Responsive navigation adaptation

A product supports multiple viewport widths, device classes, orientations, split-screen sizes, or foldable postures.

Type
UI-UX
Sources
6
Trail checked
standardNavigation model that changes presentation across viewport, posture, and destination-count constraints
Test in lab
Lesson 02 / 15

Focus traversal

A workflow has multiple focusable elements, dynamic content, or overlays that users traverse with keyboard or assistive technology.

Type
UI-UX
Sources
4
Trail checked
standardSequential keyboard focus path through interactive controls and state changes
Test in lab
Lesson 03 / 15

Touch gesture

A touchscreen interaction needs deliberate design for gesture vocabulary, thresholds, feedback, target sizing, cancellation, and equivalent controls.

Type
UI-UX
Sources
4
Trail checked
standardTouch-first gesture vocabulary and fallback contract for taps, swipes, drags, pinches, and multi-touch interactions
Test in lab
Lesson 04 / 15

Keyboard shortcut

Users repeat a known command often enough that memorized keyboard acceleration saves meaningful time.

Type
UI-UX
Sources
5
Trail checked
standardDirect key combination that activates a known command, focus target, or mode
Test in lab
Lesson 05 / 15

Offline mobile retry

A mobile user action can be saved locally and retried after network or app lifecycle conditions improve.

Type
UI-UX
Sources
6
Trail checked
standardMobile queued-action retry and background replay recovery
Test in lab
Lesson 06 / 15

QR scan

Users need to decode a QR or barcode to pair a device, open a known destination, redeem a ticket, start payment, enroll an authenticator, join Wi-Fi, identify inventory, or import a short payload.

Type
UI-UX
Sources
8
Trail checked
standardCamera or image-based QR detection, decode, validation, and action preview flow
Test in lab
9 entries

Apply the patterns

Choose a specialized form for the context and follow its complete behavior.

Lesson 07 / 15

Swipe action

Users repeatedly perform simple item-level actions on touch-first rows.

Type
UI-UX
Sources
6
Trail checked
standardHorizontal item-level gesture that reveals or commits row-scoped actions
Test in lab
Lesson 08 / 15

Long press

A touch-first interface needs an optional shortcut to secondary item actions, selection mode, preview, or drag pickup.

Type
UI-UX
Sources
5
Trail checked
standardSustained pointer contact that reveals secondary actions, selection, preview, or drag pickup
Test in lab
Lesson 09 / 15

Drag and drop

Users need spatial or ordered movement between clear sources and destinations.

Type
UI-UX
Sources
5
Trail checked
standardPointer or touch movement that picks up an object and drops it onto a valid destination
Test in lab
Lesson 10 / 15

Pull to refresh

A top-of-list gesture is a platform-expected shortcut for checking freshness on a scrollable data surface.

Type
UI-UX
Sources
5
Trail checked
standardTop-of-scroll vertical drag gesture that refreshes the current data set
Test in lab
Lesson 11 / 15

Camera capture

Users need to create new photo or video media inside the task.

Type
UI-UX
Sources
6
Trail checked
standardPermission-gated live camera preview for creating and reviewing new photo or video media
Test in lab
Lesson 12 / 15

Location permission flow

Current device coordinates materially improve a task such as nearby search, route start, check-in, delivery estimate, safety sharing, field work, or support diagnostics.

Type
UI-UX
Sources
8
Trail checked
standardPermission-gated current-location request and recovery flow
Test in lab
Lesson 13 / 15

Voice command

Users need hands-free control, accessibility speech input, or rapid spoken command activation.

Type
UI-UX
Sources
7
Trail checked
standardSpoken command interface that listens, recognizes, confirms, and executes bounded user intents
Test in lab
Lesson 14 / 15

Haptic feedback

A tactile pulse reinforces a direct manipulation, boundary, threshold, selection, successful action, recoverable warning, or physical game/control event.

Type
UI-UX
Sources
6
Trail checked
standardTactile feedback vocabulary for physical interaction events
Test in lab
Lesson 15 / 15

Wearable glance

A wearable surface can answer a status, next-step, or current-metric question in seconds.

Type
UI-UX
Sources
7
Trail checked
standardFew-second wearable status surface with safe handoff
Test in lab

Saved items