Device Resilience Implementation Guide 🔗

Overview 🔗

This document describes the comprehensive device resilience system implemented for the Cantina WebRTC application. The system handles device disconnections/reconnections (like AirPods), OS default device changes, and device persistence across sessions.

Problem Statement 🔗

WebRTC applications face several device-related challenges:

1. Device ID Changes: Devices like AirPods get new IDs when disconnecting/reconnecting
2. Device Loss on Sleep/Wake: Mobile devices lose camera/microphone access when waking from sleep
3. OS Default Switching: Users expect the app to follow OS audio device changes
4. No Built-in Persistence: WebRTC doesn't remember user device preferences

Solution Architecture 🔗

1. Device Preference Tracking 🔗

File: react/src/common/redux/interfaces.ts

export interface DevicePreference {
  isDefault: boolean        // User selected OS default
  preferredLabel?: string   // Device name for recovery
  lastSeenId?: string      // Last known device ID
}

Purpose: Store user intent separately from current device state. This allows recovery based on what the user wanted, not just what's currently selected.

2. Enhanced Redux State 🔗

File: react/src/common/redux/features/device/deviceSlice.ts

Key additions:

• cameraPreference, microphonePreference, speakerPreference - Store user preferences
• defaultCameraActualId, defaultMicrophoneActualId, defaultSpeakerActualId - Track actual device when "Default" is selected

Behavior:

• Preferences are automatically created when user selects a device
• Actual default IDs are updated when OS default changes

3. Device Preference Manager 🔗

File: react/src/common/services/devicePreferenceManager.ts

Core service that handles device recovery logic:

async recoverDevice(
  deviceType: 'camera' | 'microphone' | 'speaker',
  currentId: string,
  preference?: DevicePreference
): Promise<DeviceRecoveryResult>

Recovery Priority:

1. If user selected "Default" → Use OS default
2. Try exact ID match → Device still connected
3. Try label match → Same device, new ID (AirPods case)
4. Fallback to OS default → Device truly gone

Key Features:

• Label-based recovery for devices with changing IDs
• Validates devices before applying
• Returns recovery status for user notification

4. Real-time Device Monitoring 🔗

File: react/src/common/redux/features/device/deviceMonitorSaga.ts

Saga that continuously monitors for device changes:

function* handleDeviceChange(devices: MediaDeviceInfo[]): SagaIterator {
  // 2-second delay for device stabilization (phone wake)
  yield delay(2000)

  // Validate each device type
  // Recover if needed
  // Update Redux state
  // Notify user of changes
}

Features:

• Polls every 3 seconds (fallback for browsers without devicechange event)
• Detects device additions/removals
• Automatically recovers preferred devices
• Shows toast notifications for device changes
• 2-second delay prevents race conditions on wake

5. Device Validation on Use 🔗

File: react/src/common/redux/features/calls/callSaga.ts

Enhanced unmute actions to revalidate devices:

case restoreLocalAudioTrack.type: {
  // Revalidate microphone before unmuting
  const validatedDevice = yield call(
    checkMicrophone, 
    devices.microphoneId, 
    devices.microphoneLabel,
    microphonePreference
  )

  // Update if device changed
  if (validatedDevice && validatedDevice.deviceId !== devices.microphoneId) {
    yield put(deviceActions.microphoneChanged({
      deviceId: validatedDevice.deviceId,
      label: validatedDevice.label
    }))
    // Update constraints with new device
  }
}

Purpose: Ensures device is still valid when user unmutes, preventing errors.

6. Enhanced Device Checking 🔗

File: react/src/common/services/checkDevices.ts

Added preference support to device validation:

export const checkMicrophone = async (
  microphoneId: string,
  microphoneLabel?: string,
  preference?: DevicePreference
): Promise<DeviceInfo | null>

Behavior:

• First tries the provided device ID
• On failure, attempts recovery using preference
• Returns validated device or null

Implementation Flow 🔗

Device Selection Flow: 🔗

1. User selects device → Redux action dispatched
2. Device slice updates current device + creates preference
3. If "Default" selected → Store actual device ID for monitoring
4. Apply device to active call

Device Recovery Flow: 🔗

1. Device change detected (polling/event)
2. 2-second delay for stabilization

3. For each device type:

4. Get current selection and preference

5. Validate current device
6. If invalid, attempt recovery
7. Update Redux state
8. Update active call
9. Show user notification

Unmute Recovery Flow: 🔗

1. User clicks unmute
2. Validate selected device with preference
3. If device changed, update selection
4. Apply recovered device
5. Then unmute with working device

Key Implementation Details 🔗

1. Label-based Recovery 🔗

const deviceByLabel = devices.find(d => 
  d.label === preference.preferredLabel && 
  d.deviceId !== MediaDevice.Default &&
  d.deviceId !== MediaDevice.None
)

This finds devices by name when IDs change.

2. OS Default Tracking 🔗

if (cameraId === MediaDevice.Default) {
  const actualDevice = cameras.find(d => 
    d.deviceId !== MediaDevice.Default && 
    d.deviceId !== MediaDevice.None
  )
  // Track actual device for change detection
}

3. Race Condition Prevention 🔗

yield delay(2000) // Wait for devices to stabilize

Prevents errors when devices are still initializing.

4. User Notifications 🔗

if (result.recovered) {
  yield call(toastInfo, `Camera switched to: ${result.deviceLabel}`)
} else if (result.fallbackUsed) {
  yield call(toastInfo, `Camera disconnected. Using default.`)
}

Keeps users informed of automatic changes.

Benefits 🔗

1. Seamless Experience: Devices reconnect automatically
2. User Intent Preserved: Remembers if user wanted specific device or OS default
3. Resilient to Changes: Handles ID changes, disconnections, sleep/wake
4. Transparent: Users see notifications for changes
5. Prevents Errors: Validates before use, preventing mid-call failures

Applying to Other Applications 🔗

To implement similar resilience:

1. Separate Intent from State: Store what user selected vs current state
2. Use Multiple Identifiers: Don't rely only on device IDs
3. Implement Recovery Hierarchy: Exact → Label → Fallback
4. Monitor Continuously: Don't trust devices to stay connected
5. Validate Before Use: Always check device validity before critical operations
6. Add Stabilization Delays: Prevent race conditions on device initialization
7. Inform Users: Show non-intrusive notifications for automatic actions

This approach works for any WebRTC application or system dealing with dynamic hardware devices.