Device Resilience Event System Documentation 🔗

Overview 🔗

This document details all events, triggers, and recovery mechanisms implemented in the Cantina WebRTC application for device resilience and video recovery. The system handles device disconnections, reconnections, focus changes, and various edge cases to provide a seamless user experience.

Event Categories 🔗

1. Device Change Detection Events 🔗

Location: deviceMonitorSaga.ts

Polling Mechanism 🔗

• Interval: Every 3 seconds
• Method: setInterval(checkDevices, 3000)
• Purpose: Fallback for browsers without native devicechange event support

Native Device Change Event 🔗

• Event: navigator.mediaDevices.addEventListener('devicechange', checkDevices)

• Triggers:

• Device physical connection/disconnection

• Device ID changes (common with Bluetooth devices like AirPods)
• Device label changes
• OS default audio device changes
• USB device plug/unplug
• Bluetooth pairing/unpairing

Detection Logic 🔗

const devicesChanged = devices.length !== previousDevices.length ||
  devices.some((device, index) => {
    const prevDevice = previousDevices[index]
    return !prevDevice || 
      device.deviceId !== prevDevice.deviceId ||
      device.label !== prevDevice.label
  })

2. Visibility and Focus Events 🔗

Location: visibilitySaga.ts

Document Visibility 🔗

• Event: document.addEventListener('visibilitychange', handleVisibilityChange)

• States:

• document.hidden === true: Tab/window is hidden

• document.hidden === false: Tab/window is visible

• Triggers:

• User switches browser tabs

• User minimizes browser window
• Mobile: User switches apps
• Mobile: Screen lock/unlock

Window Focus 🔗

• Event: window.addEventListener('focus', handleFocus)

• Triggers:

• User clicks on browser window

• User returns to browser tab
• Mobile: App comes to foreground

Window Blur 🔗

• Event: window.addEventListener('blur', handleBlur)

• Triggers:

• User clicks outside browser window

• User switches to another application
• Mobile: App goes to background

3. User-Initiated Events 🔗

Location: callSaga.ts

Video Unmute 🔗

• Action: restoreLocalVideoTrack
• Trigger: User clicks video unmute button

• Pre-flight checks:

• Validate camera device still exists

• Check camera preference for recovery
• Update device if changed

Audio Unmute 🔗

• Action: restoreLocalAudioTrack
• Trigger: User clicks audio unmute button

• Pre-flight checks:

• Validate microphone device still exists

• Check microphone preference for recovery
• Update device if changed
• Note: Also checks video since audio issues can affect video streams

4. Device Selection Events 🔗

Location: deviceSlice.ts

Camera Selection 🔗

• Action: cameraChanged
• Payload: { deviceId: string, label: string }

• Side effects:

• Creates/updates camera preference

• Tracks if user selected "Default" vs specific device
• Stores actual device ID when "Default" selected

Microphone Selection 🔗

• Action: microphoneChanged
• Payload: { deviceId: string, label: string }

• Side effects:

• Creates/updates microphone preference

• Tracks if user selected "Default" vs specific device
• Stores actual device ID when "Default" selected

Speaker Selection 🔗

• Action: speakerChanged
• Payload: { deviceId: string, label: string }

• Side effects:

• Creates/updates speaker preference

• Tracks if user selected "Default" vs specific device
• Stores actual device ID when "Default" selected

Recovery Actions by Event Type 🔗

Device Change Recovery Flow 🔗

1. Initial Delay
2. Duration: 2000ms

3. Purpose: Allow devices to stabilize after wake-from-sleep or reconnection

4. Device Validation

5. Check each device type (camera, microphone, speaker)

6. Compare current device ID against available devices

7. Recovery Attempt (if device missing)

8. Priority 1: Exact ID match (device still exists)
9. Priority 2: Label match (same device, new ID - AirPods case)

10. Priority 3: OS default fallback

11. State Updates

12. Update Redux device state
13. Update device preference if needed

14. Apply to active WebRTC call

15. User Notification

16. Toast for successful recovery: "Camera switched to: [device name]"

17. Toast for fallback: "Camera disconnected. Using default camera."

18. Video Recovery

19. Check and restore remote video
20. Check and restore local video overlay

Visibility/Focus Change Recovery Flow 🔗

On Focus Loss (Mobile Only) 🔗

1. Check Video State

2. Only proceed if video is currently unmuted and sending

3. Auto-mute Video

4. Send DTMF '*0' to notify server
5. Stop local video track
6. Set wasAutoMuted = true
7. Remember previousVideoState

On Focus Gain 🔗

1. Initial Delay
2. Duration: 200ms

3. Purpose: Let browser stabilize after focus change

4. Video Restoration (Mobile Only)

5. If wasAutoMuted && previousVideoState:

   • Send DTMF '*0' to notify server
   • Restore local video track
   • Clear auto-mute flag

6. Device Re-enumeration

7. Force enumerate cameras
8. Force enumerate microphones

9. Force enumerate speakers

10. Video Stream Recovery

11. Step 1: Try to play() paused video
12. Step 2: Request keyframe if still not playing (wait 500ms)

13. Step 3: Trigger re-invite if still failing (wait 1500ms)

14. Local Video Recovery

15. Find all .local-video elements
16. Reconnect local stream to elements
17. Call play() on each element

Unmute Recovery Flow 🔗

1. Device Validation javascript const validatedDevice = yield call( checkMicrophone, devices.microphoneId, devices.microphoneLabel, microphonePreference )

2. Device Update (if changed)

3. Update Redux state with new device
4. Update WebRTC constraints

5. Apply to active call

6. Video Recovery

7. Fork checkRemoteVideo (non-blocking)

8. Fork checkLocalVideo (non-blocking)

9. Complete Unmute

10. Execute actual unmute with validated device

Timing Configuration 🔗

Delays and Intervals 🔗

Event Priority 🔗

1. Immediate Response (0ms delay):

2. User device selection

3. Unmute actions validation

4. Quick Response (200ms delay):

5. Focus gain recovery

6. Stabilization Required (2000ms delay):

7. Device change after wake

8. Device reconnection

Mobile-Specific Behaviors 🔗

Auto-mute on Background 🔗

• Trigger: Window blur or visibility hidden
• Condition: Only on mobile devices (isMobile === true)
• Action: Auto-mute video if currently unmuted
• Server Notification: DTMF '*0' sent

Auto-unmute on Foreground 🔗

• Trigger: Window focus or visibility visible
• Condition: Only if we auto-muted previously
• Action: Restore video to previous state
• Server Notification: DTMF '*0' sent

DTMF Commands 🔗

Error Handling 🔗

Device Enumeration Failures 🔗

try {
  const devices = await navigator.mediaDevices.enumerateDevices()
} catch (error) {
  console.error('Failed to enumerate devices:', error)
}

Video Play Failures 🔗

videoEl.play().catch((error) => {
  console.error('Failed to play video:', error)
})

Device Recovery Failures 🔗

• Graceful fallback to OS default
• User notification of fallback
• No interruption to active call

Testing Scenarios 🔗

Device Disconnection 🔗

1. Connect Bluetooth headphones
2. Start call with Bluetooth selected
3. Turn off Bluetooth device
4. Verify: Toast notification, fallback to default, call continues

Sleep/Wake Recovery 🔗

1. Start video call on mobile
2. Lock screen for 30+ seconds
3. Unlock and return to app
4. Verify: 2-second delay, devices recover, video resumes

Tab Switching (Mobile) 🔗

1. Start video call with video unmuted
2. Switch to another app
3. Return to browser
4. Verify: Video auto-muted on leave, auto-unmuted on return

Focus Change Recovery 🔗

1. Start video call
2. Click outside browser window
3. Click back on browser
4. Verify: Video streams recover, no black screens

Implementation Benefits 🔗

1. Transparent Recovery: Users rarely need manual intervention
2. Consistent State: Device preferences persist across disconnections
3. Mobile Optimization: Auto-mute prevents battery drain and bandwidth waste
4. Server Awareness: DTMF commands keep server in sync with client state
5. Graceful Degradation: Always falls back to working device
6. User Feedback: Toast notifications for transparency

Future Enhancements 🔗

1. Persistent Storage: Save preferences to localStorage
2. Device Priority Lists: Multiple fallback devices
3. Custom Recovery Delays: User-configurable timing
4. Recovery Analytics: Track recovery success rates
5. Permission Recovery: Handle permission revocation/restoration