Web Audio API Skill 1. Overview This skill provides Web Audio API expertise for creating audio feedback, voice processing, and sound effects in the JARVIS AI Assistant. Risk Level : LOW Audio processing with minimal security surface Primary Use Cases : HUD audio feedback (beeps, alerts) Voice input processing Spatial audio for 3D HUD elements Real time audio visualization Text to speech integration 2. Core Responsibilities 2.1 Fundamental Principles 1. TDD First : Write tests before implementation for all audio components 2. Performance Aware : Optimize for 60fps with minimal audio latency 3. User Gesture Required : Audio context must be started after user interaction 4. Resource Cleanup : Close audio contexts and disconnect nodes on unmount 5. AudioWorklet for Processing : Use AudioWorklet for heavy DSP operations 6. Accessibility : Provide visual alternatives to audio feedback 7. Volume Control : Respect system and user volume preferences 8. Error Handling : Gracefully handle audio permission denials 3. Technology Stack & Versions 3.1 Browser Support Browser AudioContext AudioWorklet Chrome 35+ 66+ Firefox 25+ 76+ Safari 14.1+ 14.1+ 3.2 TypeScript Types 4. Implementation Patterns 4.1 Audio Context Management 4.2 HUD Beep Feedback 4.3 Audio Visualization 4.4 Spatial Audio for 3D HUD 4.5 Microphone Input 5. Implementation Workflow (TDD) Step 1: Write Failing Test First Step 2: Implement Minimum to Pass Step 3: Refactor Following Patterns After tests pass, refactor to: Extract shared audio context logic Add proper TypeScript types Implement cleanup on unmount Step 4: Run Full Verification 6. Performance Patterns 6.1 AudioWorklet for Processing 6.2 Buffer Pooling 6.3 Offline Rendering 6.4 Node Graph Optimization 6.5 Memory Management 7. Quality Standards 8. Testing & Quality 9. Common Mistakes & Anti Patterns 9.1 Critical Anti Patterns 10. Pre Implementation Checklist Phase 1: Before Writing Code [ ] Tests written for audio node creation and connections [ ] Tests written for volume clamping and validation [ ] Performance requirements identified (latency, concurrent sounds) [ ] AudioWorklet needed for DSP? Worklet file created [ ] Buffer pool size calculated for expected usage Phase 2: During Implementation [ ] User gesture required for AudioContext initialization [ ] Audio context reused (not created per sound) [ ] Nodes disconnected in onended callbacks [ ] Volume bounds validated (0 1 range) [ ] Microphone permissions handled gracefully [ ] Error states provide visual feedback Phase 3: Before Committing [ ] All audio tests pass: npm test grep "audio" [ ] Type checking passes: npm run typecheck [ ] No memory leaks (tested in DevTools Memory tab) [ ] Audio context closed on component unmount [ ] Visual alternatives provided for accessibility [ ] Sound can be disabled via user preferences [ ] Volume respects system preferences 11. Summary Web Audio API for JARVIS: Initialize after user gesture, cleanup on unmount, handle permission denials, provide visual alternatives. See references/advanced patterns.md

web-audio-api