HomeAssistantVS/.session_nest_snapshot_20260729.md

Nest Snapshot Session — July 29, 2026

Goal

Capture still images from Nest cameras (Google Nest SDM API) for AI (Ollama) analysis.

What Was Built

Custom integration nest_snapshot at /homeassistant/custom_components/nest_snapshot/:

• manifest.json, services.yaml, __init__.py
• Uses WebRTC (aiortc) to grab one video frame → saves as JPEG
• Test automation automation.test_nest_snapshot_capture

Bug Fixes Applied

Fix 1: Nest integration data not available

• Root cause: HA 2026.7 stores Nest runtime data in config_entry.runtime_data (NestData dataclass), not in hass.data["nest"]. Camera entity unique_id has -camera suffix, but device_manager.devices is keyed by raw Google device name.
• Changes: Line 69: config_entry.runtime_data instead of hass.data.get(...). Lines 78-81: -camera suffix stripping before device_manager lookup.
• Status: ✅ Fixed

Fix 2: aiortc module not installed

• Root cause: aiortc and Pillow not in manifest.json requirements.
• Changes: Added "requirements": ["aiortc", "Pillow"] to manifest.json.
• Status: ✅ Fixed

Fix 3: SDP format — "Offer must contain audio, video and application m lines"

• Root cause: Only added video transceiver; Google needs audio + video + application in order.
• Changes: Added pc.addTransceiver("audio", ...), pc.addTransceiver("video", ...), pc.createDataChannel(...).
• Status: ✅ Fixed

Fix 4 (current): invalid literal for int() with base 10: 'udp'

• Root cause: aiortc ICE candidate parser expects candidate:ID <int:component> udp ... but Google Nest API sends candidate:ID udp ... (missing component number).
• Attempt 1: Sanitize via _sanitize_sdp() checking parts[1].lower() == "udp" — didn't match actual format.
• Attempt 2: Changed to not parts[1].isdigit() — still failing. Possibly the line format is a=candidate:udp ... where parts[0].split(':')[1] is udp and parts[1] IS a digit.
• Current state: Added debug logging of raw SDP from Google. Need restart to see logs.
• Status: 🔴 In progress — add logging to see actual SDP format

File States

• __init__.py — has _sanitize_sdp() with not parts[1].isdigit() check + debug logs
• manifest.json — has ["aiortc", "Pillow"] requirements
• services.yaml — unchanged, defines capture service
• AGENTS.local.md — restart rule added (must ask each time)

Pending Changes (committed but not tested after latest edit)

• _sanitize_sdp() now uses not parts[1].isdigit() (broader match)
• Debug logging added for raw + sanitized SDP
• Need restart → test → check system log for SDP content