46 lines
2.6 KiB
Markdown
46 lines
2.6 KiB
Markdown
# Nest Snapshot Session — July 29, 2026
|
|
|
|
## Goal
|
|
Capture still images from Nest cameras (Google Nest SDM API) for AI (Ollama) analysis.
|
|
|
|
## What Was Built
|
|
Custom integration `nest_snapshot` at `/homeassistant/custom_components/nest_snapshot/`:
|
|
- `manifest.json`, `services.yaml`, `__init__.py`
|
|
- Uses WebRTC (`aiortc`) to grab one video frame → saves as JPEG
|
|
- Test automation `automation.test_nest_snapshot_capture`
|
|
|
|
## Bug Fixes Applied
|
|
|
|
### Fix 1: `Nest integration data not available`
|
|
- **Root cause:** HA 2026.7 stores Nest runtime data in `config_entry.runtime_data` (NestData dataclass), not in `hass.data["nest"]`. Camera entity `unique_id` has `-camera` suffix, but `device_manager.devices` is keyed by raw Google device name.
|
|
- **Changes:** Line 69: `config_entry.runtime_data` instead of `hass.data.get(...)`. Lines 78-81: `-camera` suffix stripping before device_manager lookup.
|
|
- **Status:** ✅ Fixed
|
|
|
|
### Fix 2: `aiortc` module not installed
|
|
- **Root cause:** `aiortc` and `Pillow` not in manifest.json requirements.
|
|
- **Changes:** Added `"requirements": ["aiortc", "Pillow"]` to manifest.json.
|
|
- **Status:** ✅ Fixed
|
|
|
|
### Fix 3: SDP format — "Offer must contain audio, video and application m lines"
|
|
- **Root cause:** Only added video transceiver; Google needs audio + video + application in order.
|
|
- **Changes:** Added `pc.addTransceiver("audio", ...)`, `pc.addTransceiver("video", ...)`, `pc.createDataChannel(...)`.
|
|
- **Status:** ✅ Fixed
|
|
|
|
### Fix 4 (current): `invalid literal for int() with base 10: 'udp'`
|
|
- **Root cause:** `aiortc` ICE candidate parser expects `candidate:ID <int:component> udp ...` but Google Nest API sends `candidate:ID udp ...` (missing component number).
|
|
- **Attempt 1:** Sanitize via `_sanitize_sdp()` checking `parts[1].lower() == "udp"` — didn't match actual format.
|
|
- **Attempt 2:** Changed to `not parts[1].isdigit()` — still failing. Possibly the line format is `a=candidate:udp ...` where `parts[0].split(':')[1]` is `udp` and `parts[1]` IS a digit.
|
|
- **Current state:** Added debug logging of raw SDP from Google. Need restart to see logs.
|
|
- **Status:** 🔴 In progress — add logging to see actual SDP format
|
|
|
|
## File States
|
|
- `__init__.py` — has `_sanitize_sdp()` with `not parts[1].isdigit()` check + debug logs
|
|
- `manifest.json` — has `["aiortc", "Pillow"]` requirements
|
|
- `services.yaml` — unchanged, defines `capture` service
|
|
- `AGENTS.local.md` — restart rule added (must ask each time)
|
|
|
|
## Pending Changes (committed but not tested after latest edit)
|
|
- `_sanitize_sdp()` now uses `not parts[1].isdigit()` (broader match)
|
|
- Debug logging added for raw + sanitized SDP
|
|
- Need restart → test → check system log for SDP content
|