Skip to content

Development

Getting started

git clone https://github.com/mnvsk97/eyeroll.git
cd eyeroll
pip install -e '.[dev,all]'

Or with uv:

git clone https://github.com/mnvsk97/eyeroll.git
cd eyeroll
uv sync

Running tests

# Unit tests (mocked -- no API calls)
pytest

# With coverage
pytest --cov --cov-report=term-missing

# Integration tests (real API calls -- requires credentials)
pytest tests/test_integration.py -v -m integration

Integration tests

Integration tests hit real APIs and download real videos. They require valid API keys and are skipped by default in CI. Run them manually before releases.

The test suite has 269 unit tests and 8 integration tests.

Test patterns

  • All API calls, Ollama calls, ffmpeg, and yt-dlp are mocked in unit tests
  • Synthetic test videos are generated via ffmpeg fixtures in conftest.py
  • acquire.py is tested with both URL and local file paths
  • Backend tests verify the abstract interface and each implementation

Project structure

eyeroll/                          # Python CLI package
  __init__.py
  acquire.py                      # Download from URLs, resolve local files
  extract.py                      # ffmpeg: key frame extraction, audio extraction
  analyze.py                      # Backend-agnostic analysis, synthesis prompts
  backend.py                      # Backend ABC + Gemini, OpenAI, Ollama implementations
  watch.py                        # Pipeline orchestrator, caching
  history.py                      # Cache listing, retrieval, clearing
  cli.py                          # Click CLI (init, watch, history)

plugins/eyeroll/                  # Claude Code plugin
  .claude-plugin/plugin.json      # Plugin manifest
  commands/                       # Slash commands
    init.md                       # /eyeroll:init
    watch.md                      # /eyeroll:watch
    fix.md                        # /eyeroll:fix
    history.md                    # /eyeroll:history
  skills/                         # Background skills
    video-to-skill/               # Activated by "create a skill from this video"

tests/
  conftest.py                     # Shared fixtures (synthetic video generation)
  test_acquire.py
  test_analyze.py
  test_backend.py
  test_cli.py
  test_extract.py
  test_history.py
  test_watch.py
  test_integration.py             # Real API tests (marked, skipped by default)

CI/CD

A single GitHub Actions workflow (.github/workflows/ci.yml) handles both testing and publishing:

Testing: Runs on every push to main and feat/** branches, and on all pull requests. Tests against Python 3.11, 3.12, and 3.13.

Publishing: On push to main, builds the package and publishes to PyPI if the version does not already exist. Uses trusted publishing (OIDC) -- no API tokens stored.

Key design decisions

Backend abstraction: The Backend ABC defines analyze_image, analyze_video, analyze_audio, and generate. Each backend declares its capabilities via supports_video and supports_audio properties. The orchestrator uses these to choose the right strategy.

Cache intermediates, not reports: The expensive operations are frame analysis and audio transcription. Synthesis is cheap. Caching intermediates lets users re-run with different context without re-analyzing.

Content-adaptive reports: The synthesis prompt classifies the video content (bug, tutorial, demo, etc.) and adapts the analysis sections accordingly, rather than assuming everything is a bug.

Confidence tiers: For bug reports, every claim is categorized as directly observed, informed by codebase context, or hypothesis. This prevents coding agents from hallucinating file paths.

No OpenCV: Frame deduplication uses JPEG file size comparison instead of perceptual hashing. This avoids a heavy dependency while being good enough for screen recordings where static frames compress similarly.

Build

eyeroll uses hatchling as the build backend:

pip install build
python -m build

Release checklist

  1. Bump version in pyproject.toml and plugins/eyeroll/.claude-plugin/plugin.json
  2. Run full test suite: pytest
  3. Run integration tests: pytest tests/test_integration.py -v -m integration
  4. Push to main -- CI publishes to PyPI automatically