Guaardvark v2.6 introduces lesson pearls — explicit training sessions where every thumbs-up you give becomes a structured, editable memory the agent carries forward. Bracket a workflow with Begin Lesson / End Lesson, watch pearls accumulate in real time, and finish with an LLM-distilled step summary you can edit before it lands in the agent’s context. The new Comments Trainer provides a scripted practice ground so the agent can rehearse click-and-type workflows before touching real applications.
Think of how a pearl necklace is made — one pearl at a time, threaded onto the strand in order, until the whole becomes a finished piece. A lesson works the same way. Each pearl is one step the agent learned: a single thumbs-up captured the moment a behaviour worked. Begin Lesson opens the strand, every pearl gets threaded on as you confirm the right move, and End Lesson knots it shut into a finished necklace — a structured workflow the agent can replay end-to-end, exactly as you taught it.
Every time you give a positive rating during a lesson, the agent records a “pearl” — a timestamped snapshot of what it just did and why it worked. Pearls accumulate in real time in the floating Lesson Pearls panel, giving you live confirmation that the agent is internalising the right behaviours.
Begin Lesson / End Lesson buttons create a named training block. All thumbs-ups inside the block are tagged with the same lesson_id column on the ToolFeedback table. End Lesson triggers a single distillation pass over the full pearl sequence, producing a structured JSON summary with ordered steps.
The lesson summary modal lets you reorder steps, fix wording, or delete steps before saving. Summaries are stored as AgentMemory rows with source=“lesson_summary” and can be re-edited retroactively from Settings → Memory. At read-time they are flattened into plain text so Gemma4 receives structured workflow instructions, not raw JSON.
The Comments Trainer at data/agent/files/comments_trainer.html is a scripted mock video platform. The agent reads task banners via vision, clicks targets, and types text — no keyboard shortcuts allowed. Successful task completions post positive feedback to the same API that drives pearl distillation, so trainer results flow seamlessly into AgentMemory.
The original Guaardvark self-improvement loop fired a distillation call on every thumbs-up, mutating a single rolling AgentMemory row. It worked, but it produced low-value first-person self-reflection with no signal about lesson boundaries — the agent couldn’t tell when it had genuinely learned a new workflow versus when it had simply reflected on noise. The v2.6 lesson pearls system replaces that with explicit training sessions. You press Begin Lesson when you want the agent to start absorbing a workflow, give thumbs-ups on the replies that demonstrate correct behaviour, and press End Lesson when the session is complete. The accumulated pearl sequence is then distilled by the LLM into a structured summary with a title and numbered steps, which the agent can follow verbatim the next time it needs to perform that workflow.
The architecture is deliberately minimal. A single new nullable column — lesson_id — on the ToolFeedback table is the only schema change. Lesson state is tracked in an in-memory dict (ACTIVE_LESSONS) in backend/api/lessons_api.py, a new blueprint auto-discovered at startup. The AgentMemory rows created by End Lesson are stored with source=“lesson_summary”, and the memory context formatter in backend/api/memory_api.py treats them specially: instead of truncating at 300 characters, it parses the JSON and flattens the steps into plain text like LESSON (Reply to comment): 1. Click Comment input → 2. Type “Great video” → 3. Click Comment button. This is the critical fix that makes lessons actually influence the model — raw JSON would be silently truncated before reaching Gemma4’s context window.
When no lesson is active, the original per-thumbs-up distillation path still runs unchanged — the gate is simply if not entry.get(“lesson_id”) around the background thread spawn. This means the existing self-improvement behaviour is fully preserved as a fallback for informal feedback outside structured training sessions. Both paths feed the same AgentMemory store and the same memory retrieval pipeline, so the agent benefits from both structured lesson knowledge and ad-hoc positive signals gathered during ordinary use.
Lessons API and storage. The backend/api/lessons_api.py blueprint exposes four endpoints: POST /api/lessons/start mints a UUID lesson ID and stores it in ACTIVE_LESSONS[session_id]; POST /api/lessons/<lesson_id>/end calls _distill_lesson_pearls, pops the active lesson, and returns the structured summary; GET /api/lessons/<lesson_id> returns pearls plus active flag for floater hydration after a browser refresh; and GET /api/lessons/active?session_id=... supports recovery after a backend restart. The distillation function queries all ToolFeedback rows tagged with the lesson ID, ordered by created_at ascending, then prompts Ollama for strict JSON {title, steps:[{order, text}]}. If JSON parsing fails, it synthesises steps from the raw pearl task descriptions. A new PATCH /api/memory/<memory_id> endpoint allows retroactive editing of any AgentMemory row from the Settings panel.
Comments Trainer. The self-contained HTML file at data/agent/files/comments_trainer.html renders a 1280×720 mock video platform — matching the resolution of the agent’s isolated display — and rotates the agent through ten scripted tasks covering every interaction surface: clicking video thumbnails, focusing comment inputs, typing text, clicking submit buttons, hitting Reply and Like buttons, navigating back, clicking the search bar, typing a query, and clicking the Search button (Enter-to-submit is intentionally absent, forcing the agent to click the visible button). Each task is displayed in a high-contrast banner the agent reads via its normal vision loop. On correct completion the trainer fires a POST /api/agent-control/feedback with positive: true and the task description — the same endpoint that drives pearl distillation — so trainer successes flow into AgentMemory exactly like chat thumbs-ups. If a lesson is active when the trainer is opened (via ?session_id=<uuid> in the URL), the pearls are tagged to that lesson and distilled as a structured workflow at End Lesson.
# Begin a lesson in the UI: click "Begin Lesson" in the chat header
# Thumbs-up agent replies that demonstrate correct behaviour
# Each thumbs-up emits a lesson:pearl_added socket event
# The floating panel shows: "🟢 Recording — 3 pearls"
# End Lesson calls the distillation endpoint:
POST /api/lessons/<lesson_id>/end
# Response: { success, memory_id, summary: { title, steps: [{order, text}] } }
# Resulting AgentMemory content (flattened for Gemma4):
LESSON (Reply to comment): 1. Click Comment input → 2. Type "Great video" → 3. Click Comment button
# Open Comments Trainer with active session:
# data/agent/files/comments_trainer.html?session_id=<uuid>Open a video platform, press Begin Lesson, then walk the agent through the full comment cycle: clicking the comment field, typing text, clicking the Comment button, clicking Reply on an existing comment. Thumbs-up each correct action. Press End Lesson; the modal presents an LLM-generated step summary. Edit the wording if needed, save. The agent now carries a structured “how to comment on a video” memory it can execute reliably in future sessions.
Before directing the agent at a real platform, open comments_trainer.html in the agent’s browser. The agent works through ten click-and-type tasks on a mock video UI where mistakes carry no real-world consequences. Each successful task fires a positive feedback event that becomes a pearl if a lesson is active. Once the trainer score stabilises above 90% accuracy, the agent has demonstrated readiness for the real application.
Run a series of lessons to teach the agent distinct workflows: “how to post a YouTube comment,” “how to like and share a video,” “how to search for a specific channel.” Each lesson produces a named, numbered summary in AgentMemory. Over time the agent accumulates a library of reliable procedures it can retrieve and execute on demand — without you re-explaining the steps every session.
Most approaches to teaching an AI agent a new behaviour require either retraining (expensive, requires labelled data, restarts the model), prompt engineering (brittle, must be re-described every session), or commercial agent platforms that charge per distillation call and store your training data on their servers. Guaardvark’s lesson pearls run entirely on your machine: the distillation call goes to your local Ollama endpoint, the structured summary is stored in your local SQLite database, and the memory retrieval happens inside your own backend. There are no per-lesson fees, no data sharing, and no lock-in to a proprietary format — lesson summaries are plain JSON you can export, edit, or delete from Settings → Memory at any time. The editable-steps modal means you can fix a distillation that missed the point without triggering a new training run.
The ACTIVE_LESSONS dict is in-memory and does not survive a backend restart. However, the individual pearl rows in the ToolFeedback table are written to the database as each thumbs-up is recorded, so no pearl data is lost. After restarting, the floater will show the session as inactive. You can close and reopen a lesson from the UI; if you click End Lesson from a stale frontend, you will receive a clear “lesson not found” toast. Recovery is a known limitation of the single-user local design and is documented in the release notes.
Yes. Navigate to Settings → Memory. Any memory row with source=“lesson_summary” shows an Edit button next to the delete affordance. Clicking it opens the same LessonSummaryModal used at End Lesson, pre-populated with the current title and steps. You can reorder steps with the up/down arrows, edit step text directly, add new steps, or remove steps that do not reflect correct behaviour. Saving PATCHes the AgentMemory row via PATCH /api/memory/<memory_id>.
Yes. When no lesson is active, every thumbs-up triggers the original _distill_pearl_memory background thread exactly as before. The gate is a single if not entry.get(“lesson_id”) check around the thread spawn. If you never use Begin/End Lesson, the platform behaves identically to v2.5. The two paths are additive, not exclusive.
The Comments Trainer (data/agent/files/comments_trainer.html) is a self-contained 1280×720 mock video platform with a task banner the agent reads via its vision loop. Tasks cover clicking thumbnails, focusing inputs, typing text, and clicking buttons — every action is a visible click or keystroke, with no keyboard shortcuts. The trainer scores hits and misses, fires positive feedback events to /api/agent-control/feedback on correct task completions, and optionally tags them to an active lesson via ?session_id=<uuid>. It is opened in the agent’s Firefox instance the same way as vision_trainer.html.
At memory retrieval time, get_memories_for_context() in backend/api/memory_api.py checks each memory row’s source field. For source=“lesson_summary” rows, it parses the JSON content, sorts steps by order, and flattens them into a single line like LESSON (Reply to comment): 1. Click Comment input → 2. Type “Great video” → 3. Click Comment button. This plain-text form gets a 1,000-character budget instead of the default 300-character truncation applied to other memory types. The Gemma4 direct path in agent_brain.py also requests max_tokens=800 of memory context (bumped from 300) so multiple full lessons can coexist without starving each other.