Drop-in plugins/<name>/plugin.json architecture. 10+ managed services with live VRAM bars, conflict detection (Ollama vs ComfyUI), per-plugin start/stop/config, and real-time model downloads. Currently: ComfyUI (Wan2.2 video), Audio Foundry, Swarm, LoRA Trainer, Upscaling, GPU Embedding, Training, Discord Bot, and more.
Every plugin declares its identity and capabilities in plugin.json: name, version, description, entry point, and a capabilities array that tells Guaardvark which hooks the plugin implements. The manifest format is minimal by design — the minimum viable plugin is a manifest file and one Python module. No registration process, no API keys, no package publication required.
backend/utils/plugin_loader.py scans the plugins/ directory at startup, reads each plugin.json manifest, and loads the plugin's entry point. Plugins are loaded in alphabetical order by directory name. A failed plugin load logs the error and skips the plugin without preventing the rest of the platform from starting — plugin failures are isolated from core functionality.
ComfyUI (video gen), Audio Foundry (ACE-Step music + TTS), Swarm (worktree orchestration), LoRA Trainer (bf16 character/env LoRAs), Upscaling (Real-ESRGAN), GPU Embedding (5–20× faster indexing), Vision Pipeline, Training data capture, Discord Bot. Each declares port, VRAM estimate, health in its manifest.
Real-time nvidia-smi VRAM bar (used/total + per-plugin segments). Automatic conflict detection and offers (exclusive access for Ollama vs. ComfyUI). Model download modals with MB/s progress for Wan2.2, SD checkpoints, voices, RIFE, ESRGAN. Pre-flight checks return 409 Conflict when GPU is busy.
Full UI: cards with status/logs, enable/disable (persisted in plugin_state.json), start/stop, inline config. Auto-discovery at startup via plugin_loader.py. Failures are isolated; the rest of the platform continues. Discord bot and outreach cogs load only when their plugin dir is present.
Guaardvark's plugin system follows a simple principle: adding a new capability should not require modifying the core application. Core code is stable, tested, and versioned. Plugin code is experimental, optional, and owned by the plugin author. The plugin.json manifest declares what a plugin provides; the plugin_loader.py auto-discovery system handles the rest. This separation means plugins can be developed, iterated, and deployed independently of the core release cycle.
The capabilities system in the manifest is the key to hook registration. A plugin that declares "capabilities": ["embedding_provider"] registers as a candidate embedding provider during startup. The core's embedding_router.py checks whether any loaded plugin has registered for the embedding_provider hook and routes embedding requests to the plugin if one is available. The gpu_embedding plugin works exactly this way: it declares the capability, registers its embedding function, and the rest of the platform uses it transparently without any manual wiring.
The Discord bot integration is managed through the plugin architecture for the same reason: the Discord bot is optional, has its own dependency tree (discord.py), and should not be loaded on instances where it's not configured. Placing it in the plugin framework means it loads only when its plugin directory is present and its plugin.json is valid. The outreach automation described in the Social Outreach feature also runs under this model: the social outreach cog is a Discord plugin capability that loads only when configured.
Plugin loading mechanics. backend/utils/plugin_loader.py is called from backend/app.py during the Flask application factory setup. It iterates plugins/, reads plugin.json for each directory, validates the manifest schema, and imports the entry point Python module. Loaded plugin instances are stored in a module-level registry dict keyed by plugin name. The plugins_api.py blueprint exposes GET /api/plugins (list all with status), POST /api/plugins/<name>/enable, POST /api/plugins/<name>/disable, and POST /api/plugins/<name>/reload. Reload triggers a re-import of the plugin's entry module, useful during development without a full server restart.
gpu_embedding plugin internals. The plugins/gpu_embedding/ directory contains plugin.json declaring embedding_provider capability and a Python module that wraps a HuggingFace sentence-transformers model loaded via PyTorch. On registration, the plugin calls a core hook to replace the default Ollama embedding function with its own batched CUDA implementation. The batch size is configurable in the plugin's settings (default 64, adjustable based on VRAM). Embedding calls are synchronous from the caller's perspective; the batching is handled internally. The plugin respects the PYTORCH_CUDA_ALLOC_CONF environment variable set by the core configuration for VRAM management.
# Plugin directory structure
plugins/
└── gpu_embedding/
├── plugin.json # Manifest
└── gpu_embedding.py # Entry point
# plugin.json format
{
"name": "gpu_embedding",
"version": "1.0.0",
"description": "GPU-accelerated text embeddings via PyTorch",
"entry": "gpu_embedding.py",
"capabilities": ["embedding_provider"],
"settings": {
"batch_size": 64,
"model": "sentence-transformers/all-MiniLM-L6-v2"
}
}On a machine with a CUDA GPU, install the gpu_embedding plugin and restart Guaardvark. Document indexing throughput improves immediately — the embedding step that previously ran at ~50 documents per minute on CPU now runs at 500+ documents per minute on GPU. Large knowledge bases that took hours to index take minutes. No configuration changes to the rest of the platform are needed; the embedding router picks up the plugin automatically.
You need to integrate Guaardvark with a proprietary internal API — a ticketing system, a data warehouse, or a monitoring platform. Write a plugin that implements a new agent tool calling your internal API, declare it in plugin.json, and drop it in plugins/. The agent gains the new tool without any changes to the core codebase. When Guaardvark releases a new version, your plugin continues to work as long as the capability hook API is stable.
Some capabilities have large dependency trees that are inappropriate for all instances. The Discord bot requires discord.py; the camera feed plugin requires OpenCV; a future video upscaling plugin might require BasicSR. The plugin architecture keeps these dependencies optional: the dependencies are only installed when the plugin directory is present, and the core requirements.txt stays lean for deployments that don't need every optional feature.
Many platforms control optional features with environment variable flags or config file switches. This approach works for binary on/off but doesn't support independent versioning, isolated dependency management, or community-contributed extensions. Guaardvark's plugin architecture treats optional capabilities as first-class packages: they have their own version numbers, their own dependency specifications, their own settings, and their own lifecycle independent of the core release. A plugin can be updated by replacing its directory without touching the core application. This is the foundation for a future community plugin ecosystem.
Create a directory under plugins/ with your plugin name. Add a plugin.json manifest and the plugin's Python entry point module. Restart Guaardvark (or call POST /api/plugins/<name>/reload for a live reload without a full restart). The plugin appears in the PluginsPage and its capabilities are registered immediately.
Plugin loading failures are caught and logged without preventing startup. The plugin_loader.py wraps each plugin load in a try/except block: if importing the entry module raises an exception, the error is logged and the plugin is marked as failed in the registry. The server starts normally and all other plugins load. A failed plugin can be reloaded or removed without a server restart.
The plugins/gpu_embedding/ directory serves as the reference implementation. The plugin.json schema is documented in the repository's docs/ directory. The capability hook API is documented in backend/utils/plugin_loader.py. A formal plugin development guide is planned for the v2.6 documentation release.
Yes. A plugin can register a Flask Blueprint as part of its entry module setup. The blueprint is registered with the Flask application during the plugin load phase, making its routes available immediately. This is how the social outreach API blueprint is handled in the outreach plugin.
Plugins run in the same Python process as the core application — there is no OS-level sandboxing. Plugins have access to all core modules and can call any function in the application. The trust model assumes plugins are authored by the platform operator or trusted developers. Community plugin sandboxing (via separate processes or restricted imports) is on the longer-term roadmap.