Worker Types Guide

Choose the right worker binary for your hardware and workload.

Available Worker Types

rbee supports multiple worker types for different hardware platforms and use cases.

LLM Workers (Inference)

Future Worker Types

rbee is building a complete AI suite with specialized workers for different tasks:

Planned Worker Types

Worker Marketplace

What You Can Find

Official Workers:

• LLM inference (CPU, CUDA, Metal, ROCm)
• Image generation (Stable Diffusion, FLUX)
• Video generation
• Audio processing
• Fine-tuning workers

Community Workers:

• Custom model implementations
• Specialized inference engines
• Domain-specific optimizations
• Niche AI tasks

Commercial Workers:

• Premium models
• Enterprise features
• Professional support
• Advanced optimizations

Coming Soon

The marketplace is currently in development. When launched, you’ll be able to:

• Browse workers by category (LLM, image, video, audio)
• Install with one click - automatic download and setup
• Rate and review - community feedback
• Publish your own - share or sell custom workers
• Auto-updates - keep workers up to date

Building Custom Workers

Want to create your own worker? rbee makes it easy!

The Worker Contract

All workers implement a simple HTTP contract defined in /bin/97_contracts/worker-contract/.

Required endpoints:

• GET /health - Health check
• GET /info - Worker information
• POST /v1/infer - Execute inference/task

Required behavior:

• Send heartbeat to Queen every 30 seconds
• Report status (Starting → Ready → Busy → Ready)
• Handle graceful shutdown

Implementation Checklist

When building a custom worker:

# 1. Clone the worker-contract
cd /bin/97_contracts/worker-contract/
cat README.md  # Read the complete specification
 
# 2. Implement required endpoints
# - GET /health
# - GET /info
# - POST /v1/infer
 
# 3. Implement heartbeat
# - Send to Queen every 30 seconds
# - POST /v1/worker-heartbeat
 
# 4. Test locally
rbee-hive spawn-worker --worker your-worker --device 0
 
# 5. Package and distribute
# - Via marketplace (coming soon)
# - Via GitHub releases
# - Via custom distribution

Example: Minimal Worker Structure

use worker_contract::{WorkerInfo, WorkerStatus, InferRequest, InferResponse};
use axum::{routing::{get, post}, Router, Json};
 
// Health check
async fn health() -> &'static str {
    "ok"
}
 
// Worker info
async fn info() -> Json<WorkerInfo> {
    Json(WorkerInfo {
        id: "my-worker-123".to_string(),
        model_id: "my-model".to_string(),
        device: "GPU-0".to_string(),
        port: 9301,
        status: WorkerStatus::Ready,
        implementation: "my-custom-worker".to_string(),
        version: "1.0.0".to_string(),
    })
}
 
// Inference
async fn infer(Json(req): Json<InferRequest>) -> Json<InferResponse> {
    // Your inference logic here
    Json(InferResponse {
        text: "Generated output".to_string(),
        tokens_generated: 42,
        duration_ms: 1500,
    })
}
 
// HTTP server
let app = Router::new()
    .route("/health", get(health))
    .route("/info", get(info))
    .route("/v1/infer", post(infer));

Contract Reference

Key types:

• WorkerInfo - Worker metadata and status
• WorkerStatus - Current state (Starting, Ready, Busy, Stopped)
• WorkerHeartbeat - Periodic status update
• InferRequest - Inference parameters
• InferResponse - Inference result

Extension points:

• Multi-model workers (vLLM, ComfyUI)
• Dynamic VRAM reporting
• Workflow progress tracking
• Batch inference support

Device Selection

Single GPU

# Use device 0 (first GPU)
rbee-hive spawn-worker --worker cuda --device 0

Multiple GPUs

# Spawn workers on different GPUs
rbee-hive spawn-worker --worker cuda --device 0 --model llama-3-8b
rbee-hive spawn-worker --worker cuda --device 1 --model llama-3-70b

Mixed Hardware

# Gaming PC: CUDA on GPU 0, CPU as fallback
rbee-hive spawn-worker --worker cuda --device 0
rbee-hive spawn-worker --worker cpu --device 0
 
# Mac M3: Metal only
rbee-hive spawn-worker --worker metal --device 0

Next Steps