Alpha.so

Core loop: Vercel AI SDK Tool Loop Agent running inside Temporal workflows. Model gets a task, skills, and a prompt that lets it load more skills on demand — it decides when it has enough information to complete the task. Custom sandbox per bot: persistent virtual filesystem backed by S3 directory buckets, with tools that mimic real fs operations (list, read, write, delete) and virtual CLIs that route commands and return help text the model can iterate on. No processes spawned — just function routing that looks like a terminal to the LLM. On top of that, a skills system — composable tool+prompt bundles the bot loads depending on context: market analysis, portfolio management, news digestion, macro environment, options flow.

Three search dimensions: Meilisearch + Typesense for full-text hybrid search across financial data, Milvus for vector semantic similarity across documents, and FalkorDB for graph relationships between entities, sectors, and market signals. Exa.js for AI-native web search feeding into bot context.

Temporal harness runs everything. Workflows for agent execution, activities for LLM calls and trade ops, scheduling producers for recurring tasks like market open checks and rebalancing. Custom webpack bundling for workflow sandboxing. Workers auto-scale on EKS with custom controllers based on queue depth. Self-hosted Temporal via Pulumi IaC.

PostHog for AI tracing — session attribution, full tracking of prompts and outcomes. Evals running on captured traces with LLM-as-a-judge: checking tone, accuracy, and whether bots stay friendly for beginners who don't know what a stop-loss is. Prompt changes rolled out via feature flags with evals comparing control vs experiment groups. E2E tests with real LLM calls, evals as assertions, middleware that caches LLM responses when model + prompts remain unchanged.

Built the test harness that lets coding agents ship confidently. Full replica of production systems available for E2E runs, middleware that caches LLM calls and responses for deterministic replay when model + prompts remain unchanged. Agents can run tests autonomously, iterate on failures, and ship without human intervention on the testing side. Part of the job now is enabling agents to work — this infrastructure is what makes 10-15 PRs/day possible.