LLM Council critique loops: 3-vendor fan-out + chairman synthesis as a strategic substrate

There’s an obvious problem with using one AI model as your primary strategic advisor: you’re getting one perspective, shaped by one training distribution, with one set of implicit assumptions baked in. Most of the time this is fine. For decisions that matter — positioning, business model, architectural choices with long-term consequences — it’s not.

The LLM Council is the system I built to fix that.

The architecture

The council runs as part of /consult, which I invoke manually when I need strategic input on a domain — system design, business positioning, career decisions, brand strategy. It fans out to three external vendors via OpenRouter, synthesizes the responses with Claude in-session, and produces a structured output that surfaces where the models agreed, where they diverged, and what the divergence signals.

The three tracks:

• GPT-5 (OpenAI) — American frontier model, strong on business reasoning and market positioning. First choice for anything involving product strategy or go-to-market.
• Grok-4 (xAI) — Different training distribution, useful for sanity checks on assumptions the other models might share. Particularly good at surfacing contrarian takes.
• Gemini 2.5 (Google) — Strong on technical depth, useful for system design questions where the other two might be too business-oriented.

All three are routed through OpenRouter. Anthropic models are explicitly excluded from the OpenRouter payload — Claude Max handles Anthropic intelligence natively in-session, and mixing Anthropic-via-API into the council would add cost without adding diversity.

What synthesis actually does

The synthesis step is not “average the three responses.” That would wash out the useful disagreements. The synthesis step is:

1. Identify where all three models converge — that’s high-confidence signal.
2. Identify where two agree and one diverges — that’s where the outlier needs explaining. Either the outlier is wrong (and the majority is right) or it’s surfacing something the majority missed.
3. Identify where all three diverge — that’s genuine uncertainty. The right output is “here are the live options and their tradeoffs,” not a false synthesis that papers over the disagreement.

Claude in-session does this synthesis because it has the full context of my system, my constraints, and my decision history. The council vendors provide external perspective; Claude provides local context integration.

The operator lens stays in-house

One important constraint: the OPERATOR lens of /consult — analysis of my own decision patterns, mistake clusters, principles adherence — is never sent to external vendors. That analysis uses only Claude in-session, with MEMORY.md as the ground-truth principles document.

The reason is straightforward: my session patterns, decision velocity, and mistake history are not information that should leave the system. Business and technical questions can be externalized for council input. Operator self-analysis stays local.

What this produces in practice

The best council outputs I’ve gotten have been on positioning questions where my instinct and the models’ instinct diverged. The council surface the divergence clearly: here’s why you might be right, here’s why the models’ framing is worth considering, here’s the specific assumption that explains the disagreement.

That’s the goal. Not “here’s the answer” — I’m skeptical of any system that promises the right answer to a strategic question. The goal is “here are the live considerations, surfaced clearly enough that you can make a better decision than you would have made alone.”

The council is a thinking tool, not a decision-making tool. The decision is still mine.

---

The /consult skill is at Dev/.claude/skills/consult/SKILL.md. The council doctrine is documented in wiki/_global/decisions/2026-05-23-llm-council-architecture.md.