LLM Selection Tradeoffs for Meeting Intelligence Pipelines

If a meeting transcript has errors, they flow forward into every summary, every action item, and every CRM field built on top of it, but a bad summary never reaches back to corrupt the words that were actually spoken. That one-way relationship is the central fact of meeting intelligence architecture, and it means the two layers that make up the pipeline, speech-to-text with diarization on one side and LLM-based understanding on the other, cannot be evaluated by the same standard or chosen with the same criteria. Summarization, action-item extraction, CRM field mapping, and cross-meeting search all sit downstream of a transcript, and all of them inherit whatever mistakes that transcript already contains. Because the dependency runs in one direction only, a team can spend heavily on the best available language model for understanding and still produce unreliable output if the words feeding that model were wrong to begin with, and a team can get transcription right and still waste that advantage if the reasoning layer on top of it is weak. Every tradeoff discussed in this piece, accuracy, cost, latency, context length, and routing, belongs to one layer or the other, and conflating the two is where most production failures start.

Why ASR accuracy in meetings is the load-bearing wall

Marketing copy for meeting tools tends to quote accuracy figures from read-speech benchmarks, but those numbers describe a different task than the one a meeting pipeline actually performs. Read speech is clean, paced, and spoken by someone who just narrates text aloud. Meetings are not. Meeting audio carries background noise, people talking over each other, room acoustics that vary from one conference room to the next, and the stops, restarts, and filler words of unscripted speech, none of which read-speech test sets capture. When the two conditions diverge this much, a downstream summary or action-item list can end up with different content.

That gap appears directly in the Hugging Face Open ASR Leaderboard, where a field of more than 60 speech recognition systems was tested on the AMI meeting corpus. None of them reached the word error rate that most vendor marketing implies you should expect, but the same systems did far better on read audiobooks. The comparison matters because it shows the shortfall is a property of meeting audio itself, one that holds across dozens of different models and architectures.

Microphone placement adds a second, compounding source of error. If you swap a headset microphone for a distant room microphone, Whisper's error rate on the AMI corpus more than doubles. If you build for in-person meetings, not calls where every participant wears a headset or speaks close to a laptop microphone, you need to account for that gap before you trust a vendor's published accuracy number.

Diarization, the task of attributing each stretch of speech to the correct participant, fails independently of word-level accuracy and deserves its own scrutiny. A transcript can get every word right and still assign half the sentences to the wrong speaker. In a roundtable setting with several people talking over one another, diarization accuracy can fall to 70%, regardless of how clean the underlying transcription is.

English transcription in clean audio with native speakers has gotten tight across the major providers, so the real differentiation among ASR systems now happens under meeting-realistic conditions: accents, multiple languages in the same call, specialized vocabulary, and the acoustic variation of real rooms. A leaderboard position measured on clean, scripted audio says very little about how a system will behave under meeting-realistic conditions like accents and a conference-room speakerphone. If you build a pipeline, you need to test ASR candidates against audio that resembles your own meetings, not against the benchmark a vendor chose to publish, and you need to run that test before you decide which language model will sit on top of the transcript.

How the LLM layer's tradeoffs differ from ASR

Once a transcript exists, the work remaining is not one task but several, each with its own demands on cost, speed, and reasoning depth. Transcript normalization, meeting summarization, action-item extraction, CRM field-mapping inference, and cross-meeting search are distinct jobs, and a model well suited to one is not automatically well suited to the rest. Normalization is mostly cleanup work. CRM field mapping and summary generation need actual judgment about what a sentence in a meeting meant and whether it counts as a commitment worth recording. Treating all five as a single model-selection decision ignores how differently they behave.

Cost reinforces the same point. A production analysis of frontier language models from May 2026 found that the price gap between the most and least expensive models scoring in the same performance range is wide enough to turn model choice into a major line item at volume. For a pipeline processing many recordings a day, every one of those five subtasks running on the most expensive available model adds up fast, and the spread between models of comparable quality means that cost is a design choice, not a fixed tax.

Speed and ceiling quality trade off against each other at this layer in a way that doesn't apply to ASR. The same May 2026 analysis lays out the tradeoff: when a pipeline is processing dozens of recordings at once, a high-throughput model is the right pick, and a higher-ceiling model only earns its cost on tasks that genuinely require nuanced reasoning. That distinction carries a real user-facing consequence too. A sales rep waiting on a summary immediately after a call ends experiences latency as a product failure, not a backend inefficiency, so the speed-quality tradeoff is a user-experience decision as much as an infrastructure one.

Context length is the fourth place where the LLM layer diverges from ASR. A transcript from a full day of back-to-back meetings can run long enough to exceed the context window of a cheaper model, forcing the pipeline into chunking strategies that risk losing coherence across the summary. Gemini 3.1 Pro's true production context window matters specifically for this case: long single meetings or multi-meeting summarization tasks that would otherwise require breaking the transcript apart and reassembling the summary through a map-reduce process. That capability is not relevant to every subtask in the pipeline. You need it for the subtasks where losing the thread across a long document costs real accuracy, and naming it here shows a tradeoff dimension rather than crowning a winner.

Open-weight models bring in a further variable you should track, even though it falls outside the cost-speed-context frame above. For meetings full of domain-specific vocabulary, legal depositions, clinical consultations, engineering design reviews, a smaller model fine-tuned on a company's own data can outperform a generalist frontier model while costing less and running faster. That option exists because fine-tuning narrows the task, and a narrower task needs less general capability to perform well.

Task routing as a design decision, not an optimization afterthought

Routing each subtask to the model suited to it belongs at the architecture stage, not as a patch applied after costs or quality problems surface. Research on a method called Sequential Bandits, built on neural contextual multi-armed bandits, gives this approach formal backing, routing the best-suited language model to each step of a multi-step pipeline while jointly optimizing for accuracy and cost, and it outperformed existing bandit baselines on pipeline task benchmarks, including medical diagnosis prediction and telecommunications question answering. The underlying principle translates directly to meeting pipelines: send cheap, fast models to transcript normalization and structured extraction, and reserve the expensive, high-reasoning models for the steps, like nuanced summary generation or CRM field-mapping inference, where getting the reasoning wrong actually breaks a downstream workflow.

Production evidence backs the same conclusion. A paper from IC2E 2026, examining LLM tradeoffs in enterprise automation, documents a workflow-generation task broken into stages, variable scaffolding, base block assembly, and nested block generation. That decomposition raised structural success rates substantially across every model tested, and it let a smaller, cheaper model, mistral-small, reach production viability without depending on a frontier-tier model at all. The cost spread in that study was wide: the best-performing model in the comparison carried a 19x cost premium over the cheapest model that still worked in production. Decomposition is what made that 19x gap something a team could actually navigate instead of a cost they simply had to absorb.

The same paper draws a distinction that applies directly to meetings: a workflow can be structurally valid and well-formed, executing without error, while still being semantically wrong. A meeting summary behaves the same way. It can read cleanly, with proper formatting and plausible-sounding bullet points, and still miss the one decision that the meeting was actually about. Structural success and semantic correctness are different measurements, and a pipeline that only checks the first one will miss failures of the second.

A 2026 guide to LLM workflow patterns names the architecture that fits this situation: the router pattern, where a classifier model reads each input and sends it to a handler built for that category. You pay one extra model call's worth of latency, but you get meaningfully better output quality per category in return.

Put together, the sources support a specific routing heuristic: send transcript normalization and boilerplate extraction to smaller, cheaper models, in the range of what Haiku or GPT-4o mini represent in terms of capability and cost, and reserve the larger, more expensive models for the steps where reasoning quality decides whether a commitment gets captured correctly or a CRM field gets filled with something actually meaningful. Because every input already passes through a classification step before it reaches a handler, you also get a natural place to enforce compliance and data-governance rules. Routing architecture and security policy function better as one layer than as two systems bolted together after the fact.

Evaluation built into the pipeline at each layer

You can build a pipeline correctly on day one, and it can still fail quietly, months later, without a single visible error. Picture the sequence: an ASR provider ships a model update that improves word error rate on clean speech but makes it worse for accented speakers; the LLM layer keeps generating summaries that read just as plausible as before; no alert fires anywhere in the system; action items start getting attributed to the wrong person; and users lose trust in the tool well before anyone on the team notices why. Nothing in that chain throws an exception. Each piece just degrades a little, and the degradation compounds across the two layers until the output is unreliable in a way that no single log line reveals.

The defense against that failure mode is evaluation that runs continuously and is structured in layers of its own, cheap tests first, expensive tests last. A guide to building LLM evaluation pipelines lays out that ordering directly: deterministic tests run first and catch obvious breakage, heuristic and statistical scoring catches more subtle semantic drift, and LLM-as-a-judge evaluation, the most expensive method, runs only on whatever survives the earlier, cheaper stages. So you can run evaluation on every change instead of once before launch, because it stays affordable.

Meeting pipelines need two separate evaluation datasets, built for two separate purposes. ASR evaluation needs a golden dataset of meeting-realistic audio paired with known-correct transcripts. LLM evaluation needs a golden dataset of meetings paired with known-correct summaries, action items, and CRM field values. These are different annotation tasks that need different expertise, so treating them as one combined dataset tends to produce something that tests neither layer well.

Benchmark leaderboards move quickly, so a model's current standing says little about how it will perform in a specific pipeline six months from now. The model that scored best when a team first built its pipeline may not be the right choice half a year later, and the only reliable way to know is a pipeline-specific evaluation run against a team's own golden datasets, not a general leaderboard ranking. That is the operational cost of running a two-layer pipeline correctly: evaluation has to be built in at both layers, running continuously, measured against data that resembles the meetings the pipeline actually processes.