LLM Selection Tradeoffs for Meeting Intelligence Pipelines

A meeting intelligence pipeline runs through five or six distinct processing stages, and each one has its own input format, latency budget, and tolerance for error. The model that's right for one stage is often wrong, or just wasteful, for the next. Treating "which LLM should we use" as a single question is the most common architectural mistake teams make when they build or buy these systems. The right question changes depending on where in the pipeline you're asking it, and most teams never get around to asking it more than once.

Roughly three out of four professionals now use some form of AI note-taker, according to research published on dev.to. That number matters because model-selection decisions stopped being theoretical a while back. They're live, in production, inside sales orgs and product teams and executive staffs right now, and whatever gets chosen today is already shaping how much value those teams recover per meeting hour.

The stages of a meeting intelligence pipeline and their demands on a model

Start with transcription. This is an ASR problem, not strictly an LLM problem, though some architectures lean on a language model for punctuation cleanup, diarization correction, or injecting domain vocabulary (ticker symbols, drug names, internal product codenames) that a generic acoustic model would otherwise mangle.

From there, the raw transcript hits the extraction stage, where the core LLM task turns a wall of text into a summary, a decision log, and an action-item list with owners and deadlines attached. This is the stage most people picture when they hear "meeting intelligence," and for good reason: it's the one that produces the artifact a human actually reads.

Downstream of that sits workflow routing, where structured output gets pushed into a CRM field, a Linear ticket, or a Slack digest. Then there's organizational memory: embedding transcripts, connecting them into a knowledge graph, making cross-meeting search possible. A fifth stage is emerging on top of all of it, a proactive layer that monitors commitments across meetings and surfaces reminders without being asked.

Each stage has a different output contract, and that's the whole ballgame. Extraction has to produce fluent, human-readable prose. Routing has to produce a JSON payload a CRM field will accept without choking on it. Retrieval has to produce embeddings or metadata a search index can rank against. Fluency matters enormously at one stage and barely at all at another. Schema adherence is nearly irrelevant to extraction quality but existential to routing. Picking one flagship model for the whole chain answers a question nobody asked. Nobody was ever asking "which model is best." It's "which property does this stage need most."

Latency vs. depth: why the extraction stage is where model choice has the largest impact on perceived quality

Extraction is what the end user reads first, and it's the stage where a wrong model choice becomes immediately, embarrassingly visible. A bad summary doesn't hide behind an API call somewhere. It lands in someone's inbox looking exactly as wrong as it is.

Transcription accuracy is close to a solved problem at this point, a commodity layer most vendors clear without much differentiation, a framing echoed on Granola's blog. The real differentiation happens in what the LLM does with a clean transcript afterward. That's a reasoning and judgment problem, not a speech-recognition problem, and it splits into two demands that pull against each other.

Depth is one axis. A model that reads the full transcript, rather than skimming for keywords, catches the gap between what a founder said in prepared remarks and what they admitted three questions later under pressure. It flags hesitation, pulls exact figures, and preserves a caveat that a shallower model would flatten into a confident-sounding but wrong summary. Latency is the other axis, and it pulls the opposite direction. Someone walking out of a call wants notes inside a minute, not ten, and a full reasoning pass across a ninety-minute transcript with a large frontier model can blow well past that window.

The architecture split that follows is genuine, and here's where most teams get it backwards by defaulting to one model for every meeting on the calendar. A recurring internal standup doesn't need depth: a distilled model produces a usable draft fast, and the stakes of missing a nuance are low. A board meeting, an investor call, or a high-value sales conversation is a different animal, where a missed caveat costs real money and the wait for a frontier model is worth paying. Some pipelines split the difference outright: a fast model delivers a working draft the moment the call ends, and a deeper model runs asynchronously in the background, delivering a refined version a few minutes later.

Human-in-the-loop design changes the math, too. When a user's own rough notes anchor what the AI enhances, the model doesn't have to guess what mattered in the meeting, because the human already told it. That lets a faster model punch above its weight, matching the relevance of a slower one simply because it isn't doing the hardest part of the job alone. Latency tolerance is a function of meeting type and role, a variable property of the pipeline rather than a fixed one. Segment first, choose the model second.

Context window size and the real cost of long meetings

A one-hour meeting with five or six active speakers can produce a transcript that pushes against the practical limits of a smaller context window. A multi-hour workshop or company all-hands blows past it without much trouble.

Dumping the entire transcript into whichever model has the largest context window available works, technically. It also scales badly, and this is where a lot of teams overspend without noticing. Token-based pricing means cost rises faster than transcript length, so "just use the biggest window" turns expensive fast once a team runs dozens of long calls a day instead of one.

Chunking is the usual fix, and every version of it fails in its own particular way. Splitting by time is cheap and fast, but a commitment made early in a call and confirmed again later can land in two chunks that never get read together, and the model misses the connection. Splitting by topic, using a cheaper model to segment first, preserves meaning better but adds an entire extra stage to the pipeline, with its own error surface and its own chance to go sideways. Hierarchical summarization, summarizing chunks and then summarizing the summaries, tends to blur exactly the details that matter most: specific dollar figures, names, dates.

The organizational memory layer makes this worse. If the memory stage stores full transcripts for later retrieval, the embedding model's context limit becomes a second, separate constraint stacked on top of whatever limit the summarization model already imposed. These are not the same bottleneck, even though teams routinely treat them as one.

None of this is free. Reserve the large-context frontier models for meetings that clear a length threshold or a business-value threshold and let smaller models with smarter chunking handle the routine stuff. Anything else is paying frontier prices for a standup, and the savings a pipeline is supposed to generate get eaten by inference cost while nobody's watching the meter.

Instruction-following vs. raw capability: why schema adherence matters more than benchmark scores for workflow routing

Workflow routing doesn't want prose. It wants a payload in an exact format that a downstream system expects and will reject if it's even slightly off, whether that's a CRM field, a task object, or a Slack message.

The failure mode here is a malformed object that breaks an integration outright, or worse, one that silently writes the wrong value into a live contact record with nobody noticing until a sales rep pulls up a deal and finds garbage sitting in the close-date field. That's a different category of risk than a summary reading slightly off, and it calls for a different kind of model evaluation.

Instruction-following is a distinct, measurable property, separate from raw capability, and this is where most teams pick the wrong axis to optimize. A model with a lower aggregate benchmark score can still be the better choice for this stage if it's been tuned specifically to stick to a schema, because consistency keeps outputs valid and predictable. Cleverness alone won't stop a model from breaking the schema the moment it gets creative. Function-calling and tool-use support aren't optional extras here, they're the baseline requirement. A model that can't reliably call a tool has no business writing to a CRM field unsupervised, full stop.

This is where the Model Context Protocol becomes relevant. MCP standardizes how an AI assistant calls external tools, and by December 2025, according to Anthropic's own announcement, it had cleared more than 97 million monthly SDK downloads and over 10,000 active public servers, production plumbing rather than a research demo. That's production plumbing, not a research demo. A meeting intelligence pipeline built on MCP exposes action items, contact records, and deal stages as callable tools, but the whole arrangement depends on the model respecting the tool schema every time, not most of the time. At MCP's scale, a model that occasionally ignores the schema doesn't fail quietly. It writes bad data into live systems at volume.

Native integrations, which write directly into CRM objects, carry a higher bar for model reliability than Zapier-style connections do, precisely because no human checks the output before it lands. For this stage, look at instruction-following and tool-call reliability first and benchmark scores second. A smaller, more disciplined model beats a larger, more "capable" one on this job more often than the industry's marketing wants to admit, and any team still routing on benchmark rank alone is optimizing for the wrong axis.

Model selection at the intersection with capture architecture: bot-based vs. device-level pipelines have different LLM requirements

How a meeting gets captured shapes what the extraction model receives, and that upstream choice constrains which model can do a good job downstream, whether anyone planned it that way or not.

Bot-based capture, where a virtual participant joins the call and processes audio from the platform's own stream, tends to produce cleaner diarization, since the platform assigns separate speaker channels on its end. The tradeoff is visibility: the bot sits in the participant list, and people talk differently when they know it's there. Device-level capture, listening to system audio and microphone input directly, skips that visibility problem but loses the clean channel separation, so the transcript arrives messier, with speaker attribution that has to be inferred from context rather than read off a label.

That difference dictates which model belongs at the extraction stage, not just which model is nice to have. A pipeline built on device-level capture needs a model genuinely good at inferring who said what from conversational cues, tone shifts, turn-taking patterns, rather than one trained on clean, pre-labeled input. Feed that same model a bot-based transcript with proper speaker channels, and its job gets noticeably easier.

Platform-native tools sidestep the whole tradeoff by controlling both ends of it themselves. Teams' Copilot runs on Microsoft's own AI infrastructure, Zoom's AI Companion runs on its own mix of AI models, and Google Meet's assistant is integrated into Workspace's AI features. Owning capture and inference together costs the platform flexibility but removes the burden of making this tradeoff at all. For any team building or evaluating a third-party pipeline, capture method is a constraint on model selection. The two get chosen as a pair, or the pipeline inherits a mismatch nobody notices until the transcripts start coming back garbled.

Organizational memory and retrieval: the embedding and retrieval layer has its own model selection problem

Organizational memory means something specific: a structured, persistent layer that lets a company's AI systems retain, connect, and reuse what's been learned across meetings, in contrast to a pile of static transcripts nobody revisits. The architecture generally runs capture into consolidation (semantic processing, pruning stale knowledge) into retrieval (a hybrid of knowledge graph lookup, metadata-tagged RAG, and plain file search). Each of those sub-stages can, and often should, run on a different model.

Embedding models carry their own version of the capability tradeoff, and this is where general-purpose defaults quietly fail. A general-purpose embedding model trained mostly on web text underperforms on a transcript full of internal product codenames, industry jargon, and acronyms nobody outside the company would recognize. Domain-adapted embeddings do better, especially for legal, medical, or engineering teams with genuinely specialized vocabulary, though they need upkeep as that vocabulary shifts. The choice determines something concrete: whether a search for "pricing objection on the Acme call" actually returns that transcript segment, or something loosely adjacent that wastes the searcher's time.

The more advanced end of this layer goes past retrieval into reasoning across many stored meetings at once, flagging that a feature under discussion right now was tried and abandoned eighteen months ago. That's a multi-document reasoning task, distinct from summarizing a single transcript, and it needs a model built for cross-referencing, with tuning suited to many documents at once rather than a single clean one.

The cost of getting this wrong isn't abstract. New hires at large companies typically take six to twelve months to reach full productivity, according to Brandon Hall Group data cited by coworker.ai, and a retrieval layer that surfaces the reasoning behind a past decision compresses that ramp directly. Without it, each meeting stays an island. Three out of four professionals may be using an AI note-taker, but if the tool stops at the meeting room door, nothing connects one captured meeting to the next. That's not memory; that's a growing pile of separate files. The retrieval model has to be judged on recall across a large corpus, not accuracy on a single document, and treating that like the same problem as extraction is a common, costly mistake.

The proactive agency layer's risk profile in the pipeline

Active agency flips the interaction model on its head. Instead of waiting to be asked, the system watches commitments accumulate across meetings and pushes a reminder unprompted: "you promised Acme Corp the proposal by Friday," straight into Slack.

That's a fundamentally different task than summarizing, and it carries fundamentally different stakes. The model is now judging what to surface, to whom, and when, and a wrong call here has direct consequences in a way a slightly-off summary doesn't. Three failure modes matter, and they are not equally bad. A false positive, surfacing a commitment that was never made, erodes trust in the system fast. A false negative, missing a real one, defeats the entire point of building the layer. An attribution error, pinning the commitment on the wrong person, is arguably the worst of the three, because it doesn't just fail technically, it creates an awkward, credibility-damaging moment between colleagues who now have to sort out who actually said what.

What this stage needs from a model is narrow and specific: high precision on detecting an actual commitment, reliable speaker attribution, and calibrated confidence that lets it say nothing rather than guess. This is the one stage in the pipeline where cutting cost with a cheaper or faster model raises the failure rate immediately and visibly. Treating it like the extraction stage, where a faster model is often the right call, is exactly backwards here. Quality investment earns its keep at this position in a way it doesn't elsewhere in the chain.

The category is still young, and best practices haven't caught up to deployment speed. Institutional knowledge retention is increasingly recognized as a core AI investment priority, and this layer is landing in production environments faster than anyone has worked out the failure modes for. That's a real risk to build around carefully, not a corner to cut for speed. Favoring speed and cost over precision at this position, out of habit carried over from earlier stages, is the mistake to avoid.

Mapping model selection decisions to pipeline position: a practical decision framework

The organizing principle produces a rule that's simple to state and easy to ignore under deadline pressure: assign model properties to pipeline stages based on the stage's output contract alone, regardless of whichever model has the flashiest benchmark chart this quarter. A vendor's marketing claim is not the input to this decision. The stage's requirement is, and nothing else gets a vote.

Applied stage by stage: evaluate ASR and diarization correction on word-error rate against actual audio conditions. Assess the extraction stage on depth of reasoning and contextual inference, with latency tolerance set by who's in the meeting and what's at stake. Judge the routing stage almost entirely on instruction-following and tool-call reliability, since a brilliant model that occasionally ignores a schema is a liability at that position, not an asset. Judge the retrieval and embedding layer on recall across a full corpus of meetings, a different test entirely from anything upstream. Treat the proactive layer as the highest-risk position in the chain, worth the extra cost of a more careful, more conservative model, because its failure modes touch trust between actual people, not just data quality in a database.

None of this resolves into a single "best model" recommendation, and that's deliberate. The question itself is the wrong one. A fleet of models, chosen stage by stage against each stage's actual demands, consistently outperforms one flagship model asked to do every job in the chain. Teams getting real value out of meeting intelligence right now have stopped asking which model is best. They're asking, for each stage, what that stage actually needs, and buying accordingly.