Multi-source AI: how a real assistant connects email, calendar, and tasks

A typical AI assistant has detectors. A meeting-prep detector reads your calendar. An email-action detector reads your inbox. A deadline-warning detector reads your task list. They run in parallel, each doing their narrow thing.

Each one is fine. Each one will surface the obvious moments. None of them will catch the moments that matter most.

The moments that matter most live across sources. Your flight got delayed AND you have a 4pm school pickup AND your spouse is travelling AND there is a tense work thread you have not replied to. No single detector sees this picture. A real assistant has to.

This is what we call multi-source fusion. It is one of the hardest pieces of AI assistant design. It is also where most of the actual value is.

Why single-source detectors keep missing the real moments

A single-source detector is bounded by its data. The meeting-prep detector knows you have a meeting at 3pm. It does not know your child's school just emailed at 1pm with a permission slip due Friday. The email detector reads the email but does not connect it to the fact that you already have evening commitments and would have to do this between 9 and 10pm tonight.

This is not a hypothetical edge case. This is most of life. The interesting moments are almost always cross-source:

• Your dentist appointment is at 2pm AND there is a 1:1 with your CFO at 3pm AND traffic is heavy. The agent should suggest moving the dentist.
• Your investor update is due Friday AND your traction numbers spiked AND you have a board meeting Monday. The agent should pull the data and start drafting.
• Your kid's birthday is in 14 days AND you are travelling that week AND last year you sent flowers from Bloom & Wild. The agent should suggest ordering early so they arrive in time.
• Your renewal for some SaaS tool is in 30 days AND usage has been low for the last quarter AND a competitor reached out. The agent should pull usage data and propose alternatives.

Each of these is multiple signals weaving into a single decision. No single detector finds them.

The architectural problem

Why do most AI products not do multi-source fusion? Two reasons, both real.

The signal-to-noise problem is much worse. A single-source detector that runs ten times a day with three useful results is a 30% useful rate. If you start fusing across five sources, the combinatorial space explodes — there are far more possible patterns than real ones. Most of what looks like a pattern is just two unrelated things happening at the same time. Bad fusion produces noise; users learn to ignore it permanently.

The latency and cost problem. Single-source detectors can be cheap deterministic rules. "Calendar event in 30 minutes with 5+ attendees, surface as meeting prep." Multi-source fusion almost certainly needs an LLM pass to detect actual patterns. That is more expensive and slower.

The verification problem. "Did the agent get this right?" is hard to answer for cross-source patterns. The user sees the suggestion and either agrees or disagrees, but there is no objective measure of whether the underlying pattern was real.

What good multi-source fusion looks like

There is a way to do this without exploding either cost or noise. The shape is:

Single LLM pass, strict. One call per cron tick, with a clear instruction: "Find at most ONE composite pattern that connects 2+ sources. Return null if nothing genuinely high-leverage connects." Most cron ticks return null. That is the right answer.

Bounded snapshot. Pull the recent 36 hours from each source — calendar, email, tasks, recent activity, existing single-source insights. Compress aggressively (subjects only on emails, titles only on events). Snapshot serialises to maybe 3-4KB total.

Explicit one-click question. The fusion output must end with a single question framed as a one-click decision. "Want me to handle that?" or "Want me to draft the reply?" — not a sprawling multi-paragraph analysis.

Time-bounded expiry. Composite insights are time-sensitive. The same pattern at 3pm is not the same pattern at 6pm. Expire after 12 hours and let the next cron tick reassess.

Stable deduplication. Use a hash of the source ids the LLM identified as connected. If the same sources connect again the next tick, the same insight upserts (does not double-emit).

This shape gives you the upside of multi-source fusion (the "school email + work thread + grocery list" moments) without the typical downside (noise, cost spikes).

The cost math

Worst case cost for a fusion pass with a fast cheap model (Claude Haiku class) is about 600 tokens output, maybe 1500 tokens input. At current pricing that is well under a cent per call.

If you run fusion every six hours, that is four calls per user per day. About four cents per user per month. For most products that is below the noise floor of operating cost. If a third of those calls produce a useful insight that saves the user 20 minutes, the ROI is enormous.

The trick is not to spread the cost across many shallow passes. The trick is to do one good pass per cycle.

Why this is also where calendar hygiene lives

Multi-source fusion has a quieter cousin: cross-time pattern detection within a single source. Calendar hygiene is the canonical example.

Most calendar entries are real. Some are not. They are commitments you accepted out of politeness, recurring meetings that died but the entry did not, time blocks you set up months ago and never removed. A naive AI agent treats every entry as real and ends up generating noise about meetings that do not actually happen.

Calendar hygiene is the cleanup pass. It looks back 60 days at recurring events. If the user has declined three of the last four occurrences — that is a ghost meeting. If a recurring entry has never been accepted, only "tentative" or "no response" — that is a ghost. Surface a low-priority hygiene insight. "You keep declining this. Want me to suggest cancelling?"

This is technically a single-source detector, but it is fusion across time rather than across sources. Same architectural principle: look at the patterns the obvious detectors miss because they only look at the present moment.

What this looks like for founders specifically

The founder use case for multi-source fusion is dense. A few examples we have seen surface:

The high-stakes meeting plus low signal. "Q3 board meeting in 5 days. Last quarter you sent the deck 7 days ahead. You haven't started this quarter's prep. Want me to pull metrics and start a draft?" — fuses calendar (board meeting), action history (prep cadence), task list (no prep task created).

The tense thread plus calendar conflict. "There is a tense email thread from a key customer. You have a 1:1 with your engineering lead in 15 minutes. Want me to draft a careful reply for after the 1:1, or schedule a callback?" — fuses inbox, calendar, time.

The renewal plus low usage plus competitor. "Salesforce auto-renews in 28 days at $X. Usage is down 40% from last quarter. A competitor reached out two weeks ago. Want me to pull together a renewal vs cancellation brief?" — fuses contract data, usage analytics, inbox.

The travel plus family commitment. "You are flying out Wednesday morning. Your spouse's birthday is Wednesday afternoon. Want me to send flowers ahead of time and book dinner for when you are back?" — fuses calendar, action history, contacts.

In each case, no single detector finds the pattern. A meeting-prep agent does not know about declining renewals. A renewal-watch agent does not know about competitor inbound. A travel agent does not know about anniversaries. Multi-source fusion is the only way to surface these.

Why most consumer AI does not ship this yet

Three reasons, in roughly this order.

First, the underlying single-source detectors have to work first. If your calendar agent is unreliable, fusing it into a multi-source pass just multiplies the unreliability. Fusion sits on top of solid detectors, not in place of them.

Second, the LLM cost-and-latency story has only recently become workable. Two years ago, fusing snapshots through a model would have been prohibitively slow and expensive for a per-user-per-day operation. With current frontier models at fast-cheap tier (Haiku, Gemini Flash, GPT-mini), it is now affordable.

Third, the false-positive risk is real, and most teams correctly do not want to ship fusion until they have telemetry to back it up. We feature-flag fusion off by default per user and turn it on after we see the false-positive rate is low enough not to burn the channel.

The takeaway

If you are evaluating an AI assistant, ask whether it does anything that requires looking at multiple sources at once. The school-email-plus-work-thread example is the diagnostic. If the agent only ever surfaces one-source insights, you are working with a tool, not an assistant.

If you are building one, fusion is one of the highest-leverage pieces of architecture you can ship — but only after the single-source detectors are reliable and the trust contract is in place. Done well, fusion is what makes the agent feel like a chief of staff. Done badly, it becomes the loudest source of noise in the user's life.

We have written separately about the anticipation gap, year-over-year memory, and outcome learning. Multi-source fusion is the layer that pulls them all into one coherent system. Without it the agent is smart in pieces. With it the agent is smart in moments — which is what users actually feel.