AI

How AI Call Summaries Actually Work (And Why Most Vendors Get Them Wrong)

A technical look at what happens between the moment a call ends and the moment a one-paragraph summary lands in your inbox. And why "AI summary" on most vendors means something different than it does here.

Vocatech Team·April 12, 2026·8 min read

Every vendor in business phones now puts the phrase AI call summaries on a feature list. Most of the time, that phrase means the same thing. A recording is shipped to OpenAI. OpenAI ships back a paragraph. The vendor charges you extra for it.

That is not what it means here. This article walks through the pipeline we built, the choices we made, and why the word summary is doing very different work when you see it in different product sheets.

The Ten-Second Version

A call ends. Inside a minute, a three to five sentence paragraph shows up in Reports. Who called. What they wanted. What was agreed. That paragraph rides out to the customer record in Salesforce, HubSpot, Zoho, QuickBooks, or whatever CRM the account is wired to.

The steps in between are where the product gets built.

Step One. Recording the Call

When a call runs on Vocatech, our sip_server (written in C++) captures the audio at the SIP layer. Not from a phone, not from a browser. From the switch itself. That matters for a reason most people never think about.

The recording is dual-channel. The caller is on one channel. The agent is on the other. They never mix. When the call is done, the two channels are merged with a C++ utility called pad_silence that aligns the timing, then passed to ffmpeg with -b:a 16k to produce a compact MP3.

Why dual-channel matters. When transcription runs, the model knows which words came from which speaker before it does any diarization work. On telephony audio, that cuts the speaker-labeling error rate to almost nothing. Most hosted VoIP platforms record mixed-mono because it is easier. Their summaries suffer for it.

The merged MP3 uploads to Google Cloud Storage. The file lives there. Every downstream step reads from GCS.

Step Two. Transcription

The audio is handed to a speech model inside our own Google Cloud environment, under the same agreement that covers the rest of the platform. It is never uploaded to a hosted speech-to-text service outside that environment.

A few things matter for telephony audio specifically:

  • The whole call. The model is given the call as it happened: every stop along the way, the holds, the earlier parts of a parked call, and the same number's calls in the last 12 hours, so names and context carry across.
  • Telephony audio. A phone call is narrowband, and the model reads it as it is. The words are there.
  • Language detection. Spanish calls transcribe in Spanish. A bilingual call flips mid-sentence without breaking. For the homecare agencies on the platform, this is not a bonus feature, it is the feature.

A transcript lands within the hour of the call ending, usually much sooner. The pipeline is designed so transcription never blocks recording storage. The recording is durable the instant it hits GCS. Transcription catches up behind it.

Step Three. Summary Generation

The summary is written by a language model running inside our own Google Cloud environment, under the same agreement that covers the rest of the platform. The transcript goes to it with a narrow system prompt. The prompt asks for three to five sentences, in a specific format: who called, what they needed, what was agreed as the next step.

Nothing about the call leaves that environment. Not the audio, not the transcript, not the phone number metadata, not the CRM mapping. That is a very different privacy posture from a platform that ships the raw MP3 to a third party with everything attached.

If the model is unavailable, the call still gets its transcript. It lands in Reports without a summary, and the summary catches up when the model returns.

Step Four. Storage and Surfacing

The finished record has a recording URL, a transcript, and a summary. It writes to the Recordings database with the call metadata (numbers, duration, user, timestamps). It shows up in Reports, our admin-portal call log, with the summary preview inline and the transcript one click away.

If the customer has an integration configured, the summary pushes out to the mapped CRM record. Salesforce activity timeline. HubSpot contact note. Zoho call log. QuickBooks customer notes. HHAeXchange visit note for the homecare agencies. Whatever the customer's system of record is, that is where the summary ends up, with no manual step from the agent.

Integrations run on our public API and webhooks. Your developer or a no-code tool connects what you use; the call.transcription event carries the summary and transcript to your system once they are ready, usually minutes after the call.

Why It Stays Inside Our Environment

Three reasons, in order of how often they come up.

Privacy. Customer audio does not cross our trust boundary during transcription. For medical offices, law firms, and anyone with a compliance officer, that sentence is the whole pitch. The agreement that covers the rest of the platform covers this step too; there is no second vendor you have never heard of.

Cost. Transcription and summaries are included in the flat $29.95. There is no AI tier and no per-minute add-on.

Latency. The pipeline stays inside one region. A transcript lands within the hour of the call ending, usually much sooner.

What the Summary Actually Looks Like

A real example, from a property management customer, with the names changed:

Marcus Fletcher called about a leak in unit 4B at the 858-area-code property. The tenant reports water coming from the ceiling in the bathroom. Agent scheduled a plumber to arrive tomorrow between 9 and 11 AM and agreed to follow up with the tenant once the repair is confirmed.

That is the shape. Three sentences. Who. What. Next step. A receptionist reading twelve of these at the start of a shift knows exactly what the building walked in on. A sales manager spot-checking a rep's week reads forty and knows where the deals are stuck.

No adjectives. No tone analysis. No AI-generated pleasantries about how the customer was satisfied with the service. Those are the tells of a summary that is not doing any real work. Ours does the work, then stops talking.

The Short Answer to the Question in the Title

Most vendors ship audio to a third party and call it a feature. We recorded the call ourselves, transcribed it inside our own environment, summarized the text through a narrow prompt, and delivered the result to the CRM before the agent finished their after-call notes.

The distance between those two sentences is the distance between AI summary as a marketing phrase and AI summary as a thing your operations team actually uses.

About Vocatech

Vocatech is a business phone service built on Cisco BroadWorks with our own platform layered on top. Callpop for desktop caller context. Reports with AI transcription and summaries. Textdock for SMS and WhatsApp from your business number inside Cisco Webex. A public API and webhooks so your own tools connect.

Flat $29.95 per seat. Month to month. Founded 2008. Over a thousand businesses. 97% retention.

Start a trial at vocatech.com/contact or see the platform at vocatech.com/platform.


Vocatech is a business phone service with an enterprise Cisco backbone, layered with our own AI. One flat price, every feature included, month to month, real humans on the line. Start a free trial or see pricing.