Learn AI Create the Future — with Pranay
Main site

Summarisation

Summarising meetings and conversations

Summarising a meeting means pulling out decisions and action items from back-and-forth dialogue, not condensing prose the way an article summariser does.

Read these first

On this page 5
  1. Why it exists
  2. How it works
  3. Where you have already seen it
  4. Remember this
  5. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

Meeting summarisation pulls out the decisions and action items from a conversation. It is not a shorter version of everything said.

Picture missing a work meeting. You ask a colleague "what did I miss?" You do not want a transcript read back. You want three things: what got decided, who owns what, and what changed.

That is a different job from summarising a news article. A meeting is not one voice explaining something. It is several people, interrupting, agreeing, changing their minds. A meeting summariser has to track all of that.

Why it exists

Article summarisers assume one author, writing in order, building one argument. Meetings break every one of those assumptions.

People talk over each other. The same topic gets revisited three times before a decision sticks. Half the content is small talk, throat-clearing, or discussion that leads nowhere.

A useful meeting summary skips almost all of that. It keeps decisions, such as "we're pushing the demo to Friday". It keeps commitments, such as "Priya will update the risk log". Everything else gets dropped, even though that was most of what was said.

How it works

Asha:  Let's start with the delayed shipment.
Rahul: The vendor says parts arrive Thursday, three days late.
Asha:  Can we push the demo to Friday?
Priya: Friday works, I'll update the client.
Rahul: I'll ask the vendor for a discount.
                |
                v
        Filter for decisions and owners
                |
                v
Summary: "Demo moved to Friday due to vendor delay.
          Priya to update client. Rahul to negotiate discount."

The model has to connect "Friday works" back to "push the demo to Friday" several lines earlier. It also has to connect "I'll update the client" to Priya specifically, not whoever spoke last.

Where you have already seen it

  • Zoom, Google Meet and Teams auto-generated meeting notes. All produce a decisions-and-action-items summary, not a transcript.
  • WhatsApp and Slack thread summaries. The same filtering logic, applied to text chat instead of speech.
  • Call centre summary tools. Turning a ten-minute support call into three lines: issue, resolution, follow-up needed.
  • Doctor's visit summary apps. Extracting diagnosis and next steps from a spoken consultation.

Remember this

  • Meeting summarisation extracts decisions and action items, not a shorter version of everything said.
  • It has to track who said what, and connect later agreement back to an earlier proposal.
  • Most of a meeting's actual words are discussion, not decisions — and get dropped entirely.

What to learn next

Developer — Code and libraries.

A general-purpose summariser like t5-small was never trained specifically on dialogue. Below, we see exactly what that gap looks like in practice.

Setup

bash
pip install transformers torch

Summarising a short transcript

meeting.py
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("t5-small")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")

transcript = """summarize: Asha: Let's start with the delayed shipment.
Rahul: The vendor says the new parts will arrive Thursday, three days late.
Asha: Okay. Can we push the demo to Friday instead of Wednesday?
Priya: Friday works for me, I'll update the client.
Rahul: I'll also ask the vendor for a discount given the delay.
Asha: Good. Someone needs to update the risk log too.
Priya: I'll do the risk log by tomorrow morning."""

inputs = tokenizer(transcript, return_tensors="pt", truncation=True)
ids = model.generate(**inputs, max_length=40, min_length=10, num_beams=4)
print(tokenizer.decode(ids[0], skip_special_tokens=True))
Output
the vendor says the new parts will arrive Thursday, three days late. Asha: Okay. can we push the demo to Friday instead of Wednesday?

Line by line

The transcript uses Speaker: text formatting. This is the simplest way to hand a model turn-by-turn dialogue as plain text. No special dialogue format is required by t5-small. It only sees a longer block of text with colons in it.

Look closely at what the output actually is. It is close to a direct extract of two lines, not a true summary of decisions. It even kept the raw label "Asha:" inside the generated text. This is not a bug in the code. It is a real limitation of a general-purpose summariser on dialogue.

Two of the three action items vanished. Rahul's vendor discount and Priya's risk log commitment never made it into the output at all. The model had no training signal telling it "action items matter most". It fell back on generic sentence importance instead.

Common mistakes

Treating a general summariser as dialogue-ready out of the box. As shown above, it is not. Production systems use models fine-tuned specifically on meeting or chat transcripts, commonly trained on datasets like SAMSum or AMI. That fix is covered in fine-tuning your own summariser.

Losing speaker attribution during preprocessing. If speaker labels get stripped before summarisation, "I'll do it" becomes unattributable. Keep speaker tags through every processing step, and only remove them from the final displayed summary if needed.

Feeding in raw, unedited speech-to-text. Spoken transcripts are full of "um", false starts and repeated words. Cleaning this first, covered in punctuating a speech transcript, usually improves summary quality more than any decoding parameter would.

Assuming silence means agreement. A meeting summary that lists "everyone agreed" because nobody objected on the transcript can be badly wrong. Silence in a transcript is not a labelled signal a model can safely lean on.

Try it yourself

Strip the speaker labels entirely, so the model sees only plain sentences with no Name: prefixes, and rerun. Compare whether losing "who said what" makes the output better or worse.

What to learn next

Researcher — Mathematics and papers.

Why dialogue breaks single-author assumptions

Standard abstractive summarisation assumes text x was produced by one coherent authorial process. Dialogue x = (u_1, s_1), ..., (u_n, s_n) is instead a sequence of utterances u_i. Each is attributed to a speaker s_i from a small set. Its meaning depends on cross-turn reference resolution and topic reintroduction.

This has two direct consequences for modelling. First, speaker-aware encoding matters. Naive concatenation of utterances loses turn boundaries unless explicitly marked — exactly the failure visible in the developer block's output. Second, the target summary is structurally different from a news summary. It typically enumerates decisions and action items, each with an owner, rather than following the source's narrative order.

Datasets and their structure

SAMSum (Gliwa et al., 2019) pairs short, chat-style dialogues with abstractive summaries, crowd-sourced specifically to resemble messaging-app conversations. AMI (Carletta et al., 2005) is the standard meeting corpus. It has real multi-party meetings with audio and transcripts, annotated with both abstractive summaries and extractive action items.

One quirk of this domain: AMI reference summaries often name "the group" as an implicit agent, not any individual speaker. That complicates attribution-based evaluation.

Modelling approaches

Speaker-tagged input. Fix the failure shown earlier with a consistent speaker token on every turn, such as <spk1> or <spk2>. The model then learns speaker identity as a first-class signal, distinct from arbitrary names that vary between transcripts.

Hierarchical encoders. HMNet (Zhu et al., 2020) and related architectures encode each turn locally, then add a second layer over turn-level representations. This mirrors the meeting's own two-level structure: turns within topics.

Extractive action-item detection as a subtask. Some production systems skip one abstractive pass entirely. A dedicated classifier scores each utterance for "is this an action item" first. A natural-language summary is then generated from only the utterances marked positive, trading end-to-end simplicity for controllability.

Evaluation

ROUGE against a reference summary applies as usual. But it correlates poorly with what actually matters to a meeting summary's user: did it correctly capture who owns what. Task-specific metrics compare extracted (action, owner) pairs directly against gold annotations, using set-based precision and recall rather than n-gram overlap.

Key references

  • Carletta, J. et al. (2005). The AMI Meeting Corpus. Machine Learning for Multimodal Interaction.
  • Gliwa, B. et al. (2019). SAMSum Corpus. arXiv:1911.12237
  • Zhu, C. et al. (2020). A Hierarchical Network for Abstractive Meeting Summarization. arXiv:2004.02016

Current state and open problems

Prompt an LLM with the full transcript. Ask it to extract decisions and action items, each with an owner. That now beats most fine-tuned dialogue summarisers on standard benchmarks, provided the transcript fits the context window.

The open problem is speaker diarisation error propagation. A meeting summariser often sits downstream of automatic speech recognition and speaker diarisation, which decides who spoke when. An upstream error there silently corrupts attribution. "Priya will update the client" becomes "Asha will update the client". Nothing in the text alone signals that anything went wrong.

What to learn next