Agents and perception · Medium · 20 min

Whisper in your browser: from audio to minutes

Record a clip or use our sample, transcribe it with Whisper without the audio leaving your tab, and turn its segments into minutes with decisions and tasks.

ASR is what whisper-tiny returned for a short meeting: { name, duration, language, segments }, each segment { start, end, text } in seconds. It is our sample, or your own recording if you transcribe it in "A model in your browser". Turn it into minutes with three functions:

fmtTime(sec): whole seconds (round down) as mm:ss; from one hour, h:mm:ss. 65.9 is "01:05" and 3725 is "1:02:05".

sentences(segments): Whisper cuts at pauses, not at sentences. Join consecutive segments until one ends in ., ?, ! or …, with one space between them; the sentence starts at the first one's start and ends at the last one's end. Skip segments with no text, leave one space between words, trim the ends and turn a word repeated straight away ("el el") into one. Whatever is left at the end without a full stop is a sentence too. Do not change the input segments.

acta(segments) (Spanish for "minutes"): { duracion, lineas, acuerdos, tareas } (length, lines, decisions, tasks). duracion is fmtTime of the highest end ("00:00" with no segments). lineas has one line per sentence: "[mm:ss] sentence". A sentence goes into acuerdos when its text, in lower case and without accents, contains one of the ACUERDO keywords, and into tareas when it contains one of TAREA; it can go into both.

Challenges 0/4

  • Minutes and seconds, and hours from the first one
  • Sentences: join Whisper's pieces, skip the empty ones and drop the repeated word
  • The minutes: one line per sentence with its time, decisions and tasks regardless of accents and case
  • With the model's transcript (your recording) or the sample: every sentence on its line, with its time

function fmtTime(sec) {
  // whole seconds as mm:ss, or h:mm:ss from one hour
  return '';
}

function sentences(segments) {
  // join consecutive segments until one ends in . ? ! or ...; skip empty ones; keep the first start and the last end
  // collapse spaces, trim, and turn a word repeated straight away (el el) into one
  return [];
}

function acta(segments) {
  // { duracion, lineas: ['[mm:ss] sentence'], acuerdos, tareas }: a sentence goes in a list when its normalised text contains a keyword
  return { duracion: '', lineas: [], acuerdos: [], tareas: [] };
}

console.log(acta(ASR.segments));

Tab indents, Ctrl+Z undoes. To leave the editor with the keyboard: Esc, then Tab.

Console output appears here (console.log).

Why this matters

Transcribing a meeting with a cloud service means sending a third party the voices and what was said. Whisper runs on your machine and hands you segments with timestamps, and the real work starts there: joining split segments, dropping the empty ones and the looping repetitions the model produces on silence, and pulling decisions and tasks out of that text. That is what you write here. The full setup outside the browser, with larger models, is in private meeting notes with Whisper on your own hardware.

The idea

Whisper does not return sentences or minutes: it returns pieces of text between two timestamps, cut where the speaker pauses. In our sample, "Acordamos cambiar de proveedor a partir de diciembre" (we agreed to change supplier from December) arrives split over two segments, and the last sentence over two more. The first job is to put the sentences back together without losing when each one started, because that time is what lets you check the minutes against the audio.

The smallest model makes mistakes, and it is worth seeing them before trusting it. whisper-tiny heard "Cordamos" instead of "Acordamos", "marcas en carga" instead of "Marta se encarga" (Marta is in charge) and "el el 9" instead of "el lunes a las nueve" (Monday at nine). The first mistake costs your minutes their main decision, because the keyword "acord" is no longer there. Keywords are a net so nothing gets lost, not a judge: someone who was in the meeting reads the minutes before they go out. A bigger model, such as small or medium on your own machine, makes far fewer of these mistakes.

What we measured on 2026-11-08 in headless Chromium on the DGX Spark (ONNX Runtime Web, WebAssembly, one thread): for the 16.8 s clip, 0.2 s of spectrogram, 0.8 s of encoder and 1 s to decode 84 tokens. It detected Spanish without being told. A typical laptop will be slower; the model takes at most 30 s per pass, so the recording stops there.

Everything happens in your tab. Pressing "Load the model" downloads about 56 MB from this same site (41 MB of Whisper plus the 14 MB ONNX engine that every playground model shares), and the microphone opens only if you press "Record". The audio never leaves your browser: for minutes of meetings with other people's data, that is the difference between one more outside service and none.

Run it on your machine

With Python and faster-whisper (MIT), the same tiny model in 8 bits on our clip (CC BY 4.0). Swap tiny for small or medium and compare the mistakes:

pip install faster-whisper
curl -O https://vorluxai.com/data/labs/reunion-taller.mp3
python3 -c "
from faster_whisper import WhisperModel
m = WhisperModel('tiny', device='cpu', compute_type='int8')
segments, info = m.transcribe('reunion-taller.mp3')
print('idioma:', info.language)
for s in segments:
    print(f'[{int(s.start) // 60:02d}:{int(s.start) % 60:02d}]', s.text.strip())
"

Check what you learned

1. Why join segments before looking for decisions?
2. whisper-tiny wrote "Cordamos" and the minutes lost the decision. What do you do?
3. Why keep the time on every line of the minutes?
4. When you transcribe your recording here, what leaves your computer?

Go deeper: the related guide →

This in production, with your data? Let's talk for 15 minutes →