mirror of
https://github.com/patw/AudioSumma.git
synced 2026-10-18 23:08:08 +00:00
Improve UI, processing pipeline, and test coverage
Replace FFmpeg silence removal with dependency-free PCM processing, add in-app activity and status indicators, extract pipeline orchestration, prevent duplicate daily summaries, remove obsolete image assets, and add audio, pipeline, and Qt tests.
This commit is contained in:
parent
8d0724e356
commit
9cf3f879d0
13 changed files with 741 additions and 77 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -9,7 +9,6 @@ __pycache__/
|
|||
# C extensions
|
||||
*.so
|
||||
|
||||
CLAUDE.md
|
||||
|
||||
# Config and history files
|
||||
*.json
|
||||
|
|
@ -18,7 +17,6 @@ CLAUDE.md
|
|||
*.wav
|
||||
*.tns
|
||||
|
||||
# Ignore my ffmpeg
|
||||
*.exe
|
||||
|
||||
# Ignore my summary files
|
||||
|
|
@ -145,4 +143,3 @@ dmypy.json
|
|||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
.aider*
|
||||
|
|
|
|||
|
|
@ -2,17 +2,17 @@
|
|||
|
||||
Record your local audio and summarize it with whisper.cpp and llama.cpp! Open source, local on-prem transcription and summarization!
|
||||
|
||||

|
||||
|
||||
## Installation
|
||||
|
||||
### macOS Requirements
|
||||
For macOS users, you'll need to have [Homebrew](https://brew.sh/) installed first. Then install the required dependencies:
|
||||
For macOS users, you'll need to have [Homebrew](https://brew.sh/) installed first. Then install PortAudio:
|
||||
|
||||
```bash
|
||||
brew install ffmpeg portaudio
|
||||
brew install portaudio
|
||||
```
|
||||
|
||||
Silence removal is implemented directly against the recorded PCM WAV data using Python's standard library; FFmpeg is not required.
|
||||
|
||||
### All Platforms
|
||||
```
|
||||
pip install -r requirements.txt
|
||||
|
|
|
|||
152
SPEC.md
Normal file
152
SPEC.md
Normal file
|
|
@ -0,0 +1,152 @@
|
|||
# AudioSumma Specification
|
||||
|
||||
## Purpose
|
||||
|
||||
AudioSumma is a small desktop meeting recorder. It records microphone audio locally, removes extended quiet sections, sends the resulting WAV files to a locally hosted Whisper-compatible transcription service, and summarizes the resulting transcripts through a local or hosted OpenAI-compatible language-model service.
|
||||
|
||||
The application is MIT-licensed and does not require FFmpeg.
|
||||
|
||||
## Runtime and dependencies
|
||||
|
||||
- Python 3.13 or newer
|
||||
- PyQt5 for the desktop UI
|
||||
- PyAudio for microphone capture
|
||||
- `requests` for the Whisper-compatible HTTP request
|
||||
- `openai` for the OpenAI-compatible chat-completions request
|
||||
- Python standard library for WAV handling, silence removal, threading, persistence, and file management
|
||||
- PortAudio is required by PyAudio on macOS; install it with Homebrew when needed
|
||||
|
||||
## Entry point and modules
|
||||
|
||||
- `meetings.py` contains the PyQt application, recording controls, in-app activity log, service calls, pipeline orchestration, configuration dialog, and cleanup behavior.
|
||||
- `audio_processing.py` contains the dependency-free PCM WAV silence-removal implementation.
|
||||
- `pipeline.py` contains the UI-independent transcription, summarization, and cleanup orchestration.
|
||||
- `test_audio_processing.py`, `test_pipeline.py`, and `test_ui.py` contain standard-library `unittest` coverage for audio processing, pipeline behavior, and focused Qt state behavior.
|
||||
- `meetings.sh` and `meetings.bat` launch the application.
|
||||
|
||||
## Activity display
|
||||
|
||||
Routine operational messages are shown in the read-only **Activity** panel above the controls. Messages from recording, transcription, summarization, cleanup, configuration migration, and errors are delivered through a Qt signal so worker threads do not update widgets directly. Starting the processing pipeline changes the status indicator to 🟡 and disables the Transcribe button; completion or failure returns it to 🟢 and re-enables the button. The application does not write routine progress messages to the terminal.
|
||||
|
||||
## Recording indicator
|
||||
|
||||
The application state is shown with a large Unicode emoji indicator rather than image assets: 🟢 means idle, 🔴 means recording, and 🟡 means transcription/summarization is processing. The indicator uses the Apple Color Emoji font at 63pt on macOS, is left-aligned in the device row, includes a tooltip, and remains text-based so it scales with the UI without maintaining separate image files. The macOS dock/taskbar icon is rendered at startup from the 🎧 emoji into a Qt icon, avoiding a checked-in image asset while still providing a native icon.
|
||||
|
||||
## User workflow
|
||||
|
||||
1. Select an available input device.
|
||||
2. Click **Record** and provide a filename.
|
||||
3. AudioSumma records until **Stop Recording** is clicked.
|
||||
4. Click **Transcribe** to process pending WAV files and summarize their transcripts.
|
||||
5. Click **Clean** to remove WAV and transcript files while retaining summary Markdown files.
|
||||
|
||||
## Recording format
|
||||
|
||||
Recordings are written as WAV files under `OUTPUT_DIR` with:
|
||||
|
||||
- 16,000 Hz sample rate
|
||||
- One channel
|
||||
- Signed 16-bit PCM samples
|
||||
- `.wav` extension
|
||||
|
||||
This format is selected for compatibility with the Whisper service and for straightforward in-process signal analysis.
|
||||
|
||||
## Silence removal
|
||||
|
||||
Before transcription, each untranscribed WAV is processed by `audio_processing.remove_silence()`.
|
||||
|
||||
The detector:
|
||||
|
||||
- Reads the PCM samples with Python's `wave` and `array` standard-library modules.
|
||||
- Requires mono, 16-bit PCM input.
|
||||
- Divides the signal into approximately 25 ms windows.
|
||||
- Computes RMS amplitude for each window.
|
||||
- Classifies a window as quiet when its RMS amplitude is below the configured threshold.
|
||||
- Uses a default threshold of `-40 dBFS`.
|
||||
- Removes quiet runs lasting at least one second.
|
||||
- Retains 150 ms of padding around retained audio to reduce the chance of cutting off words.
|
||||
- Preserves shorter pauses inside speech.
|
||||
- Writes through a temporary file and uses an atomic replacement for in-place processing.
|
||||
- Leaves an all-quiet recording unchanged rather than producing an empty or malformed WAV.
|
||||
|
||||
The current pipeline modifies the original WAV in place because these files are intermediate inputs to transcription. If retaining original recordings becomes a requirement, the pipeline should write a separate trimmed copy instead.
|
||||
|
||||
## Transcription pipeline
|
||||
|
||||
`ProcessingPipeline.process_wav_files()` scans `OUTPUT_DIR` for `.wav` files. A file is skipped when a same-stem `.tns` file already exists.
|
||||
|
||||
For each pending recording:
|
||||
|
||||
1. Remove extended silence in place.
|
||||
2. Open the WAV in binary mode.
|
||||
3. POST it to `WHISPERCPP_URL` using the multipart field `file`.
|
||||
4. Request JSON output and write the returned `text` field to a same-stem `.tns` file.
|
||||
|
||||
The expected Whisper endpoint is compatible with whisper.cpp's `/inference` endpoint, although the URL is configurable.
|
||||
|
||||
## Summarization pipeline
|
||||
|
||||
After transcription, `ProcessingPipeline.summarize_transcripts()` scans `OUTPUT_DIR` for `.tns` files and appends results to `summary-YYYYMMDD.md`. Before processing each transcript, it checks for the existing `# Call Transcript - <filename>` heading in that day's summary and skips transcripts already summarized that day. This keeps repeated runs idempotent without requiring a database or separate state file.
|
||||
|
||||
Each configured summarization step has a name and prompt template. The transcript is substituted into the template using `{chunk}`. The default steps are:
|
||||
|
||||
- Summary
|
||||
- Facts
|
||||
- Sentiment
|
||||
|
||||
Requests are sent through the `openai` Python client to `LLAMACPP_URL`, which may point to llama.cpp, Ollama, or another OpenAI-compatible service. The configured model, API key, system message, and prompts are persisted through Qt `QSettings`.
|
||||
|
||||
## Configuration
|
||||
|
||||
Default settings include:
|
||||
|
||||
| Setting | Purpose |
|
||||
|---|---|
|
||||
| `OUTPUT_DIR` | Directory containing recordings, transcripts, and summaries |
|
||||
| `WHISPERCPP_URL` | Whisper-compatible transcription endpoint |
|
||||
| `LLAMACPP_URL` | OpenAI-compatible LLM base URL |
|
||||
| `LLM_API_KEY` | API key or placeholder for the LLM service |
|
||||
| `LLM_MODEL` | Model name sent to the LLM service |
|
||||
| `SYSTEM_MESSAGE` | System message for summarization requests |
|
||||
| `SUMMARIZATION_STEPS` | Ordered list of named prompt templates |
|
||||
|
||||
The settings dialog edits these values. `SUMMARIZATION_STEPS` is serialized as JSON inside `QSettings`.
|
||||
|
||||
## File lifecycle
|
||||
|
||||
For a recording named `meeting.wav`, the pipeline creates:
|
||||
|
||||
- `meeting.wav` — recorded and then silence-trimmed audio
|
||||
- `meeting.tns` — plain-text transcription
|
||||
- `summary-YYYYMMDD.md` — accumulated summaries for the day
|
||||
|
||||
The **Clean** action deletes `.wav` and `.tns` files from `OUTPUT_DIR`; it does not delete Markdown summaries.
|
||||
|
||||
## Testing
|
||||
|
||||
Run the full test suite with:
|
||||
|
||||
```bash
|
||||
./.venv/bin/python -m unittest -v
|
||||
```
|
||||
|
||||
The tests verify that:
|
||||
|
||||
The test suite also verifies pipeline file selection, mocked transcription and summarization, cleanup behavior, processing completion, and the green/yellow/red UI state transitions.
|
||||
|
||||
|
||||
- Long quiet sections are removed.
|
||||
- Short pauses are preserved.
|
||||
- In-place processing works.
|
||||
- WAV channel count, sample width, and sample rate are preserved.
|
||||
|
||||
## Non-goals
|
||||
|
||||
The current application does not attempt to:
|
||||
|
||||
- Perform speaker diarization.
|
||||
- Detect speech using a neural voice-activity model.
|
||||
- Preserve the untrimmed source WAV separately.
|
||||
- Provide an embedded Whisper or LLM runtime.
|
||||
- Process arbitrary WAV formats such as stereo or floating-point PCM.
|
||||
- Depend on FFmpeg or another external media-processing executable.
|
||||
134
audio_processing.py
Normal file
134
audio_processing.py
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
"""Small, dependency-free audio processing helpers for AudioSumma."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import os
|
||||
import tempfile
|
||||
import wave
|
||||
from array import array
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _rms(samples: array) -> float:
|
||||
"""Return the RMS amplitude of signed 16-bit samples."""
|
||||
if not samples:
|
||||
return 0.0
|
||||
return math.sqrt(sum(sample * sample for sample in samples) / len(samples))
|
||||
|
||||
|
||||
def remove_silence(
|
||||
input_path: str | os.PathLike[str],
|
||||
output_path: str | os.PathLike[str] | None = None,
|
||||
*,
|
||||
threshold_db: float = -40.0,
|
||||
min_silence_seconds: float = 1.0,
|
||||
padding_seconds: float = 0.15,
|
||||
) -> Path:
|
||||
"""Copy a mono, 16-bit PCM WAV while omitting extended quiet regions.
|
||||
|
||||
Audio is classified in 25 ms RMS windows. Quiet runs shorter than
|
||||
``min_silence_seconds`` remain part of the surrounding audio, while longer
|
||||
runs are removed. Padding helps preserve word beginnings and endings.
|
||||
|
||||
If output_path is omitted, the input is replaced atomically. The returned
|
||||
path is the resulting WAV path.
|
||||
"""
|
||||
input_path = Path(input_path)
|
||||
output_path = input_path if output_path is None else Path(output_path)
|
||||
|
||||
if not math.isfinite(threshold_db):
|
||||
raise ValueError("threshold_db must be finite")
|
||||
if min_silence_seconds < 0 or padding_seconds < 0:
|
||||
raise ValueError("silence duration and padding must not be negative")
|
||||
|
||||
with wave.open(str(input_path), "rb") as source:
|
||||
channels = source.getnchannels()
|
||||
sample_width = source.getsampwidth()
|
||||
sample_rate = source.getframerate()
|
||||
frames = source.readframes(source.getnframes())
|
||||
|
||||
if channels != 1 or sample_width != 2:
|
||||
raise ValueError("silence removal requires mono, 16-bit PCM WAV")
|
||||
if sample_rate <= 0:
|
||||
raise ValueError("WAV has an invalid sample rate")
|
||||
|
||||
samples = array("h")
|
||||
samples.frombytes(frames)
|
||||
if os.sys.byteorder != "little":
|
||||
samples.byteswap()
|
||||
|
||||
window_size = max(1, round(sample_rate * 0.025))
|
||||
min_quiet_windows = max(
|
||||
1, math.ceil(min_silence_seconds * sample_rate / window_size)
|
||||
) if min_silence_seconds else 1
|
||||
padding_samples = round(padding_seconds * sample_rate)
|
||||
threshold = 32767.0 * (10.0 ** (threshold_db / 20.0))
|
||||
|
||||
loud = [
|
||||
_rms(samples[start : start + window_size]) >= threshold
|
||||
for start in range(0, len(samples), window_size)
|
||||
]
|
||||
|
||||
regions: list[tuple[int, int]] = []
|
||||
speech_start: int | None = None
|
||||
quiet_start: int | None = None
|
||||
|
||||
for index, is_loud in enumerate(loud):
|
||||
start = index * window_size
|
||||
end = min(start + window_size, len(samples))
|
||||
if is_loud:
|
||||
if speech_start is None:
|
||||
speech_start = start
|
||||
quiet_start = None
|
||||
elif speech_start is not None:
|
||||
quiet_start = start if quiet_start is None else quiet_start
|
||||
if end - quiet_start >= min_silence_seconds * sample_rate:
|
||||
regions.append((
|
||||
max(0, speech_start - padding_samples),
|
||||
min(len(samples), quiet_start + padding_samples),
|
||||
))
|
||||
speech_start = None
|
||||
quiet_start = None
|
||||
|
||||
if speech_start is not None:
|
||||
regions.append((max(0, speech_start - padding_samples), len(samples)))
|
||||
|
||||
if not regions:
|
||||
# Avoid turning an all-silence recording into a malformed/empty file.
|
||||
return input_path
|
||||
|
||||
merged: list[tuple[int, int]] = []
|
||||
for start, end in regions:
|
||||
if merged and start <= merged[-1][1]:
|
||||
merged[-1] = (merged[-1][0], max(merged[-1][1], end))
|
||||
else:
|
||||
merged.append((start, end))
|
||||
|
||||
trimmed = array("h")
|
||||
for start, end in merged:
|
||||
trimmed.extend(samples[start:end])
|
||||
if os.sys.byteorder != "little":
|
||||
trimmed.byteswap()
|
||||
audio_bytes = trimmed.tobytes()
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, temporary_name = tempfile.mkstemp(
|
||||
prefix=f".{output_path.name}.", suffix=".tmp", dir=output_path.parent
|
||||
)
|
||||
os.close(fd)
|
||||
try:
|
||||
with wave.open(temporary_name, "wb") as destination:
|
||||
destination.setnchannels(channels)
|
||||
destination.setsampwidth(sample_width)
|
||||
destination.setframerate(sample_rate)
|
||||
destination.writeframes(audio_bytes)
|
||||
os.replace(temporary_name, output_path)
|
||||
except Exception:
|
||||
try:
|
||||
os.unlink(temporary_name)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
raise
|
||||
|
||||
return output_path
|
||||
BIN
headphones.png
BIN
headphones.png
Binary file not shown.
|
Before Width: | Height: | Size: 3.4 KiB |
167
meetings.py
167
meetings.py
|
|
@ -4,24 +4,40 @@ import pyaudio
|
|||
import wave
|
||||
import threading
|
||||
import requests
|
||||
import datetime
|
||||
import tempfile
|
||||
import subprocess
|
||||
import json
|
||||
from openai import OpenAI
|
||||
from PyQt5.QtWidgets import (QApplication, QWidget, QPushButton, QVBoxLayout,
|
||||
QHBoxLayout, QInputDialog, QLabel, QComboBox,
|
||||
QDialog, QFormLayout, QLineEdit,
|
||||
QTextEdit, QTabWidget)
|
||||
from PyQt5.QtCore import QSettings
|
||||
from PyQt5.QtGui import QIcon, QPixmap
|
||||
QTextEdit, QPlainTextEdit, QTabWidget)
|
||||
from PyQt5.QtCore import QSettings, Qt, pyqtSignal
|
||||
from PyQt5.QtGui import QColor, QFont, QIcon, QPainter, QPixmap
|
||||
from audio_processing import remove_silence
|
||||
from pipeline import ProcessingPipeline
|
||||
|
||||
|
||||
def create_app_icon():
|
||||
"""Render the app's headphone emoji as a real macOS dock/taskbar icon."""
|
||||
canvas = QPixmap(128, 128)
|
||||
canvas.fill(QColor("transparent"))
|
||||
painter = QPainter(canvas)
|
||||
painter.setRenderHint(QPainter.Antialiasing)
|
||||
painter.setFont(QFont("Apple Color Emoji", 92))
|
||||
painter.drawText(canvas.rect(), Qt.AlignCenter, "🎧")
|
||||
painter.end()
|
||||
return QIcon(canvas)
|
||||
|
||||
|
||||
class RecordingApp(QWidget):
|
||||
activity_message = pyqtSignal(str)
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.recording_pixmap = QPixmap("recording.png")
|
||||
self.not_recording_pixmap = QPixmap("notrecording.png")
|
||||
self._activity_ready = False
|
||||
self._pending_activity = []
|
||||
self.recording_indicator = "🔴"
|
||||
self.processing_indicator = "🟡"
|
||||
self.not_recording_indicator = "🟢"
|
||||
self.is_recording = False
|
||||
self.audio_thread = None
|
||||
self.stream = None
|
||||
|
|
@ -45,15 +61,54 @@ class RecordingApp(QWidget):
|
|||
self.load_config()
|
||||
self.initUI()
|
||||
|
||||
def log_activity(self, message):
|
||||
"""Send activity text safely from both the UI and worker threads."""
|
||||
if self._activity_ready:
|
||||
self.activity_message.emit(message)
|
||||
else:
|
||||
self._pending_activity.append(message)
|
||||
|
||||
def append_activity(self, message):
|
||||
if message == "__PROCESSING_FINISHED__":
|
||||
self.set_processing_indicator(False)
|
||||
self.transcribe_button.setEnabled(True)
|
||||
return
|
||||
self.activity_log.appendPlainText(message)
|
||||
|
||||
def set_processing_indicator(self, processing):
|
||||
if processing:
|
||||
self.status_label.setText(self.processing_indicator)
|
||||
self.status_label.setToolTip("Processing")
|
||||
else:
|
||||
self.status_label.setText(self.not_recording_indicator)
|
||||
self.status_label.setToolTip("Not recording")
|
||||
|
||||
def initUI(self):
|
||||
self.setWindowTitle('Meeting Recorder')
|
||||
self.setGeometry(500, 500, 500, 150)
|
||||
|
||||
main_layout = QVBoxLayout()
|
||||
|
||||
activity_label = QLabel("Activity")
|
||||
main_layout.addWidget(activity_label)
|
||||
self.activity_log = QPlainTextEdit(self)
|
||||
self.activity_log.setReadOnly(True)
|
||||
self.activity_log.setMaximumBlockCount(1000)
|
||||
self.activity_log.setMinimumHeight(180)
|
||||
main_layout.addWidget(self.activity_log)
|
||||
self.activity_message.connect(self.append_activity)
|
||||
self._activity_ready = True
|
||||
for message in self._pending_activity:
|
||||
self.append_activity(message)
|
||||
self._pending_activity.clear()
|
||||
|
||||
top_layout = QHBoxLayout()
|
||||
self.status_label = QLabel(self)
|
||||
self.status_label.setPixmap(self.not_recording_pixmap)
|
||||
self.status_label.setText(self.not_recording_indicator)
|
||||
self.status_label.setFont(QFont("Apple Color Emoji", 63))
|
||||
self.status_label.setAlignment(Qt.AlignLeft | Qt.AlignVCenter)
|
||||
self.status_label.setMinimumWidth(108)
|
||||
self.status_label.setToolTip("Not recording")
|
||||
top_layout.addWidget(self.status_label)
|
||||
self.device_combo = QComboBox(self)
|
||||
self.device_combo.addItems(self.get_device_names())
|
||||
|
|
@ -100,7 +155,8 @@ class RecordingApp(QWidget):
|
|||
if ok and filename:
|
||||
self.is_recording = True
|
||||
self.record_button.setText('Stop Recording')
|
||||
self.status_label.setPixmap(self.recording_pixmap)
|
||||
self.status_label.setText(self.recording_indicator)
|
||||
self.status_label.setToolTip("Recording")
|
||||
selected_device_index = self.device_combo.currentIndex()
|
||||
self.audio_thread = threading.Thread(target=self.record_audio, args=(filename, selected_device_index))
|
||||
self.audio_thread.start()
|
||||
|
|
@ -109,7 +165,8 @@ class RecordingApp(QWidget):
|
|||
if self.is_recording:
|
||||
self.is_recording = False
|
||||
self.record_button.setText('Record')
|
||||
self.status_label.setPixmap(self.not_recording_pixmap)
|
||||
self.status_label.setText(self.not_recording_indicator)
|
||||
self.status_label.setToolTip("Not recording")
|
||||
if self.audio_thread:
|
||||
self.audio_thread.join()
|
||||
if self.stream:
|
||||
|
|
@ -143,14 +200,19 @@ class RecordingApp(QWidget):
|
|||
input_device_index=device_index
|
||||
)
|
||||
|
||||
print(f"Recording to {file_path}...")
|
||||
self.log_activity(f"Recording to {file_path}...")
|
||||
while self.is_recording:
|
||||
data = self.stream.read(chunk_size)
|
||||
self.wf.writeframes(data)
|
||||
print(f"Audio saved to {file_path}")
|
||||
self.log_activity(f"Audio saved to {file_path}")
|
||||
|
||||
def transcribe(self):
|
||||
threading.Thread(target=self.run_pipeline).start()
|
||||
if self.is_recording:
|
||||
self.log_activity("Stop recording before starting transcription")
|
||||
return
|
||||
self.set_processing_indicator(True)
|
||||
self.transcribe_button.setEnabled(False)
|
||||
threading.Thread(target=self.run_pipeline, daemon=True).start()
|
||||
|
||||
def whisper_api(self, file):
|
||||
response = requests.post(
|
||||
|
|
@ -178,64 +240,29 @@ class RecordingApp(QWidget):
|
|||
return response.choices[0].message.content
|
||||
|
||||
def trim_silence(self, filename):
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp:
|
||||
tmp_path = tmp.name
|
||||
subprocess.run(
|
||||
["ffmpeg", "-i", filename,
|
||||
"-af", "silenceremove=stop_threshold=-40dB:stop_duration=1:stop_periods=-1",
|
||||
"-y", tmp_path], # -y overwrites without prompting
|
||||
capture_output=True, text=True, check=True
|
||||
remove_silence(filename)
|
||||
|
||||
def processing_pipeline(self):
|
||||
return ProcessingPipeline(
|
||||
self.config,
|
||||
log=self.log_activity,
|
||||
trim_silence=self.trim_silence,
|
||||
transcribe=self.whisper_api,
|
||||
summarize=self.llm_local,
|
||||
)
|
||||
os.replace(tmp_path, filename)
|
||||
|
||||
def process_wav_files(self):
|
||||
output_dir = self.config["OUTPUT_DIR"]
|
||||
for wav_file in [f for f in os.listdir(output_dir) if f.endswith(".wav")]:
|
||||
stem = os.path.splitext(wav_file)[0]
|
||||
transcript_path = os.path.join(output_dir, stem + ".tns")
|
||||
if os.path.exists(transcript_path):
|
||||
print(f"Skipping {wav_file}, transcript already exists")
|
||||
continue
|
||||
wav_path = os.path.join(output_dir, wav_file)
|
||||
print(f"Trimming silence: {wav_path}")
|
||||
self.trim_silence(wav_path)
|
||||
print(f"Transcribing: {wav_path}")
|
||||
with open(wav_path, "rb") as f:
|
||||
with open(transcript_path, "w") as out:
|
||||
out.write(self.whisper_api(f))
|
||||
|
||||
def summarize_transcripts(self):
|
||||
output_dir = self.config["OUTPUT_DIR"]
|
||||
today = datetime.datetime.now().strftime('%Y%m%d')
|
||||
summary_path = os.path.join(output_dir, f"summary-{today}.md")
|
||||
for tns_file in [f for f in os.listdir(output_dir) if f.endswith(".tns")]:
|
||||
full_path = os.path.join(output_dir, tns_file)
|
||||
print(f"Summarizing: {full_path}")
|
||||
transcript = open(full_path).read()
|
||||
with open(summary_path, "a") as md:
|
||||
md.write(f"# Call Transcript - {tns_file}\n\n")
|
||||
for step in self.config["SUMMARIZATION_STEPS"]:
|
||||
print(f" Running step: {step['name']}")
|
||||
result = self.llm_local(step["prompt"].format(chunk=transcript))
|
||||
md.write(f"## {step['name']}\n\n{result}\n\n")
|
||||
md.write("---\n")
|
||||
print("Summarizing complete")
|
||||
|
||||
def run_pipeline(self):
|
||||
self.process_wav_files()
|
||||
self.summarize_transcripts()
|
||||
self.log_activity("Starting transcription and summarization")
|
||||
try:
|
||||
self.processing_pipeline().run()
|
||||
self.log_activity("Processing complete")
|
||||
except Exception as e:
|
||||
self.log_activity(f"Processing failed: {e}")
|
||||
finally:
|
||||
self.activity_message.emit("__PROCESSING_FINISHED__")
|
||||
|
||||
def clean(self):
|
||||
print("Cleaning files...")
|
||||
try:
|
||||
for file in os.listdir(self.config["OUTPUT_DIR"]):
|
||||
if file.endswith(('.wav', '.tns')):
|
||||
full_path = os.path.join(self.config["OUTPUT_DIR"], file)
|
||||
os.remove(full_path)
|
||||
print(f"Deleted: {full_path}")
|
||||
print("Cleaning complete.")
|
||||
except Exception as e:
|
||||
print(f"An error occurred while cleaning files: {e}")
|
||||
self.processing_pipeline().clean()
|
||||
|
||||
def load_config(self):
|
||||
settings = QSettings("AudioSumma", "AudioSumma")
|
||||
|
|
@ -261,9 +288,9 @@ class RecordingApp(QWidget):
|
|||
value = old[key]
|
||||
settings.setValue(key, json.dumps(value) if key == "SUMMARIZATION_STEPS" else value)
|
||||
os.rename("config.json", "config.json.bak")
|
||||
print("Migrated settings from config.json (backed up as config.json.bak)")
|
||||
self.log_activity("Migrated settings from config.json (backed up as config.json.bak)")
|
||||
except Exception as e:
|
||||
print(f"Error migrating config.json: {e}")
|
||||
self.log_activity(f"Error migrating config.json: {e}")
|
||||
|
||||
def save_config(self):
|
||||
settings = QSettings("AudioSumma", "AudioSumma")
|
||||
|
|
@ -379,7 +406,7 @@ class ConfigDialog(QDialog):
|
|||
|
||||
if __name__ == '__main__':
|
||||
app = QApplication(sys.argv)
|
||||
app.setWindowIcon(QIcon("headphones.png"))
|
||||
app.setWindowIcon(create_app_icon())
|
||||
ex = RecordingApp()
|
||||
ex.show()
|
||||
sys.exit(app.exec_())
|
||||
|
|
|
|||
BIN
notrecording.png
BIN
notrecording.png
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 KiB |
95
pipeline.py
Normal file
95
pipeline.py
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
"""Transcription, summarization, and cleanup orchestration for AudioSumma."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import os
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
class ProcessingPipeline:
|
||||
"""Run the file-processing workflow independently of the Qt UI."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: dict[str, Any],
|
||||
*,
|
||||
log: Callable[[str], None] = lambda _message: None,
|
||||
trim_silence: Callable[[str], Any] | None = None,
|
||||
transcribe: Callable[[Any], str] | None = None,
|
||||
summarize: Callable[[str], str] | None = None,
|
||||
) -> None:
|
||||
self.config = config
|
||||
self.log = log
|
||||
self.trim_silence = trim_silence
|
||||
self.transcribe = transcribe
|
||||
self.summarize = summarize
|
||||
|
||||
@property
|
||||
def output_dir(self) -> Path:
|
||||
return Path(self.config["OUTPUT_DIR"])
|
||||
|
||||
def process_wav_files(self) -> None:
|
||||
"""Trim and transcribe WAV files without an existing transcript."""
|
||||
self.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
for wav_path in sorted(self.output_dir.glob("*.wav")):
|
||||
transcript_path = wav_path.with_suffix(".tns")
|
||||
if transcript_path.exists():
|
||||
self.log(f"Skipping {wav_path.name}, transcript already exists")
|
||||
continue
|
||||
if self.trim_silence is None or self.transcribe is None:
|
||||
raise RuntimeError("transcription services are not configured")
|
||||
|
||||
self.log(f"Trimming silence: {wav_path}")
|
||||
self.trim_silence(str(wav_path))
|
||||
self.log(f"Transcribing: {wav_path}")
|
||||
with wav_path.open("rb") as audio, transcript_path.open("w") as output:
|
||||
output.write(self.transcribe(audio))
|
||||
|
||||
def summarize_transcripts(self) -> None:
|
||||
"""Summarize every transcript into today's Markdown summary."""
|
||||
if self.summarize is None:
|
||||
raise RuntimeError("summarization service is not configured")
|
||||
|
||||
summary_path = self.output_dir / (
|
||||
f"summary-{datetime.datetime.now():%Y%m%d}.md"
|
||||
)
|
||||
existing_summary = summary_path.read_text() if summary_path.exists() else ""
|
||||
for transcript_path in sorted(self.output_dir.glob("*.tns")):
|
||||
heading = f"# Call Transcript - {transcript_path.name}\n"
|
||||
if heading in existing_summary:
|
||||
self.log(
|
||||
f"Skipping {transcript_path.name}, already summarized today"
|
||||
)
|
||||
continue
|
||||
|
||||
self.log(f"Summarizing: {transcript_path}")
|
||||
transcript = transcript_path.read_text()
|
||||
with summary_path.open("a") as summary:
|
||||
summary.write(f"# Call Transcript - {transcript_path.name}\n\n")
|
||||
for step in self.config["SUMMARIZATION_STEPS"]:
|
||||
self.log(f" Running step: {step['name']}")
|
||||
result = self.summarize(step["prompt"].format(chunk=transcript))
|
||||
summary.write(f"## {step['name']}\n\n{result}\n\n")
|
||||
summary.write("---\n")
|
||||
self.log("Summarizing complete")
|
||||
|
||||
def run(self) -> None:
|
||||
self.log("Starting transcription and summarization")
|
||||
self.process_wav_files()
|
||||
self.summarize_transcripts()
|
||||
self.log("Processing complete")
|
||||
|
||||
def clean(self) -> None:
|
||||
"""Delete intermediate WAV and transcript files, retaining Markdown."""
|
||||
self.log("Cleaning files...")
|
||||
try:
|
||||
for path in self.output_dir.iterdir():
|
||||
if path.suffix in {".wav", ".tns"}:
|
||||
path.unlink()
|
||||
self.log(f"Deleted: {path}")
|
||||
self.log("Cleaning complete.")
|
||||
except Exception as error:
|
||||
self.log(f"An error occurred while cleaning files: {error}")
|
||||
BIN
recording.png
BIN
recording.png
Binary file not shown.
|
Before Width: | Height: | Size: 2.6 KiB |
BIN
screenshot.png
BIN
screenshot.png
Binary file not shown.
|
Before Width: | Height: | Size: 7.9 KiB |
101
test_audio_processing.py
Normal file
101
test_audio_processing.py
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
import math
|
||||
import tempfile
|
||||
import unittest
|
||||
import wave
|
||||
from array import array
|
||||
from pathlib import Path
|
||||
|
||||
from audio_processing import remove_silence
|
||||
|
||||
|
||||
SAMPLE_RATE = 16_000
|
||||
|
||||
|
||||
def write_wav(
|
||||
path: Path,
|
||||
segments: list[tuple[float, int]],
|
||||
*,
|
||||
channels: int = 1,
|
||||
sample_width: int = 2,
|
||||
) -> None:
|
||||
samples = array("h")
|
||||
for duration, amplitude in segments:
|
||||
samples.extend([amplitude] * round(duration * SAMPLE_RATE))
|
||||
raw = samples.tobytes()
|
||||
if channels > 1:
|
||||
raw = b"".join(raw[i:i + sample_width] * channels
|
||||
for i in range(0, len(raw), sample_width))
|
||||
with wave.open(str(path), "wb") as wav:
|
||||
wav.setnchannels(channels)
|
||||
wav.setsampwidth(sample_width)
|
||||
wav.setframerate(SAMPLE_RATE)
|
||||
wav.writeframes(raw)
|
||||
|
||||
|
||||
def wav_duration(path: Path) -> float:
|
||||
with wave.open(str(path), "rb") as wav:
|
||||
return wav.getnframes() / wav.getframerate()
|
||||
|
||||
|
||||
class SilenceRemovalTests(unittest.TestCase):
|
||||
def test_removes_long_silence_but_preserves_short_pauses(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "meeting.wav"
|
||||
output = root / "trimmed.wav"
|
||||
write_wav(source, [(1.0, 4_000), (0.25, 0), (1.0, 4_000),
|
||||
(2.0, 0), (1.0, 4_000)])
|
||||
remove_silence(source, output, padding_seconds=0.1)
|
||||
self.assertTrue(math.isclose(wav_duration(output), 3.45, abs_tol=0.08))
|
||||
self.assertTrue(math.isclose(wav_duration(source), 5.25, abs_tol=0.01))
|
||||
|
||||
def test_in_place_removal_preserves_wav_metadata(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "in-place.wav"
|
||||
write_wav(path, [(1.0, 5_000), (1.5, 0), (1.0, 5_000)])
|
||||
remove_silence(path)
|
||||
with wave.open(str(path), "rb") as wav:
|
||||
self.assertEqual(wav.getnchannels(), 1)
|
||||
self.assertEqual(wav.getsampwidth(), 2)
|
||||
self.assertEqual(wav.getframerate(), SAMPLE_RATE)
|
||||
self.assertLess(wav.getnframes(), round(3.5 * SAMPLE_RATE))
|
||||
|
||||
def test_all_silence_is_left_unchanged(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "silent.wav"
|
||||
write_wav(path, [(2.0, 0)])
|
||||
before = path.read_bytes()
|
||||
result = remove_silence(path)
|
||||
self.assertEqual(result, path)
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
|
||||
def test_audio_without_long_silence_is_unchanged(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "speech.wav"
|
||||
write_wav(path, [(1.0, 3_000), (0.2, 0), (1.0, 3_000)])
|
||||
before = path.read_bytes()
|
||||
remove_silence(path)
|
||||
self.assertEqual(path.read_bytes(), before)
|
||||
|
||||
def test_rejects_unsupported_wav_shapes(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
stereo = root / "stereo.wav"
|
||||
write_wav(stereo, [(1.0, 3_000)], channels=2)
|
||||
with self.assertRaises(ValueError):
|
||||
remove_silence(stereo)
|
||||
|
||||
def test_rejects_invalid_parameters(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "speech.wav"
|
||||
write_wav(path, [(1.0, 3_000)])
|
||||
with self.assertRaises(ValueError):
|
||||
remove_silence(path, threshold_db=float("nan"))
|
||||
with self.assertRaises(ValueError):
|
||||
remove_silence(path, padding_seconds=-1)
|
||||
with self.assertRaises(ValueError):
|
||||
remove_silence(path, min_silence_seconds=-1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
104
test_pipeline.py
Normal file
104
test_pipeline.py
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import Mock
|
||||
|
||||
from pipeline import ProcessingPipeline
|
||||
|
||||
|
||||
CONFIG = {
|
||||
"OUTPUT_DIR": "",
|
||||
"SUMMARIZATION_STEPS": [
|
||||
{"name": "Summary", "prompt": "Summarize: {chunk}"},
|
||||
{"name": "Facts", "prompt": "Facts: {chunk}"},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class PipelineTests(unittest.TestCase):
|
||||
def make_pipeline(self, directory: Path, **kwargs):
|
||||
config = {**CONFIG, "OUTPUT_DIR": str(directory)}
|
||||
return ProcessingPipeline(config, **kwargs)
|
||||
|
||||
def test_processes_pending_wav_and_skips_existing_transcript(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
pending = root / "pending.wav"
|
||||
pending.write_bytes(b"wav")
|
||||
skipped = root / "skipped.wav"
|
||||
skipped.write_bytes(b"wav")
|
||||
(root / "skipped.tns").write_text("already done")
|
||||
trim = Mock()
|
||||
transcribe = Mock(return_value="new transcript")
|
||||
logs = []
|
||||
pipeline = self.make_pipeline(root, log=logs.append,
|
||||
trim_silence=trim,
|
||||
transcribe=transcribe)
|
||||
|
||||
pipeline.process_wav_files()
|
||||
|
||||
trim.assert_called_once_with(str(pending))
|
||||
transcribe.assert_called_once()
|
||||
self.assertEqual((root / "pending.tns").read_text(), "new transcript")
|
||||
self.assertTrue(any("Skipping skipped.wav" in log for log in logs))
|
||||
|
||||
def test_summarizes_transcripts_to_dated_markdown(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / "call.tns").write_text("hello")
|
||||
summarize = Mock(side_effect=lambda prompt: f"result:{prompt}")
|
||||
pipeline = self.make_pipeline(root, summarize=summarize)
|
||||
|
||||
pipeline.summarize_transcripts()
|
||||
|
||||
summaries = list(root.glob("summary-*.md"))
|
||||
self.assertEqual(len(summaries), 1)
|
||||
content = summaries[0].read_text()
|
||||
self.assertIn("# Call Transcript - call.tns", content)
|
||||
self.assertIn("result:Summarize: hello", content)
|
||||
self.assertEqual(summarize.call_count, 2)
|
||||
|
||||
def test_does_not_summarize_transcript_twice_in_one_day(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / "call.tns").write_text("hello")
|
||||
summarize = Mock(return_value="summary")
|
||||
logs = []
|
||||
pipeline = self.make_pipeline(root, log=logs.append, summarize=summarize)
|
||||
|
||||
pipeline.summarize_transcripts()
|
||||
pipeline.summarize_transcripts()
|
||||
|
||||
self.assertEqual(summarize.call_count, 2)
|
||||
summary_file = next(root.glob("summary-*.md"))
|
||||
content = summary_file.read_text()
|
||||
self.assertEqual(content.count("# Call Transcript - call.tns"), 1)
|
||||
self.assertTrue(any("already summarized today" in log for log in logs))
|
||||
|
||||
def test_clean_deletes_intermediates_but_keeps_markdown(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
for name in ("call.wav", "call.tns", "summary.md"):
|
||||
(root / name).write_text(name)
|
||||
pipeline = self.make_pipeline(root)
|
||||
|
||||
pipeline.clean()
|
||||
|
||||
self.assertFalse((root / "call.wav").exists())
|
||||
self.assertFalse((root / "call.tns").exists())
|
||||
self.assertTrue((root / "summary.md").exists())
|
||||
|
||||
def test_run_logs_completion(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
logs = []
|
||||
pipeline = self.make_pipeline(
|
||||
Path(directory), log=logs.append,
|
||||
trim_silence=Mock(), transcribe=Mock(return_value="text"),
|
||||
summarize=Mock(return_value="summary"),
|
||||
)
|
||||
pipeline.run()
|
||||
self.assertIn("Processing complete", logs)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
54
test_ui.py
Normal file
54
test_ui.py
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
import os
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
os.environ.setdefault("QT_QPA_PLATFORM", "offscreen")
|
||||
|
||||
from PyQt5.QtWidgets import QApplication
|
||||
|
||||
from meetings import RecordingApp
|
||||
|
||||
|
||||
class RecordingAppUITests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.application = QApplication.instance() or QApplication(sys.argv)
|
||||
|
||||
def setUp(self):
|
||||
self.devices = patch.object(RecordingApp, "get_device_names", return_value=[])
|
||||
self.devices.start()
|
||||
self.app = RecordingApp()
|
||||
|
||||
def tearDown(self):
|
||||
self.app.close()
|
||||
self.app.deleteLater()
|
||||
self.devices.stop()
|
||||
|
||||
def test_indicator_states(self):
|
||||
self.assertEqual(self.app.status_label.text(), "🟢")
|
||||
self.app.status_label.setText(self.app.recording_indicator)
|
||||
self.assertEqual(self.app.status_label.text(), "🔴")
|
||||
self.app.set_processing_indicator(True)
|
||||
self.assertEqual(self.app.status_label.text(), "🟡")
|
||||
self.assertEqual(self.app.status_label.toolTip(), "Processing")
|
||||
self.app.set_processing_indicator(False)
|
||||
self.assertEqual(self.app.status_label.text(), "🟢")
|
||||
|
||||
def test_processing_finished_reenables_transcribe_and_returns_to_idle(self):
|
||||
self.app.set_processing_indicator(True)
|
||||
self.app.transcribe_button.setEnabled(False)
|
||||
self.app.append_activity("__PROCESSING_FINISHED__")
|
||||
self.assertTrue(self.app.transcribe_button.isEnabled())
|
||||
self.assertEqual(self.app.status_label.text(), "🟢")
|
||||
|
||||
def test_transcription_is_blocked_while_recording(self):
|
||||
self.app.is_recording = True
|
||||
self.app.transcribe()
|
||||
self.assertEqual(self.app.status_label.text(), "🟢")
|
||||
self.assertTrue(self.app.transcribe_button.isEnabled())
|
||||
self.assertIn("Stop recording", self.app.activity_log.toPlainText())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue