ClipFlow
AI Silence Remover
ClipFlow is an iOS video editor that automatically detects and removes silent gaps and filler words, then carries the clip through captioning, auto-reframing, and platform-specific export in a single flow. Speech recognition and video processing all run on-device, so nothing is uploaded to a server. It is built on Clean Architecture (Domain / Data / Features / Core) with Swift 6 complete strict concurrency, isolating heavy media processing inside a pipeline of actors.
For YouTubers and short-form creators, cutting silent gaps and removing filler words like 'um' and 'uh' is the most time-consuming, repetitive part of editing each clip. ClipFlow automates this with on-device audio analysis via Accelerate (vDSP) and speech recognition via the Speech framework, drastically cutting editing time.
App preview
What it does
Automatic Silence Detection & Removal
Extracts 16-bit PCM (mono, 44.1kHz) via AVAssetReader and computes RMS per 50ms window with Accelerate's vDSP_rmsqv. The threshold (default -40dB) is converted to linear amplitude; contiguous windows below it are merged by a minimum-duration rule (default 0.5s) into silence regions. Threshold and minimum duration are adjustable via slider with live preview.
Word-Level Filler Word Detection
Recognizes the video via the Speech framework's SFSpeechURLRecognitionRequest (using requiresOnDeviceRecognition where supported), then matches word-segment timestamps from bestTranscription against Korean+English filler sets — hesitations ('um', 'uh', 'ah', '음', '어') and discourse markers ('like', 'you know', 'basically', '그', '약간') — to flag and remove fillers at the word level in one pass.
Audio-Energy Highlight Suggestions
Computes an audio energy envelope with vDSP and combines it with a NaturalLanguage NLTagger score for noun/proper-noun density in the transcript. High-energy, content-dense segments are surfaced as short-form clip candidates fitted to a target length (default 3–15s).
Auto, Word-Level Animated & Bilingual Captions
Converts speech to captions via the Speech framework and renders word-level effects — karaokeHighlight, scalePop, wordReveal, bouncyEmphasis — with CABasicAnimations keyed off each TranscriptWord's timing. When a translation exists, DualSubtitleOverlayBuilder stacks the original above the translation (EN+KR) as CALayers for bilingual subtitles.
Auto Reframe Smart Crop
The AutoReframeDetector actor samples up to 40 frames and locates faces with Vision's VNDetectFaceRectanglesRequest, falling back to VNGenerateObjectnessBasedSaliencyImageRequest saliency when none is found. It averages the per-frame bounding boxes, and ReframingTransformBuilder auto-crops around the subject when exporting to other aspect ratios such as vertical 9:16.
Platform Presets & FCPXML Export
YouTube Shorts, TikTok, Instagram Reels, YouTube, and square presets one-tap the aspect ratio (9:16, 16:9, 1:1), recommended quality (1080p, 4K), and max duration, warning when a clip exceeds the limit. The FCPXMLExporter actor serializes the edit timeline (cuts and speed segments) to Final Cut Pro XML for handoff to external editors.
Mosaic, Chroma Key & Timeline Tools
Region mosaic uses Core Image's CIPixellate/CIGaussianBlur per time range, while chroma key builds a 64³ CIColorCube LUT directly, keying out a target color by HSV hue distance and saturation with premultiplied alpha. Additional tools — segment speed, transitions, picture-in-picture, text overlays, and background audio — are exposed as discrete UseCases.
On-Device Composition & Export Pipeline
The VideoExporter actor assembles cuts, speeds, and tracks with AVMutableComposition, then chooses a CIFilter-backed AVMutableVideoComposition when filters or reframing are needed (or a layerInstruction transform otherwise) and renders via AVAssetExportSession — all on-device with progress callbacks.
How it's built
Language
UI
Media Composition
Image / Rendering
Audio / DSP
On-device AI
Interop
Data & Commerce
A Clean Architecture iOS app split into Domain / Data / Features / Core layers, isolating heavy media, speech, and vision processing inside ten actors and running all processing on-device.
- 1
Clean Architecture separated into Domain, Data, Features, and Core: Domain holds editing UseCases (speed, transition, filter, text, background audio, aspect ratio, undo/redo) and Repository protocols, while Data provides the SwiftData and file-system implementations, decoupling business logic from data access.
- 2
Under Swift 6 complete strict concurrency, ten processors — SilenceDetector, AudioAnalyzer, AutoReframeDetector, FillerWordDetector, HighlightDetector, SpeechRecognitionService, VideoExporter, FCPXMLExporter, VideoThumbnailGenerator, and TranscriptGenerator — are implemented as actors for compile-time concurrency safety.
- 3
Non-Sendable types are contained within actor boundaries by creating AVURLAsset inside the actor: AutoReframeDetector takes a URL, builds the asset and image generator internally, and bridges frame capture and Vision requests into async via withCheckedContinuation.
- 4
Fully on-device processing: speech recognition, audio DSP, video composition, and face/saliency detection all run locally with no server uploads, preserving privacy.
- 5
Audio analysis pipeline: extracts 16-bit PCM via AVAssetReader and computes RMS/energy envelopes with vDSP to derive silence, filler, and highlight regions with progress callbacks, combining text density via NLTagger.
- 6
Export composition branching: VideoExporter assembles the timeline with AVMutableComposition, then selects a CIFilter-backed AVMutableVideoComposition when filters/reframing are present, or a layerInstruction transform composition otherwise, rendering via AVAssetExportSession.
- 7
MVVM feature composition with persistence: eight per-screen ViewModels with @Observable state, projects and edit state stored via SwiftData @Model and repositories, and subscriptions handled by StoreKit 2 (Product.products, Transaction.currentEntitlements, and a Transaction.updates listener).
328
Tests
151
Swift files
10
Processor actors
iOS 17
Min iOS