Lumora
AI Beauty DNA · Y2K Filters
Lumora is an iOS AI camera app that pairs a one-selfie 'Beauty DNA' report — a combined skin, hair, makeup, and style analysis — with a Y2K / purikura-style filter camera. It runs ten on-device CoreML/TFLite models (FaceParsing, RMBG, Real-ESRGAN, StableLLVE, LaMa, MiDaS and more) for real-time skin scoring, segmentation, upscaling, and low-light correction, while offloading heavy generative transforms (age change, hair color, clothes swap) to an AWS API Gateway backend in a hybrid design. The live camera preview is GPU-accelerated through a BBMetalImage Metal pipeline with custom shaders. The codebase spans 319 Swift files across a four-layer Clean Architecture (Domain / Data / Presentation / Infrastructure).
Instead of juggling a scattered set of single-purpose beauty apps, Lumora condenses skin, hair, makeup, and style into one shareable report from a single selfie — and automatically splits instant on-device retouching from cost-heavy cloud generative transforms to balance speed and cost.
App preview
What it does
Beauty DNA Report
Generates an overall score plus four sections — skin, hair, makeup, style — from a single selfie, along with an undertone/season-based personal-color card. Each section is rendered by a dedicated report view (Skin/Hair/Makeup/Style), and BeautyReportPDFGenerator exports a PDF or share card to drive viral sharing.
AI Skin Score Engine
Computes a 0–100 score fully on-device via Vision face detection → ROI crop → CoreML (FaceParsing) skin segmentation → CIELAB color and texture analysis. Rather than a single number it produces weighted per-region scores (scoreByRegions), switching computeUnits between cpuOnly on simulator and cpuAndGPU on device, with history tracking and weekly reports.
Y2K & Purikura Filter Camera
Applies five styles — Sweet, Y2K, Sugar, Aqua, Retro — instantly via a pure CoreImage CIFilter chain (bloom, grain, chromatic aberration, vignette) with no backend call. All compositing runs on-device, so the styles are unlimited and consume no coins.
Real-time Metal Beauty Camera
Applies filters to the live preview through a GPU-accelerated Metal pipeline (BBMetalImage), with a custom bilateralFilter.metal shader handling real-time skin smoothing. Supports multiple aspect ratios (full-screen, 4:3, 16:9), front/back switching, and vertex/fragment-shader-based rendering.
On-device Body Reshaping
Detects body keypoints with PoseNet, then warps leg length and waistline on the GPU via custom Metal shaders (leg_stretch.metal, waist_shrink.metal). Exposed as domain UseCases (Taller, SlimWaist, LegModify), the reshaping runs instantly without any server call.
AI Photo Restoration & Enhancement
Upscales with Real-ESRGAN 512 and SRResNet, restores low-light shots with StableLLVE, and removes objects cleanly via LaMa (LaMaFP16_512) inpainting. RMBG handles background removal and MiDaS Swin2 Tiny estimates depth, keeping most of the restoration pipeline fully on-device.
Generative AI Studio
Offers ~20 transforms — age change, gender swap, hair-color try-on, clothes swap, virtual makeup, smile generation, cartoonization — delegating compute-heavy inference to an AWS API Gateway backend (120s timeout, Base64 JPEG payloads). Each transform is encapsulated as a UseCase (AgeTransform, GenderSwap, HairColorTryOn, ClothesChange, and more).
BeautyCoin Credits & Subscription
Runs a StoreKit 2 in-app economy (Transaction.currentEntitlements/updates via async/await) with consumable coins and a premium subscription, per-operation coin charges, lifetimeEarned/lifetimeSpent tracking, and a transaction-history dashboard. Coin balances sync to the server through BeautyCoinAPIClient, tying server inference cost to usage.
How it's built
Language
On-device CV / ML
Graphics / Camera
UI
Backend / Cloud
Commerce
Platform Services
A four-layer Clean Architecture (Domain, Data, Presentation, Infrastructure) using UseCase and Repository patterns, with a hybrid processing model that separates on-device CoreML/Metal work from server-side generative AI. The domain is encapsulated across 29 UseCases and 6 Repositories, while the Infrastructure layer modularizes CoreML, image-processing, AWS, and point services.
- 1
Hybrid AI routing: latency-sensitive skin analysis, Y2K filters, and body reshaping run on-device via CoreML/Metal, while compute-heavy generative transforms are delegated to an AWS API Gateway backend (ap-northeast-2, 120s timeout, Base64 JPEG payloads).
- 2
On-device model stack: ten models — FaceParsing (segmentation), RMBG (background removal), Real-ESRGAN/SRResNet (upscaling), StableLLVE (low-light restore), LaMa (inpainting), MiDaS (depth), PoseNet (pose) — are gathered in the Infrastructure/CoreML layer and wrapped behind single-responsibility runners.
- 3
On-device CV pipeline: a multi-stage skin-score engine chains Vision face detection → ROI crop → CoreML segmentation → CIELAB color/texture analysis, producing region-weighted scores and switching computeUnits between device and simulator.
- 4
Metal GPU acceleration: a BBMetalImage live-camera pipeline plus custom Metal shaders (bilateralFilter, leg_stretch, waist_shrink) handle real-time skin smoothing and body reshaping entirely on the GPU.
- 5
UseCase-level separation: 29 protocol-based UseCases are organized by domain (Beauty, Body, AI, Skin, Coin, Auth), with data access encapsulated behind six Repository abstractions (AITransform, Auth, BeautyCoin, HipEditing, ImageProcessing, SkinAnalysis) for testability and swappability.
- 6
Coin-based in-app economy: StoreKit 2 async/await loads and verifies consumable coins and subscriptions, with per-operation charges and lifetimeEarned/lifetimeSpent tracking synced to the server via BeautyCoinAPIClient to tie server inference cost to usage.
- 7
AWS auth/sync layer: Cognito-based user authentication, S3 upload, and CloudSyncService are modularized under Infrastructure/Service/AWS, keeping auth, storage, and commerce server integration decoupled from the camera/AI domains.
319
Swift files
10
On-device models
92
ViewControllers
29
UseCases
6
Repositories
4
Custom Metal shaders
20+
AI transform tools
5
Y2K filter styles