Full-stack ML data collection and labeling platform with privacy-preserving image processing, interactive bounding box annotation, and role-based workspace management for university courses.
Built with Jenny Leana Fotso Ngompe, under Prof. Anthony Vanky and Prof. Tian Zheng
Columbia's AiX Convergence Design Studio needed students to collect and label street imagery for computer vision research. Street photography means faces, license plates, and storefront signage, none of which belongs in a dataset that a rotating cast of students can browse each semester.
The naive fix is a policy: tell students not to upload identifiable people. Policies fail quietly, and by the time you find a violation the image is already in storage with its EXIF geotags intact. I wanted privacy enforced by the pipeline, not by instructions in a syllabus.
Every upload runs through an automated privacy pipeline before it ever reaches storage. Three face detectors (MTCNN, RetinaFace, and MediaPipe) run as an ensemble, and I take the union of their detections rather than the intersection. A union produces false positives, which cost a slightly over-blurred image; an intersection produces false negatives, which cost someone's face. That asymmetry makes the choice obvious.
EasyOCR handles text regions for signage and plates. Detected faces and text get Gaussian-blurred, EXIF metadata is stripped wholesale, and images are downsized before landing in Google Cloud Storage behind cached signed URLs.
The labeling side is a Konva.js canvas with fuzzy-search label suggestions and abstract label classes for semantic grouping. Because it's a course tool rather than a research demo, it needed real access control: JWT auth with Student/TA/Admin roles and workspace isolation so each semester's data stays scoped to that cohort.
The backend is FastAPI with SQLAlchemy against Cloud SQL Postgres, deployed on Cloud Run. ML models load as lazy singletons. Cloud Run cold starts are brutal if you initialize three face detectors eagerly on every container. Database pooling is tuned to 10 connections and 5 overflow.
upload
│
▼
PRIVACY PIPELINE ─────────────────────────────┐
├─ MTCNN ┐ │
├─ RetinaFace ├─▶ union of detections │
├─ MediaPipe ┘ │
├─ EasyOCR ──────▶ text regions │
├─ gaussian blur all regions │
├─ strip EXIF │
└─ resize (max 1024px) │
│ │
▼ │
GCS (signed URLs, cached) ◀───────────────────┘
│
▼
Konva.js labeling canvas ──▶ Cloud SQL (Postgres)
fuzzy label search RBAC · workspace isolation
abstract label classes JWT authThe platform runs as the studio's actual data pipeline, with a statistics dashboard, a gallery filterable by label, uploader, and date, and a markdown-based instruction system so course staff can update guidance without a deploy.
The privacy guarantee is structural: because blurring and EXIF stripping happen before the write to GCS, there is no code path that stores an unprocessed image. Reviewing that property is a matter of reading one function instead of auditing every caller.
Choosing union over intersection for the detector ensemble was the single decision that made the privacy claim defensible, and it took about a minute to implement. Getting the error asymmetry right upfront mattered far more than any amount of model tuning afterward.
Lazy-loading the ML singletons wasn't premature optimization, it was the difference between a usable tool and one where the first upload of the day timed out. Serverless changes which optimizations are optional.
class PrivacyProcessor:
"""Ensemble face + text detection with automatic blurring."""
def __init__(self):
self.mtcnn = MTCNN()
self.retinaface = RetinaFace()
self.mediapipe = mp.solutions.face_detection.FaceDetection()
self.ocr = easyocr.Reader(["en"])
def process(self, image: np.ndarray) -> np.ndarray:
# Ensemble face detection: union of all detections
faces = set()
faces.update(self.mtcnn.detect(image))
faces.update(self.retinaface.detect(image))
faces.update(self.mediapipe.detect(image))
# Text region detection
text_regions = self.ocr.detect(image)
# Apply gaussian blur to all detected regions
for region in faces | text_regions:
image = apply_blur(image, region, method="gaussian")
# Strip EXIF metadata and resize
image = strip_exif(image)
return resize_max(image, max_dim=1024)