Facial Aesthetics Prediction using Deep Learning and Geometric Modelling
A concise research presentation: a hybrid framework combining convolutional visual perception with dense geometric landmark modelling to predict human-rated facial attractiveness with interpretable sub-scores. Intended for faculty and PhD evaluators seeking reproducible, research-grade methodology and clear links between perceptual attributes and model outputs.
GitHub
Executive overview
Project title: Multi-Modal Facial Aesthetics Assessment using Deep Learning and Geometric Modeling.
One-line pitch: A hybrid computer-vision framework combining CNN regression with interpretable geometric feature learning to predict human attractiveness scores.
1
Dataset
5,500 human-annotated frontal images from SCUT-FBP5500; continuous scores (1–5).
2
Dual-path model
Image-based CNN for texture + geometry-based MLP for structural sub-scores.
3
Sub-score decomposition
Predicts eyes, nose, mouth, jaw, cheeks and skin sub-scores with learnable aggregation.
Motivation & problem definition
Facial aesthetics influence social interaction, HCI and biometric fairness. Existing models are often opaque, offering limited alignment with human perceptual factors. This work seeks explicit interpretability without sacrificing predictive accuracy.
Perception gap
Models must explain which facial regions drive scores to be useful for research and ethics reviews.
Fairness & insight
Integrating geometric priors reduces spurious texture cues and supports comparative analyses across groups.
Research question
Can a CNN+geometry fusion outperform single-modality approaches while remaining interpretable to evaluators?
Dataset & preprocessing
SCUT-FBP5500: 5,500 frontal images with human ratings (1–5). Preprocessing: resize to 224×224, RGB normalisation, 90/10 train–validation split. Landmark extraction via MediaPipe FaceMesh (~468–478 points).
Sample
Processed & landmarks
Landmark density enables fine-grained geometric features (ratios, curvatures, symmetry indices) critical for perceptual sub-score computation.
Model A — CNN end‑to‑end regression
Backbone: EfficientNet‑V2‑S (ImageNet pretrained). Regression head: dropout + linear output predicting continuous attractiveness (MSE loss). Optimiser: Adam (lr=1e‑4), 5 epochs. Strength: robust visual feature learning; limitation: limited interpretability.
Model B — Geometric features & interpretable learning
Extracted features: eye aspect ratio & symmetry, mouth width, lip thickness, nose width, jawline curvature, cheek prominence, and skin texture sharpness (Laplacian variance). Features normalised by face width and z-scored across the dataset.
Eyes
Aspect ratio and bilateral symmetry quantify openness and alignment.
Mouth & lips
Normalized width and thickness metrics linked to perceived attractiveness cues.
Jaw & cheeks
Curvature and prominence captured as polynomial fits and peak distances.
Hybrid model — CNN + geometry fusion
Architecture: CNN embedding (MobileNetV3 → 64‑D) concatenated with the geometric feature vector; MLP predicts six sub‑scores plus overall score via learnable softmax aggregation. Fusion preserves texture cues while restoring structural interpretability.
01
1. Visual embedding
Extracted CNN features represent texture and global appearance.
02
2. Geometric vector
Standardised structural features capture interpretable cues.
03
3. Fusion & aggregation
Concatenate → MLP → sub-scores → weighted sum to overall prediction.
Results, evaluation & analysis
Metrics: MAE and RMSE. The hybrid model yields lower MAE/RMSE than single-modality baselines and produces coherent sub-score distributions centred near perceptual midpoints. Learned aggregation weights indicate balanced importance across regions.
Example inferences
Input image, landmark overlay, predicted overall score and sub‑score breakdown are produced for each test case for transparent evaluation.
Technical stack & reproducibility
Primary tools: PyTorch (models), Torchvision, MediaPipe (landmarks), OpenCV, NumPy, Pandas, Matplotlib. Reproducibility: modular pipeline, cached features, fixed random seeds, and versioned saved weights and config files.
Computing
PyTorch + NumPy; CPU/GPU agnostic training scripts.
Pipeline
Preprocessing, cached landmarks, feature stores and reproducible eval scripts.
Artifacts
Versioned model weights and configuration for audit and replication.
Closing & academic positioning
This project demonstrates readiness for PhD‑level research at the intersection of computer vision, biomedical imaging and affective computing. It balances state‑of‑the‑art prediction with interpretable, region‑level explanations suitable for scholarly review and ethical evaluation.
Suitable for
Computer Vision, Biomedical Engineering, Affective Computing.
Contributions
Hybrid modelling, sub-score interpretability, reproducible pipeline.
Made with