catnip
a two-stage pipeline, still very much a work in progress
Catnip tries to answer one important question: is Izutsumi in this image? It runs a two-stage pipeline: first detect, then identify over Dungeon Meshi manga pages. Nothing is exported yet.
The pipeline
1. Localization (SAHI + YOLO26n). Find where bodies and faces are. A YOLO26n model runs over full manga pages that SAHI slices into overlapping 640×640 patches to handle different scales. The training set is pre-sliced with the same parameters.
2. Re-Identification (refactor/reID branch). Find Izutsumi herself. Metric learning:
- Backbone is a ResNet18 (ImageNet-pretrained) with GeM pooling.
- Loss* Triplet Loss with hard-negative mining.
- Manually labeled crops Label Studio crops plus Stage 1 detector crops.
- FAISS
IndexFlatIPover L2-normalized embeddings w/ cosine similarity for matching. Optional LogisticRegression re-ranking.
Datasets
The same annotations feed both stages:
- izutsumi: My own manually labeled dataset. A bunch of Izutsumi crops.
- manga109: Parsed and normalized with manga109api (face, body, text, and frame objects).
- deepghs/anime_head_detection: YOLOv8 head detection data (v1.0, v2.0, and the third-party
ani_face_detection). - nyuuzyou/AnimeHeadsv3: COCO-format, augmented head dataset.