Derma Synth: GPT-4-Based Synthetic Text Generation for Dermatology via Meta-Prompting and Few-Shot Conditioning

Abstract

Automated classification of dermatological conditions from free text is limited by the scarcity of labelled clinical and patient-narrative corpora. We present DermaSynth, a meta-prompting framework that uses GPT-4 to expand a seed set of 67 real dermatology texts into a synthetic dataset of 653 labelled texts covering seven conditions (acne, contact dermatitis, eczema, folliculitis, psoriasis, scabies and vitiligo; 82–100 texts per class). Generation combines a dermatologist-persona context prompt, a diversity prompt, and in-context exemplars drawn from the seed set (k ∈ {2, 4, 6, 8}). We evaluate the dataset with Random Forest (RF) and Multinomial Naïve Bayes (MNB) classifiers under repeated stratified cross-validation, and through ratings by three board-certified dermatologists. RF reaches 94.55 ± 0.55% accuracy (macro-F1 0.944) and MNB 91.58 ± 0.45% (macro-F1 0.915) in 5×5 cross-validation, and the RF advantage is statistically significant on paired predictions (McNemar p = 0.003). However, 58.7% of the texts name their own disease, and masking the names lowers accuracy to 84.6% (RF) and 83.5% (MNB), so about 8–10 points of accuracy come from keyword cues. Texts also have a low length (mean 21.7 words), while contact dermatitis is very templated. Data was evaluated 4.53/5 by dermatologists on four different dimensions (realism 4.3/5). All classification results are on synthetic text; transfer to real patient text remains to be tested.