Synthetic Data With LLMs: Generating Training and Eval Sets

When synthetic data makes sense

Synthetic data is examples generated by a model instead of collected from real users. It is useful when real data is scarce, expensive to label, or too sensitive to use directly:

• classifiers: intents, ticket categories, spam detection, especially for rare classes;
• fine-tuning: instruction-answer pairs in your domain, tone and format;
• eval sets: test questions for a RAG system or a chatbot before launch;
• edge cases: typos, mixed Russian and Uzbek, rude messages, prompt injection attempts.

It does not replace real data. Synthetic examples reflect what the generating model thinks your users say, which is not always what they actually say. Use real data to seed, validate and test, and synthetic data to fill gaps.

A generation pipeline

1. Define the target: the task, label set, output format and what a good example looks like.
2. Collect seeds: a small set of real, anonymized examples per class.
3. Design variation axes: the dimensions that should differ between examples.
4. Generate in batches with prompts that combine seeds and axes.
5. Filter automatically, then review a sample by hand.
6. Deduplicate and balance classes.
7. Validate by training or evaluating and comparing against a real-data baseline.

Ensuring diversity

The main failure of naive generation is monotony: asking "write 100 customer complaints" returns 100 variations of the same complaint. Make diversity explicit.

• Vary attributes systematically: persona, language, length, tone, channel, topic, difficulty. Sample combinations and put them in each prompt.
• Rotate seed examples between batches so the model does not anchor on the same few.
• Ask for specific scenarios, not generic text: "a returning customer whose order arrived damaged, writing briefly from a phone".
• Use more than one generator model when possible, since each has its own style.
• Measure diversity: look at embedding clusters and n-gram overlap, and check that each class covers several sub-topics.

import random

axes = {
    "persona": ["student", "small shop owner", "retiree", "IT specialist"],
    "language": ["Russian", "Uzbek (Latin)", "mixed Russian and Uzbek"],
    "tone": ["polite", "irritated", "very brief"],
    "length": ["one sentence", "three to four sentences"],
}

def make_spec():
    return {k: random.choice(v) for k, v in axes.items()}

Filtering low-quality samples

Expect a noticeable share of generated examples to be wrong, duplicated or off-target. Filter in layers:

• Format checks: valid JSON, required fields, length limits.
• Deduplication: exact and near-duplicate removal by embeddings.
• Label verification: a separate model or prompt classifies the example; drop those where the label disagrees.
• LLM-as-judge: score realism and correctness against a rubric; keep only examples above a threshold.
• Human review: read a random sample from every class. If reviewers reject many, fix the prompt rather than filtering harder.

Avoiding model collapse

Model collapse happens when models are trained repeatedly on their own outputs: rare cases disappear and outputs become bland and repetitive. To avoid it:

• always keep real data in the training mix;
• do not regenerate data from a model fine-tuned on earlier synthetic data, generation after generation;
• watch for shrinking variety: shorter answers, fewer distinct phrasings, missing rare classes;
• track where every example came from with a provenance field.

Avoiding data leakage

Leakage makes results look better than they are.

• Separate train and eval generation: different seeds, prompts and ideally different generator models. Deduplicate across splits.
• Keep a real-data test set that no synthetic example was derived from. Final decisions rely on it.
• Do not leak personal data: anonymize seeds, and check that outputs do not reproduce real names, phone numbers or documents.
• Check license terms of the generator model; some providers restrict using outputs to train competing models.