Dataset Design, Annotation, and Quality

Before a dataset can be labeled or used for training, decisions about how it was sampled shape everything downstream: a dataset collected only from one hospital, one time period, or one demographic group will systematically fail to represent cases outside that scope, no matter how large it is. Stratified sampling addresses part of this problem by deliberately drawing examples from each relevant subgroup in proportion to (or in some controlled ratio to) their real-world prevalence, rather than relying on whatever happens to be collected first; Google's Machine Learning Crash Course material on datasets emphasizes that how a dataset is split and characterized directly affects whether conclusions drawn from it will generalize. A worked example: if a disease-detection dataset is built by pulling every available image from a single clinic, it may accidentally overrepresent one patient population and underrepresent others; stratified sampling across multiple clinics and demographic groups produces a dataset whose subgroup proportions are known and intentional rather than accidental.

Class imbalance is a related but distinct problem: even a well-sampled dataset can have a rare category of interest, such as the 1-in-200 disease-positive X-rays in the scenario above. A model trained naively on such data can achieve high overall accuracy simply by always predicting the majority class, while being useless at detecting the minority class that actually matters. Common techniques to address this include resampling — either oversampling the minority class (duplicating or synthesizing more examples of it) or undersampling the majority class (using fewer of the common examples) — and class weighting, where the training process is told to penalize mistakes on the rare class more heavily than mistakes on the common class. A worked example: in a fraud-detection dataset where only 1 percent of transactions are fraudulent, training with class weights that make a missed fraud case count 99 times more costly than a false alarm forces the model to actually learn to distinguish fraud rather than ignoring it in favor of near-perfect accuracy on the easy majority class.

Sampling and balance decisions are not purely statistical — they are part of dataset design, meaning they should be made deliberately with the dataset's intended use in mind and recorded rather than left implicit. The Datasheets for Datasets framework explicitly calls for documenting a dataset's intended uses and the population it was drawn from as part of responsible dataset creation, noting that the machine learning community has historically lacked a standardized process for this kind of documentation, which can lead to serious consequences in high-stakes domains such as medical diagnosis. A worked example: a dataset built to detect a rare disease should record not just its overall size, but how many examples came from each clinic, each demographic group, and each class, so that anyone evaluating the dataset later can judge whether a specific patient population is likely to be well or poorly served by a model trained on it.

Many AI datasets require human annotators to assign labels — marking whether an X-ray shows disease, whether a tweet's sentiment is positive or negative, or whether a sentence contains toxic language — and the quality of those labels depends heavily on clear guidelines and the annotators' own backgrounds and judgment. Bender and Friedman's paper on data statements for natural language processing argues that NLP datasets should document who annotated the data and under what instructions, since annotator demographics and guidelines can systematically shape which patterns end up labeled consistently and which do not. A worked example: three radiologists reviewing the same set of borderline chest X-rays may disagree on ambiguous cases precisely because the labeling guidelines did not specify exactly which visual features count as diagnostic — a problem that better-written annotation instructions, and documentation of who wrote and applied them, can help surface and address. Recording annotator background matters too: a dataset labeled entirely by annotators from one specialty or one country may apply subtly different conventions than a more diverse annotator pool would, and without documenting who labeled the data, a downstream user has no way to know that this source of variation exists at all.

When multiple annotators label the same data, the degree to which they agree — inter-annotator agreement — is itself a signal about label quality: if trained radiologists frequently disagree on the same X-ray, that disagreement suggests either that the task is genuinely ambiguous for certain cases or that the labeling guidelines need to be clarified, and either way, low agreement should raise doubt about how much a resulting model's labels can be trusted as ground truth. A worked example: if annotators agree on 95 percent of straightforward cases but only 60 percent of borderline cases, a dataset curator might choose to have a senior expert resolve every disagreement, exclude the ambiguous cases from the training set, or record disagreement itself as a feature rather than forcing every example into a single, potentially unreliable label — any of these choices should be documented rather than silently baked into the final dataset.

Both the Datasheets for Datasets framework and the Data Statements for NLP framework converge on the same underlying idea: a dataset should be accompanied by documentation that explains its motivation, composition, collection process, intended uses, and known limitations, so that someone downstream can evaluate whether it is appropriate for their purpose. Gebru et al. propose datasheets — modeled explicitly on the datasheets that accompany electronic components in engineering — as a standardized questionnaire dataset creators fill out, covering questions like why the dataset was created, what preprocessing was done, and whether it contains data about people. Quality assurance for a dataset, then, is not a single check but an ongoing discipline: verifying sampling and class balance are appropriate for the intended use, checking annotation guidelines and agreement, and producing documentation thorough enough that a future user — or the original team six months later — can audit exactly how the dataset was built and what it can and cannot be trusted to support.