Guaranteed Augmentation Coverage on the Digits Dataset

Mikhail D. Kashpirev

Abstract


This study tests whether guaranteed transformation coverage improves classification accuracy over random mixing at the same training-data budget. Coverage-balanced multi-perturbation augmentation (CB-MPA-5) adds one copy from each of five transformation families to every training image. RandomMix-5 independently selects a family for each of five copies. Experiments use 1,797 Digits images of size 8×8, RBF-SVM, logistic regression, and twenty repeated stratified 70/30 holdouts. Seven corruption regimes belong to training families; four belong to families absent from the training policy. Ten realizations of each stochastic test corruption are averaged within each holdout. Paired comparisons use the Nadeau–Bengio corrected resampled t-test with Holm adjustment. The hypothesis of an additional benefit from guaranteed coverage is not supported for either classifier. This does not establish equivalence. Both multi-perturbation policies improve aggregate corrupted accuracy over clean training. For RBF-SVM, CB-MPA-5 raises mean accuracy from 84.19% to 89.96%, but reduces accuracy under blur from 88.26% to 82.80%. Coverage describes sample construction rather than demonstrated model superiority. Findings are restricted to this compact dataset and the specified synthetic corruptions; transfer to higher-resolution images or real acquisition defects remains untested. Code and aggregate source results accompany the article.

Full Text:

PDF (Russian)

References


C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995, doi: 10.1007/BF00994018.

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” J. Big Data, vol. 6, art. no. 60, 2019, doi: 10.1186/s40537-019-0197-0.

D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in Proc. ICLR, 2019. Available: https://openreview.net/forum?id=HJz6tiCqYm

R. Geirhos et al., “Shortcut learning in deep neural networks,” Nat. Mach. Intell., vol. 2, pp. 665-673, 2020, doi: 10.1038/s42256-020-00257-z.

E. Mintun, A. Kirillov, and S. Xie, “On interaction between augmentations and corruptions in natural corruption robustness,” in Adv. Neural Inf. Process. Syst., vol. 34, pp. 3571-3583, 2021. Available: https://proceedings.neurips.cc/paper/2021/hash/1d49780520898fe37f0cd6b41c5311bf-Abstract.html

E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “AutoAugment: Learning augmentation strategies from data,” in Proc. CVPR, pp. 113-123, 2019, doi: 10.1109/CVPR.2019.00020.

E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, “RandAugment: Practical automated data augmentation with a reduced search space,” in Adv. Neural Inf. Process. Syst., vol. 33, pp. 18613-18624, 2020. Available: https://proceedings.neurips.cc/paper/2020/hash/d85b63ef0ccb114d0a3bb7b7d808028f-Abstract.html

D. Hendrycks et al., “AugMix: A simple data processing method to improve robustness and uncertainty,” in Proc. ICLR, 2020. Available: https://openreview.net/forum?id=S1gmrxHFvB

H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” in Proc. ICLR, 2018. Available: https://openreview.net/forum?id=r1Ddp1-Rb

T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with Cutout,” arXiv:1708.04552, 2017, doi: 10.48550/arXiv.1708.04552.

Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 7, pp. 13001-13008, 2020, doi: 10.1609/aaai.v34i07.7000.

R. G. Lopes, D. Yin, B. Poole, J. Gilmer, and E. D. Cubuk, “Improving robustness without sacrificing accuracy with Patch Gaussian augmentation,” arXiv:1906.02611, 2019, doi: 10.48550/arXiv.1906.02611.

Z. Huang et al., “IPMix: Label-preserving data augmentation method for training robust classifiers,” in Adv. Neural Inf. Process. Syst., vol. 36, pp. 63660-63673, 2023. Available: https://proceedings.neurips.cc/paper_files/paper/2023/hash/c917d8b9e01427f3184d80ade22f4d1f-Abstract-Conference.html

N. Chen, J. Zhu, J. Chen, and B. Zhang, “Dropout training for support vector machines,” in Proc. AAAI Conf. Artif. Intell., vol. 28, no. 1, pp. 1752-1759, 2014, doi: 10.1609/aaai.v28i1.8965.

C. Nadeau and Y. Bengio, “Inference for the generalization error,” Mach. Learn., vol. 52, no. 3, pp. 239-281, 2003, doi: 10.1023/A:1024068626366.

S. Holm, “A simple sequentially rejective multiple test procedure,” Scand. J. Stat., vol. 6, no. 2, pp. 65-70, 1979. Available: https://www.jstor.org/stable/4615733

E. Alpaydin and C. Kaynak, “Optical Recognition of Handwritten Digits,” UCI Mach. Learn. Repository, 1998, doi: 10.24432/C50P49.

F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825-2830, 2011. Available: https://jmlr.org/papers/v12/pedregosa11a.html

P. Virtanen et al., “SciPy 1.0: Fundamental algorithms for scientific computing in Python,” Nat. Methods, vol. 17, pp. 261-272, 2020, doi: 10.1038/s41592-019-0686-2.

Scikit-learn developers, “load_digits,” scikit-learn documentation. Available: https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html (accessed Oct. 11, 2026).

D. Lakens, “Equivalence tests: A practical primer for t tests, correlations, and meta-analyses,” Soc. Psychol. Personal. Sci., vol. 8, no. 4, pp. 355–362, 2017, doi: 10.1177/1948550617697177.

Y. LeCun, C. Cortes, and C. J. C. Burges, “The MNIST database of handwritten digits.” Available: https://yann.lecun.org/exdb/mnist/ (accessed Oct. 11, 2026)


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ-КОНГРЕСС ВМК МГУ 2026 СНЭ

ISSN: 2307-8162