Possibilities of Replacing Ration Metrics When Conducting A/B Testing
Abstract
The need for testing is to check the correctness of the product's operation on a small amount of data during its implementation in order to avoid errors during subsequent use. The main types of testing of information resources are usability and A/B testing.
The similarity between A/B tests and usability testing is that:
- Both methods are aimed at improving the user experience and efficiency of the product.
- They are used to optimize the interfaces and content of products based on real data.
- They allow you to identify problem areas and identify growth points.
The differences between A/B tests and usability testing are that:
- Usability testing focuses on assessing the ease of use of a product, while A/B tests compare different versions of a product to determine effectiveness.
- In usability testing, users perform tasks and researchers observe their actions, while A/B tests compare the results of using different versions of a product.
- Usability testing falls into the category of “qualitative”, while A/B testing, in turn, falls into the category of “quantitative”.
Full Text:
PDF (Russian)References
Kozyreva N. E., Rahmanova A. Yu. (2021) A/B testirovanie kak instrument ocenki vzaimodejstviya Brenda s potrebitelyami v didzhital srede. Ekonomika i biznes tendencii i innovacii. P. 295-303.
Vysockaya A.I. Komarcheva A.R., Chzhen A.A. A/B testirovanie v digital // EO IPSO. 2024, No.5, P. 75.
Bobko D.V., Shinkevich K.A. 2020 marketingovye issledovaniya na osnove a v testirovaniya v cifrovyh kompaniyah.
Zhukovskij V.A. 2019 Povyshenie effektivnosti organizacii posredstvom razrabotki frejmvorka avtomatizirovannogo A/B testirovaniya. Mezhdunarodnyj akademicheskij vestnik. 10-88-91
Tyurinova V.A., Maurits V.G. Metody A/B testirovaniya informacionnyh resursov i58 finansovaya sistema v nacionalnoje konomike predposyl 175.
Bychkov I.V., Dedkova S.N. 2013 centralizovannoe testirovanie po-matematike nestandartnyj sposob resheniyauravnenij soderzhaschih-peremennuyu pod znakom modulya.
Soyunov H.T. Internet-marketing strategii instrument I trendy in kachestvo-upravlencheskih kadrov I ekonomicheskaya bezopasnost organizacii. 2019. p-110-113.
Bazhan Z.I. 2016 testirovanie kak odin iz effektivnyh sposobov proverki teoreticheskoj I metodicheskoj podgotovki obuchayuschihsya v vuze problem sovremennogo pedagogicheskogo obrazovaniya. 51-2-34-40.
Claeys, E., Gancarski, P., Maumy-Bertrand, M., Wassner, H.: Dynamic allocation optimization in A/B-tests using classification-based preprocessing. IEEE TKDE 35(1), 335–349 (2021)
Fabijan, A., Dmitriev, P., Arai, B., Drake, A., Kohlmeier, S., Kwong, A.: A/B integrations: 7 lessons learned from enabling A/B testing (2023)
Kaufmann, E., Cappé, O., Garivier, A.: On the complexity of A/B testing. ArXiv e-prints, May 2014?
Astakhova I. F., Makoviy K. A., Khitskova Yu. V. Intellectualization system for usability testing of information resources // Actual problems of applied mathematics, computer science and mechanics. 2022. P. 1719-1726.
Karpov Courses: website. URL: https://karpov.courses/ (date of access 10.09.2024)
Kaluza, B., Mirchevska, V., Dovgan, E., Lustrek, M., Gams, M.: UCI machine learning repository, an agent-based approach to care in independent living (2010) /
Grushka-Cockayne, Yael, et al. "A/B Testing at Vungle." Darden Business Publishing Cases (2015): 1-7.
Gui, H., Xu, Y., Bhasin, A., & Han, J. (2015, May). Network a/b testing: From sampling to estimation. In Proceedings of the 24th International Conference on World Wide Web (pp. 399-409).
Siroker, D., & Koomen, P. (2015). A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons.
King, R., Churchill, E. F., & Tan, C. (2017). Designing with data: Improving the user experience with A/B testing. " O'Reilly Media, Inc.".
Nguyen, H. Q. (2001). Testing applications on the Web: Test planning for Internet-based systems. John Wiley & Sons.
Johari, R., Koomen, P., Pekelis, L., & Walsh, D. (2017, August). Peeking at a/b tests: Why it matters, and what to do about it. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1517-1525).
Quin, F., Weyns, D., Galster, M., & Silva, C. C. (2024). A/B testing: A systematic literature review. Journal of Systems and Software, 112011.
Deng, A., & Shi, X. (2016, August). Data-driven metric development for online controlled experiments: Seven lessons learned. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 77-86).
Gui, H., Xu, Y., Bhasin, A., & Han, J. (2015, May). Network a/b testing: From sampling to estimation. In Proceedings of the 24th International Conference on World Wide Web. Р. 399-409
Fabijan, A., Dmitriev, P., McFarland, C., Vermeer, L., Holmström Olsson, H., & Bosch, J. (2018). Experimentation growth: Evolving trustworthy A/B testing capabilities in online software companies. Journal of Software: Evolution and Process, 30(12), e2113.
Kohavi, R., & Longbotham, R. (2015). Online controlled experiments and A/B tests. Encyclopedia of machine learning and data mining, 1-11.
Mosin P. “Linearization: why and how to tame ratio metrics in A/B tests.” URL: https://habr.com/ru/companies/kuper/articles/768826/, (date accessed 09/15/2024)
Refbacks
- There are currently no refbacks.
Abava Кибербезопасность IT Congress 2024
ISSN: 2307-8162