Features of the Development of High Load Data Systems
Abstract
The article develops the design features of data-intensive systems. Sources of recent years are analyzed, international experience in creating information systems is considered, the concept of high load is formulated and common practices and technologies are listed. Based on a number of sources on this topic, it has been revealed that specific solutions for developing the application infrastructure are selected individually, depending on the features of the project and the tasks to be solved. At the same time, the basic requirements are universal for the sphere of information systems: the reliability, the scalability or ease of maintenance and the architectural patterns used in the design. The article summarizes new information, including international experience in this area, and draws conclusions about the most common practices in the design of high-load systems.
Full Text:
PDF (Russian)References
G. Casale, D. Ardagna, M. Artac, F. Barbier, E. D. Nitto, A. Henry, G. Iuhasz, C. Joubert, J. Merseguer, V. I. Munteanu, “Dice: quality-driven development of data-intensive cloud applications.” in Proceedings of the Seventh International Workshop on Modeling in Software Engineering. IEEE Press, 2015, pp. 78–83.
C. A. Mattmann, D. J. Crichton, A. F. Hart, C. Goodale, J. S. Hughes, S. Kelly, L. Cinquini, T. H. Painter, J. Lazio, D. Waliser, “Architecting data-intensive software systems” in Handbook of Data Intensive Computing, Springer, 2011, pp. 25–57.
M. Felderer, B. Russo, F. Auer, “On Testing Data-Intensive Software Systems”, 2019.
D. Obuhov, Blog Highload Junior [Jelektronnyj resurs], “Highload dlja nachinajushhih”. URL: http://highload.guide/blog/highload-for-beginners.html
A. Lerner, “The Cost of Downtime”, [Jelektronnyj resurs], Gartner, 2014. URL: https://blogs.gartner.com/andrew-lerner/2014/07/16/the-cost-of-downtime/
A. Brown, G. Wilson, “Scalable Web Architecture and Distributed Systems”, in The Architecture of Open Source Applications, Volume II, Mountain View, 2012, pp. 1-23.
O. Hummel, H. Eichelberger, A. Giloj, D. Werle, K. Schmid, “A collection of software engineering challenges for big data system development”, in 2018 44th Euromicro Conferenceon Software Engineering and Advanced Applications (SEAA), IEEE, 2018, pp. 362–369.
M. Kleppmann, “Designing Data-Intensive Applications”, 1st Edition. Sebastopol: O’Reilly Media, Inc, 2017.
S. Henderson, “Building Scalable Web Sites”, 1st Edition, Sebastopol, O’Reilly Media, Inc, 2006.
O. Bunin, “Vstupitel'naja stat'ja”, [Jelektronnyj resurs], Blog Highload Junior. URL: http://highload.guide/blog/what_data_we_have.html)
O. Bunin, “Obshhaja logika masshtabirovanija”, [Jelektronnyj resurs], Blog Highload Junior. URL: http://highload.guide/blog/scaling-logic.html
B. Burns, “Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services”, 2018.
M. de Bayser, L. G. Azevedo, R. Cerqueira, “The case for devops in scientific applications”, in 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), 2015, pp. 1398–1404.
R.T. Fielding, “Architectural Styles and the Design of Network-based Software Architectures”, dissertation, 2000.
D. Ford, “Availability in Globally Distributed Storage Systems” in 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2010.
R. T. Kouzes, G. A. Anderson, S. T. Elbert, I. Gorton, D. K. Gracio, “The Changing Paradigm of Data-Intensive Computing” in Computer, vol.42, no.1, 2009, pp. 26–34.
C. Lynch, “Big data: How do your data grow?”, Nature, 2008, pp. 28–29.
C. Mattmann, “Software Connectors for Highly Distributed and Voluminous Data-Intensive Systems”, Ph.D. dissertation, University of Southern California, 2006.
S. McConnell, “Software Estimation: Demystifying the Black Art”, 1st edition, Microsoft Pres, 2006.
A. Perez "High-load services: Lecture course”, 2017.
I. C. Richard, “How Complex Systems Fail”, Chicago, Cognitive technologies Laboratory, University of Chicago, 2000.
T. White, “Hadoop: The Definitive Guide”, 2nd Edition, O’Reilly, 2010.
Refbacks
Abava Кибербезопасность IT Congress 2024
ISSN: 2307-8162