cad a22 4500 551414332 20190720041557.0 cr unu---uuuuu 181214s2018 xx s 000 0 eng 10.5167/uzh-159273 doi 10.1016/j.infsof.2018.02.010 doi (ZORA)oai:www.zora.uzh.ch:159273 000 ddc We're doing it live: A multi-method empirical study on continuous experimentation [Elektronische Daten] [Gerald Schermann, Jürgen Cito, Philipp Leitner, Uwe Zdun, Harald C Gall] openAccess eu-repo Context Continuous experimentation guides development activities based on data collected on a subset of online users on a new experimental version of the software. It includes practices such as canary releases, gradual rollouts, dark launches, or A/B testing. Objective Unfortunately, our knowledge of continuous experimentation is currently primarily based on well-known and outspoken industrial leaders. To assess the actual state of practice in continuous experimentation, we conducted a mixed-method empirical study. Method In our empirical study consisting of four steps, we interviewed 31 developers or release engineers, and performed a survey that attracted 187 complete responses. We analyzed the resulting data using statistical analysis and open coding. Results Our results lead to several conclusions: (1) from a software architecture perspective, continuous experimentation is especially enabled by architectures that foster independently deployable services, such as microservices-based architectures; (2) from a developer perspective, experiments require extensive monitoring and analytics to discover runtime problems, consequently leading to developer on call policies and influencing the role and skill sets required by developers; and (3) from a process perspective, many organizations conduct experiments based on intuition rather than clear guidelines and robust statistics. Conclusion Our findings show that more principled and structured approaches for release decision making are needed, striving for highly automated, systematic, and data- and hypothesis-driven deployment and experimentation. Department of Informatics zora Schermann Gerald joint author Cito Jürgen joint author Leitner Philipp joint author Zdun Uwe joint author Gall Harald C. joint author Information and Software Technology 99:41-57 https://www.zora.uzh.ch/id/eprint/159273/1/IST_preprint.pdf text/html WWW-Backlink auf das Repository 1 Journal Article PeerReviewed zora ZORA 856 40 https://www.zora.uzh.ch/id/eprint/159273/1/IST_preprint.pdf text/html WWW-Backlink auf das Repository ZORA 700 1- Schermann Gerald joint author ZORA 700 1- Cito Jürgen joint author ZORA 700 1- Leitner Philipp joint author ZORA 700 1- Zdun Uwe joint author ZORA 700 1- Gall Harald C. joint author ZORA 773 0- Information and Software Technology 99:41-57 BK010053 XK010053 XK010000 ZORA ZORA ZORA Journal Article openAccess