About me
I am Liang Yu at the Software Engineering ReThought in Sweden, part of the Blekinge Institute of Technology. My research focuses on quality evaluation of GenAI-based systems in software engineering. I study how organizations evaluate the quality of GenAI system outputs using metrics. Through field studies with industry partners, I develop practical and evidence-based guidance to enhance GenAI adoption and usage in software development companies. Also, I am a software developer adopting and utilizing GenAI techniques in Fintech business contexts.
PhD thesis summary
Software organizations seek gains from adopting and utilize GenAI while facing data-privacy risks and uncertain output quality. We investigated how to adopt and use GenAI in software engineering with sandboxed environment, measurable system qualities, and supportive technical choices. Our studies include a field deployment, a structured review, industrial case studies, a quality evaluation framework, a field study of coding assistants, and a controlled experiment on agent designs. We designed an LLM setup to secure enterprise proprietary data with practical steps that teams can operate in practice. We conducted a snowballing review that organizes metrics mapping to ISO/IEC quality characteristics, and investigated in industry cases how these metrics are applied, what they cost, and where they fall short. We proposed an evaluation framework that places legal, license, and security controls and quality measurement at defined points in ideation, development, and operation, with clear ownership. We studied GenAI coding assistants in the field and identified where time is saved and which integration hurdles matter to teams. We also ran a controlled experiment on programming tasks comparing single-agent and multi-agent systems to inform organizational decisions about when multi-agent setups are worth adopting. The thesis contributes a runnable process and a metrics-based framework for evaluating the quality of GenAI-based system outputs. Our findings can support organizations to move from pilots to evidence-based GenAI practices.