| 1. |
Junji Shimagaki, Yasutaka Kamei, Naoyasu Ubayashi, Abram Hindle, Automatic topic classification of test cases using text mining at an Android smartphone vendor, International Symposium on Empirical Software Engineering and Measurement, 10.1145/3239235.3268927, 32-10, 2018.10, © 2018 ACM. Background: An Android smartphone is an ecosystem of applications, drivers, operating system components, and assets. The volume of the software is large and the number of test cases needed to cover the functionality of an Android system is substantial. Enormous effort has been already taken to properly quantify "what features and apps were tested and verified?". This insight is provided by dashboards that summarize test coverage and results per feature. One method to achieve this is to manually tag or label test cases with the topic or function they cover, much like function points. At the studied Android smartphone vendor, tests are labelled with manually defined tags, so-called "feature labels (FLs)", and the FLs serve to categorize 100s to 1000s test cases into 10 to 50 groups. Aim: Unfortunately for developers, manual assignment of FLs to 1000s of test cases is a time consuming task, leading to inaccurately labeled test cases, which will render the dashboard useless. We created an automated system that suggests tags/labels to the developers for their test cases rather than manual labeling. Method: We use machine learning models to predict and label the functionality tested by 10,000 test cases developed at the company. Results: Through the quantitative experiments, our models achieved acceptable F-1 performance of 0.3 to 0.88. Also through the qualitative studies with expert teams, we showed that the hierarchy and path of tests was a good predictor of a feature's label. Conclusions: We find that this method can reduce tedious manual effort that software developers spent classifying test cases, while providing more accurate classification results.. |
| 2. |
Shane McIntosh, Yasutaka Kamei, Are Fix-Inducing Changes a Moving Target? A Longitudinal Case Study of Just-In-Time Defect Prediction, IEEE Transactions on Software Engineering, 10.1109/TSE.2017.2693980, 44, 5, 412-428, 2018.05, Just-In-Time (JIT) models identify fix-inducing code changes. JIT models are trained using techniques that assume that past fix-inducing changes are similar to future ones. However, this assumption may not hold, e.g., as system complexity tends to accrue, expertise may become more important as systems age. In this paper, we study JIT models as systems evolve. Through a longitudinal case study of 37,524 changes from the rapidly evolving Qt and OpenStack systems, we find that fluctuations in the properties of fix-inducing changes can impact the performance and interpretation of JIT models. More specifically: (a) the discriminatory power (AUC) and calibration (Brier) scores of JIT models drop considerably one year after being trained; (b) the role that code change properties (e.g., Size, Experience) play within JIT models fluctuates over time; and (c) those fluctuations yield over- and underestimates of the future impact of code change properties on the likelihood of inducing fixes. To avoid erroneous or misleading predictions, JIT models should be retrained using recently recorded data (within three months). Moreover, quality improvement plans should be informed by JIT models that are trained using six months (or more) of historical data, since they are more resilient to period-specific fluctuations in the importance of code change properties.. |
| 3. |
Xiaochen Li, He Jiang, Yasutaka Kamei, Xin Chen, Bridging Semantic Gaps between Natural Languages and APIs with Word Embedding, IEEE Transactions on Software Engineering, 10.1109/TSE.2018.2876006, 2018.01, Developers increasingly rely on text matching tools to analyze the relation between natural language words and APIs. However, semantic gaps, namely textual mismatches between words and APIs, negatively affect these tools. Previous studies have transformed words or APIs into low-dimensional vectors for matching; however, inaccurate results were obtained due to the failure of modeling words and APIs simultaneously. To resolve this problem, two main challenges are to be addressed: the acquisition of massive words and APIs for mining and the alignment of words and APIs for modeling. Therefore, this study proposes Word2API to effectively estimate relatedness of words and APIs. Word2API collects millions of commonly used words and APIs from code repositories to address the acquisition challenge. Then, a shuffling strategy is used to transform related words and APIs into tuples to address the alignment challenge. Using these tuples, Word2API models words and APIs simultaneously. Word2API outperforms baselines by 10%-49.6% of relatedness estimation in terms of precision and NDCG. Word2API is also effective on solving typical software tasks, e.g., query expansion and API documents linking. A simple system with Word2API-expanded queries recommends up to 21.4% more related APIs for developers. Meanwhile, Word2API improves comparison algorithms by 7.9%-17.4% in linking questions in Question&Answer communities to API documents.. |
| 4. |
Gopi Krishnan Rajbahadur, Shaowei Wang, Yasutaka Kamei, Ahmed E. Hassan, The Impact Of Using Regression Models to Build Defect Classifiers, International Conference on Mining Software Repositories (MSR 2017), 2017.05. |
| 5. |
Yasutaka Kamei, Takafumi Fukushima, Shane McIntosh, Kazuhiro Yamashita, Naoyasu Ubayashi, Ahmed E. Hassan, Studying just-in-time defect prediction using cross-project models, Empirical Software Engineering, 10.1007/s10664-015-9400-x, 21, 5, 2072-2106, 2016.10, © 2015, Springer Science+Business Media New York. Unlike traditional defect prediction models that identify defect-prone modules, Just-In-Time (JIT) defect prediction models identify defect-inducing changes. As such, JIT defect models can provide earlier feedback for developers, while design decisions are still fresh in their minds. Unfortunately, similar to traditional defect models, JIT models require a large amount of training data, which is not available when projects are in initial development phases. To address this limitation in traditional defect prediction, prior work has proposed cross-project models, i.e., models learned from other projects with sufficient history. However, cross-project models have not yet been explored in the context of JIT prediction. Therefore, in this study, we empirically evaluate the performance of JIT models in a cross-project context. Through an empirical study on 11 open source projects, we find that while JIT models rarely perform well in a cross-project context, their performance tends to improve when using approaches that: (1) select models trained using other projects that are similar to the testing project, (2) combine the data of several other projects to produce a larger pool of training data, and (3) combine the models of several other projects to produce an ensemble model. Our findings empirically confirm that JIT models learned using other projects are a viable solution for projects with limited historical data. However, JIT models tend to perform best in a cross-project context when the data used to learn them are carefully selected.. |
| 6. |
Junji Shimagaki, Yasutaka Kamei, Shane Mcintosh, Ahmed E. Hassan and Naoyasu Ubayashi, A Study of the Quality-Impacting Practices of Modern Code Review at Sony Mobile, the International Conference on Software Engineering (ICSE2016) Software Engineering in Practice (SEIP), 2016.05, Nowadays, a flexible, lightweight variant of the code review process (i.e., the practice of having other team members critique software changes) is adopted by open source and pro prietary software projects. While this flexibility is a blessing (e.g., enabling code reviews to span the globe), it does not mandate minimum review quality criteria like the formal code inspections of the past. Recent work shows that lax reviewing can impact the quality of open source systems. In this paper, we investigate the impact that code review- ing practices have on the quality of a proprietary system that is developed by Sony Mobile. We begin by replicating open source analyses of the relationship between software quality (as approximated by post-release defect-proneness) and: (1) code review coverage, i.e., the proportion of code changes that have been reviewed and (2) code review partic ipation, i.e., the degree of reviewer involvement in the code review process. We also perform a qualitative analysis, with a survey of 93 stakeholders, semi-structured interviews with 15 stakeholders, and a follow-up survey of 25 senior engineers. Our results indicate that while past measures of review coverage and participation do not share a relationship with defect-proneness at Sony Mobile, reviewing measures that are aware of the Sony Mobile development context are associated with defect-proneness. Our results have lead to improvements of the Sony Mobile code review process.. |
| 7. |
Junji Shimagaki, Yasutaka Kamei, Shane McIntosh, Ahmed E. Hassan, Naoyasu Ubayashi, A study of the quality-impacting practices of modern code review at Sony mobile, Proceedings - International Conference on Software Engineering, 10.1145/2889160.2889243, 212-221, 2016.05, © 2016 ACM. Nowadays, a flexible, lightweight variant of the code review process (i.e., the practice of having other team members critique software changes) is adopted by open source and proprietary software projects. While this flexibility is a blessing (e.g., enabling code reviews to span the globe), it does not mandate minimum review quality criteria like the formal code inspections of the past. Recent work shows that lax reviewing can impact the quality of open source systems. In this paper, we investigate the impact that code reviewing practices have on the quality of a proprietary system that is developed by Sony Mobile. We begin by replicating open source analyses of the relationship between software quality (as approximated by post-release defect-proneness) and: (1) code review coverage, i.e., the proportion of code changes that have been reviewed and (2) code review participation, i.e., the degree of reviewer involvement in the code review process. We also perform a qualitative analysis, with a survey of 93 stakeholders, semi-structured interviews with 15 stakeholders, and a follow-up survey of 25 senior engineers. Our results indicate that while past measures of review coverage and participation do not share a relationship with defect-proneness at Sony Mobile, reviewing measures that are aware of the Sony Mobile development context are associated with defect-proneness. Our results have lead to improvements of the Sony Mobile code review process.. |
| 8. |
Yasutaka Kamei, Emad Shihab, Defect Prediction: Accomplishments and Future Challenges, Leaders of Tomorrow / Future of Software Engineering Track at International Conference on Software Analysis Evolution and Reengineering (SANER2016), Issue 2, pp.99-104., 2016.03. |
| 9. |
Yasutaka Kamei, Takafumi Fukushima, Shane McIntosh, Kazuhiro Yamashita, Naoyasu Ubayashi and Ahmed E. Hassan, Studying Just-In-Time Defect Prediction using Cross-Project Models, Journal of Empirical Software Engineering, Online first (pp.1-35), 2015.09. |
| 10. |
Shane Mcintosh, Yasutaka Kamei, Bram Adams and Ahmed E. Hassan, An Empirical Study of the Impact of Modern Code Review Practices on Software Quality, Journal of Empirical Software Engineering, Online first (pp.1-45), 2015.05. |
| 11. |
Takafumi Fukushima, Yasutaka Kamei, Shane McIntosh, Kazuhiro Yamashita and Naoyasu Ubayashi, An Empirical Study of Just-In-Time Defect Prediction Using Cross-Project Models, International Working Conference on Mining Software Repositories (MSR 2014), pp.172-181, 2014.06. |
| 12. |
Shane Mcintosh, Yasutaka Kamei, Bram Adams and Ahmed E. Hassan, The Impact of Code Review Coverage and Code Review Participation on Software Quality: A Case Study of the Qt, VTK, and ITK Projects, International Working Conference on Mining Software Repositories (MSR 2014), pp.192-201, 2014.06. |
| 13. |
Emad Shihab, Yasutaka Kamei, Bram Adams, and Ahmed E. Hassan, Is Lines of Code a Good Measure of Effort in Effort-Aware Models?, Information and Software Technology, Vol.55, No.11, 2013.11. |
| 14. |
Emad Shihab, Akinori Ihara, Yasutaka Kamei, Walid M. Ibrahim, Masao Ohira, Bram Adams, Ahmed E. Hassan and Ken-ichi Matsumoto, Studying Re-opened Bugs in Open Source Software, Journal of Empirical Software Engineering, Vol.18, No.5, pp.1005-1042, 2013.10. |
| 15. |
Yasutaka Kamei, Emad Shihab, Bram Adams, Ahmed E. Hassan, Audris Mockus, Anand Sinha and Naoyasu Ubayashi, A Large-Scale Empirical Study of Just-In-Time Quality Assurance, IEEE Transactions on Software Engineering, Vol.39, No.6, pp.757-773, 2013.06. |
| 16. |
Masateru Tsunoda, Koji Toda, Kyohei Fushida, Yasutaka Kamei, Meiyappan Nagappan and Naoyasu Ubayashi, Revisiting Software Development Effort Estimation Based on Early Phase Development Activities, International Working Conference on Mining Software Repositories (MSR 2013), 2013.05. |
| 17. |
Yasutaka Kamei, Hiroki Sato, Akito Monden, Shinji Kawaguchi, Hidetake Uwano, Masataka Nagura, Ken-Ichi Matsumoto, Naoyasu Ubayashi, An Empirical Study of Fault Prediction with Code Clone Metrics, The Joint Conference of the 21th International Workshop on Software Measurement and the 6th International Conference on Software Process and Product Measurement (IWSM/MENSURA2011), pp.55-61, 2011.11. |
| 18. |
Emad Shihab, Audris Mockus, Yasutaka Kamei, Bram Adams, Ahmed E. Hassan,, High-Impact Defects: A Study of Breakage and Surprise Defects, the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE2011), pp.300-310, 2011.09. |
| 19. |
Shane McIntosh, Bram Adams, Thanh H. D. Nguyen, Yasutaka Kamei and Ahmed E. Hassan, An Empirical Study of Build Maintenance Effort, the 33rd International Conference on Software Engineering (ICSE2011), pp.141-150, 2011.05. |
