Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Prioritizing tasks in software development: A systematic literature review

Roles Conceptualization, Formal analysis, Methodology, Software

Affiliation Huawei, Moscow, Russia

ORCID logo

Roles Data curation, Validation, Writing – review & editing

Affiliation Institute of Software Development and Engineering, Innopolis University, Innopolis, Russia

Roles Conceptualization, Validation, Writing – original draft

Roles Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected] (MF); [email protected] (AK)

Affiliation Institute of Human and Social Sciences, Innopolis University, Innopolis, Russia

Roles Software, Supervision, Writing – original draft

Roles Conceptualization, Data curation, Formal analysis, Writing – original draft

Roles Methodology, Supervision, Writing – review & editing

Affiliation Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada

Roles Conceptualization, Funding acquisition, Methodology, Software, Supervision, Writing – original draft

Affiliation Department of Computer Science and Engineering, University of Bologna, Bologna, Italy

  • Yegor Bugayenko, 
  • Ayomide Bakare, 
  • Arina Cheverda, 
  • Mirko Farina, 
  • Artem Kruglov, 
  • Yaroslav Plaksin, 
  • Witold Pedrycz, 
  • Giancarlo Succi

PLOS

  • Published: April 6, 2023
  • https://doi.org/10.1371/journal.pone.0283838
  • Reader Comments

Table 1

Task prioritization is one of the most researched areas in software development. Given the huge number of papers written on the topic, it might be challenging for IT practitioners–software developers, and IT project managers–to find the most appropriate tools or methods developed to date to deal with this important issue. The main goal of this work is therefore to review the current state of research and practice on task prioritization in the Software Engineering domain and to individuate the most effective ranking tools and techniques used in the industry. For this purpose, we conducted a systematic literature review guided and inspired by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, otherwise known as the PRISMA statement. Based on our analysis, we can make a number of important observations for the field. Firstly, we found that most of the task prioritization approaches developed to date involve a specific type of prioritization strategy— bug prioritization . Secondly, the most recent works we review investigate task prioritization in terms of “pull request prioritization” and “issue prioritization,” (and we speculate that the number of such works will significantly increase due to the explosion of version control and issue management software systems). Thirdly, we remark that the most frequently used metrics for measuring the quality of a prioritization model are f-score , precision , recall , and accuracy .

Citation: Bugayenko Y, Bakare A, Cheverda A, Farina M, Kruglov A, Plaksin Y, et al. (2023) Prioritizing tasks in software development: A systematic literature review. PLoS ONE 18(4): e0283838. https://doi.org/10.1371/journal.pone.0283838

Editor: Bilal Alatas, Firat Universitesi, TURKEY

Received: January 4, 2023; Accepted: March 17, 2023; Published: April 6, 2023

Copyright: © 2023 Bugayenko et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data are all contained within the paper and/or Supporting information files.

Funding: This research was supported by Huawei Technologies. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

In software development, the vast majority of tasks do not have mandatory dependencies and it is up to the project manager to decide which task should be completed first. The proper continuous prioritization of tasks (known as backlog refinement in agile terminology) becomes a critical success factor for any software development project, as it guarantees that the company’s crucial goals are in focus and can be met [ 1 ].

What is a task, though? The term “task” in software engineering refers to the smallest unit of work subject to management accountability that needs to be completed as part of a software development project [ 2 ]. So, in the context of software development, the term task is an umbrella term that encompasses concepts, such as “pull request” and “issue,” commonly found in GitHub/GitLab integration (so development areas) [ 3 ], or to ideas, such as “bug,” “feature,” “improvement,” commonly used in task management. Although these concepts and ideas are considered conceptually independent, they often overlap in practice.

In an attempt to optimize the process and practice of task prioritization, researchers approached the problem from a bug-fixing perspective; that is, in terms of selecting the most appropriate developer for the given task [ 4 ]. Cubranic and Murphy were among the first to analyze the problem of task prioritization in terms of Machine Learning (ML); namely as a classification problem [ 5 ]. The datasets provided in their research, Eclipse (see https://bugs.eclipse.org/bugs/ ) and Mozilla (see http://www.mozilla.org/projects/bugzilla ), have become “de facto” the standard for training and testing ML models for this problem domain.

However, it is worth noting, that other researchers developed alternative methods and approaches to improve the process of prioritizing and assigning bug fixes. For example, Zimmermann et al. [ 6 ] provided a series of recommendations for formulating and better classifying bug reports, while Anvik et al. [ 7 ] proposed an effective strategy for developers selection. Panjer [ 8 ] formulated a method capable of predicting bugs’ lifetime and Wang et al. [ 9 ] suggested a new technique for identifying bug duplicates.

Menzies and Marcus [ 10 ] adopted another conceptual framework for dealing with the problem of task prioritization and proposed a solution based on the prediction of the severity of bug reports. Their work formed the conceptual palette necessary for the development of further research on bug priorities prediction, such as the works by Sharma et al. [ 11 ] and Tian et al. [ 12 ].

The importance, urgency, and significance of this problem for the Software Engineering community is also attested by the recent publication of several surveys, such as [ 13 – 16 ]. Among them, the work of Gousios et al. showed that the issue of task prioritization is particularly sensitive for development teams that follow a pull-based development model [ 16 – 18 ].

The considerations made above clearly demonstrate that task prioritization has become an active research topic in software engineering. On the one hand, its growth signals a positive trend: the more people get involved in the discussion of these issues, the more ideas are generated and accumulated in the scientific community. On the other hand, though, wide participation poses potentially insurmountable challenges for researchers and developers in terms of understanding the current state and capabilities of the field. Therefore, we believe that a comprehensive systematic literature review (SLR) carried out on this topic is going to be highly beneficial for researchers, project managers, developers, scrum masters, and other industry practitioners.

Research problem and objectives

Taking this important observation as a starting point this work reviews how the IT industry addresses the problem of task prioritization and attempts to produce a state-of-the-art summary of tools and techniques used for this purpose. Although we do not limit our work to specific methods, we expect to mostly gather Machine Learning (ML)-based approaches. This is because of the recent successes of ML in software engineering and computer science [ 19 , 20 ].

The objectives of the SLR are therefore to:

  • present our readership (mostly IT practitioners) with newly-developed techniques for ranking tasks that they can reliably use in their work,
  • develop new strategies in ranking and prioritizing tasks, thus filling current gaps in the relevant literature and
  • identify possible directions for future research.

The scientific contribution of this paper includes structured information on task prioritization, a survey of existing tools and approaches, methods, and metrics, as well as some estimates about their effectiveness and reliability.

Structure of the paper

This paper is organized as follows. Section Related Works provides an overview of current research on task prioritization and a helpful comparison between such research and the focus and scope of our work. Section SLR Protocol Development describes the protocol used in this systematic literature review. Section Results presents the results of our work, while section Discussion contextualizes our findings and section Critical Review of our Research Question provides their critical interpretation. Section Limitations, Threats to Validity, and Review Assessment evaluates the limitations and various other shortcomings potentially affecting our study, while section Conclusion summarizes what we achieved and points out future research directions.

Related works

There exist a number of studies devoted to requirements prioritization techniques. For example, Achimugu et al. [ 21 ] found that the most cited techniques for requirements prioritization include Analytical Hierarchy Process (AHP), Pairwise Comparison, Cost-Value Prioritization, and Cumulative Voting. More recent trends in prioritizing requirements include ML techniques (such as Case Base Ranking and Fuzzy AHP). Bukhsh et al. [ 22 ] also identified a trend toward fuzzy logic and machine learning methods. Somohano-Murrieta et al. [ 23 ] investigated the most documented techniques with regard to scalability and time consumption problems. Rashdan [ 24 ] found evidence of a shift towards computed-assisted/algorithmic methods, while Sufian et al. [ 25 ] analyzed factors that influence prioritization and identified commonly used techniques and tools aimed at improving the process. These studies underline the importance and evolution of requirements prioritization techniques, and -at the same time- emphasize the need for real-world evaluations and scalability solutions.

There are also studies aimed at analyzing other aspects of software engineering, which are typically connected with prioritization issues (such as analysis of non-functional requirements, code smells, technical debt, and software bugs). For example, Kaur et al. [ 26 ] identified existing techniques for code smell prioritization and introduced different tools for prioritizing code smells (such as Fusion, ConQAT, SpIRIT, JSpIRIT, PMD, Fica, JCodeOdor, and DT-SOA). Alfayez et al. [ 27 ] investigated technical debt prioritization and identified a number of important techniques used, which include: Cost-Benefit Analysis, Ranking, Predictive Analytics, Real Options Analysis, Analytic Hierarchy Process, Modern Portfolio Theory, Weighted Sum Model, Business Process Management, Reinforcement Learning, and Software Quality Assessment Based on Lifecycle Expectations (SQALE). However, the researchers concluded that more research is needed to develop technical debt prioritization approaches capable of effectively considering costs, values, and resource constraints. Ijaz et al. [ 28 ] looked at non-functional requirements prioritization techniques and found that AI techniques can potentially handle uncertainties in requirements while contributing to overcome the most common limitations characterizing standard approaches (such as AHP). Pasikanti and Kawaf [ 29 ] studied the latest trends in software bug prioritization and identified a series of ML techniques (such as Naive Bayes, Support Vector Machines, Random Forest, and Multinational Naive Bayes) that are most commonly used for prioritizing software bugs.

Several SLRs were also conducted to identify the most commonly used techniques for test case selection and prioritization in software testing. For example, Pan et al. [ 30 ] found that Supervised Learning, Unsupervised Learning, Reinforcement Learning, and NLP-based methods have been applied to test case prioritization; yet, due to a lack of standard evaluation procedures, the authors couldn’t draw reliable conclusions on their effective performance. Bajaj and Sangwan [ 31 ] observed that genetic algorithms bear great potential for solving test case prioritization problems, while nevertheless noting that the design of parameter settings, type of operators, and fitness function significantly affects the quality of the solutions obtained.

Another important area of research focuses on aspects of task assignment and allocation in software development projects. Filho et al. [ 32 ] reviewed works on multicriteria models for task assignment in distributed software development projects with a special focus on qualitative decision-making methods. TAMRI emerged as the most efficient and widely used approach, while McDSDS, Global Studio Project, and 24-Hour Development Model received lower scores. Fatima et al. [ 33 ] studied the models used for task assignment and scheduling in software projects. The review found that static models are the most widely used for task scheduling, while the Support Vector Machine algorithm is the most widely used for task assignment. Both these papers demonstrated the importance of considering, as crucial for the practice of software management, specific factors (such as personal aspects, team skills, labor cost, geographic issues, and task granularity).

However, the contribution of our SLR is unique and different from that of the above-mentioned studies because: ( Table 1 ):

  • Unlike other SLRs, which have focused -as we have seen above- on prioritization techniques for requirements, test cases, bugs, and/or other artifacts of software development; our own review provides a comprehensive coverage of the problem at stake. Crucially, it does so by describing the broad category of “task”, without focusing on a specific type of prioritized item.
  • In addition, our research differs from prior studies on task allocation/assignment in several aspects. Firstly, the problem of assignment/allocation involves distributing tasks based on various factors (such as skills, availability, workload, etc), whereas the problem of prioritization focuses on determining which tasks should be completed first. Secondly, prior research has predominantly relied on qualitative analyses of algorithms, methods, and tools for task allocation/assignment, without conducting detailed quantitative analyses of their effectiveness. Our research aims to bridge these important gaps in the literature by conducting a comprehensive quantitative analysis of task prioritization techniques, which are used to determine their effectiveness in different contexts.

thumbnail

  • PPT PowerPoint slide
  • PNG larger image
  • TIFF original image

https://doi.org/10.1371/journal.pone.0283838.t001

SLR protocol development

SLRs offer a comprehensive analysis of the research conducted in the field while also providing critical, original insights [ 34 ]. They are of paramount importance for scientific progress and, for this reason, represent one of the preferred methods used by researchers to investigate the state of the art of a particular research topic [ 35 ].

The quality of SLRs can vary greatly and it is important to ensure that an SLR is conducted in a rigorous and systematic manner [ 36 , 37 ]. Thus, to ensure the comprehensiveness and soundness of our work we followed the PRISMA Statement [ 38 ], which is essentially a checklist, conventionally adopted by researchers worldwide, to guide, orient, and inform the development of any SLR. The PRISMA 2020 checklist adopted for this study is included as S1 Table .

Since the Prisma checklist abovementioned is not -strictly speaking- a methodological framework; rather a series of suggestions or -better- recommendations to be implemented for the sound development of any SLR (even beyond computer science), we decided to integrate it and complement it with a more specific methodological framework; the one recently developed by Kitchenham and Charters [ 39 ]. This framework was chosen due to its focus on software engineering and because its effectiveness has been amply demonstrated in previous studies [ 40 – 42 ]. We believe that complementing the general indications or recommendations outlined in the PRISMA checklist (which are valid for any field) with a framework specifically designed for research on software engineering is highly beneficial for this study, as it guarantees better accuracy. In addition, since the checklist and the framework partially overlap (despite being also complementary), one can use them to mutually strengthen each other. The stages of the methodological framework adopted in this SLR, are:

  • Specification of the research questions.
  • Development of the review protocol.
  • Formulation of the literature log.
  • Performance of quality assessment.
  • Extraction of Data.
  • Data synthesis.
  • Formulation of the main report.
  • Evaluation of the review and of the report.

Research questions

The first step in any SLR involves the formulation of a series of research questions that can guide and inform its development.

To formulate the most appropriate research questions, we adopted the Goal Question Metric (GQM) model developed by Basili et al. [ 43 ]. This model requires specifying up front the purpose of analysis, the objects and the issues to analyze, as well as the standpoints from which the analysis is performed. The Goal Question Metric model for this work is the following:

  • Purpose Systematic literature review.
  • Object Peer review publications in computer science and software engineering.
  • Issue Approaches for ranking tasks in software development.
  • Viewpoint Software engineers and industry practitioners.

With the GQM model in place, we then formulated the Research Questions (RQs) that characterized this work:

  • RQ1 What are the existing approaches for automatic task ranking in software development?
  • RQ2 Which methods are used in automatic task ranking models and approaches and how is their effectiveness assessed?
  • RQ3 What are the most effective and versatile models for automatic task ranking developed so far?

The motivation for RQ1 is to gain a clear understanding of existing research on the topic. Then, moving from general to more specific tasks, we formulate RQ2 with the intent of finding out which methods for task ranking are currently the most popular in the software development industry and how their effectiveness can be assessed. Further research along these lines leads to RQ3, through which we try to rank such methods in terms of effectiveness, accuracy, fidelity, and reliability. This could help developing new ranking strategies and remedial approaches for the field.

Literature search process

Following the best practices in the field [ 42 ], we selected the following databases for our searches: Google Scholar, Microsoft Academic, ScienceDirect, IEEE Xplore, and ACM digital library.

We then extracted a set of basic keywords, which describe our research questions. The keywords are: a) manage, b) backlog, c) priority, d) task, e) job, f) commit, g) bug, h) pull request, i) issue, j) feature, k) software, l) rank, m) distributed software development, n) machine learning.

Searches via keywords yielded a very large number of papers. Thus, to screen out irrelevant documents and add focus and precision to our work, we formulated a set of search queries by using Boolean operators, as common in the literature. Upon conducting an initial screening of papers, we discovered that there were more papers focused on prioritizing bug reports than those focused on prioritizing pull requests and issues. Because of this, we decided to model a series of search queries around these themes for better coverage. The list of queries we used for our searches is reported below:

  • ((pull OR merge) AND request) OR Github issue) AND (prioritization or priority OR rank OR order OR ranking OR ordering)
  • (task or bug or defect or feature) AND (prioritization or priority OR rank OR order OR ranking OR ordering)
  • bug severity AND priority AND (machine learning OR neural network)

We performed our searches by using these queries on the selected databases. Table 2 displays the results we obtained.

thumbnail

Acronyms used: GS—Google Scholar, IEEE—IEEE Xplore, MA—Microsoft Academic, SD—ScienceDirect.

https://doi.org/10.1371/journal.pone.0283838.t002

Inclusion and exclusion criteria

Next, we specified inclusion (IC) and exclusion (EC) criteria as recommended by Patino and Ferreira [ 44 ]. IC and EC help the authors decide which articles found through seminal searches deserve to be considered for further analysis. In this study we used the following IC and EC:

  • IC1 The paper is written in English.
  • IC2 The paper is peer-reviewed and published by a reputable publisher.
  • IC3 The paper was published as early as 2006*.
  • IC4 The paper uses ML techniques to deal with backlog systems or tasks/todos.
  • IC5 The paper compares different ML models or compares ML models with other learning models.
  • EC1 The paper does not satisfy at least one of the ICs.
  • EC2 The paper is a duplicate or contains duplicate information.
  • EC3 The paper is an editorial, an opinion piece, or an introduction. In general, the paper is excluded if it does not contain any original insight.
  • EC4 The paper does not present any type of experimentation or comparison or results.

*Shoham et al. [ 45 ] noted that around 2006, there was a pick of interest in ML in the software engineering community. We thus selected this year as the starting point for our systematic review.

Search results by sources

In this subsection, we offer to our readers a detailed description of the process that led to the inclusion of preliminary selected papers in our final reading log ( Table 3 ).

thumbnail

The table shows the procedure through which potentially relevant papers were screened out through the adoption of IC and EC criteria. The number of papers included in the final reading log is shown in the column “Selected papers”.

https://doi.org/10.1371/journal.pone.0283838.t003

We note that we only considered the first 100 results displayed in the relevant databases for each of the four queries we formulated. This is justified by the fact that the databases we used normally sort out results by significance and credibility (e.g., h-index, number of citations, impact factor, etc.) and by the observation that usually no relevant paper is found after the first 100 results.

The PRISMA flow chart diagram shown in Fig 1 represents the process of inclusion/exclusion visually for the reader.

thumbnail

It shows the stages of the search process as a flowchart diagram [ 38 ].

https://doi.org/10.1371/journal.pone.0283838.g001

Quality assessment

To assess the quality of the manuscripts, we defined a set of criteria and applied them to all the papers selected for inclusion in our reading log:

  • QA1 Were the objectives and the research questions clearly specified?
  • QA2 Were the results evaluated critically and comprehensively?
  • QA3 Was the research process transparent and reproducible?
  • QA4 Are there comparisons with alternatives?

We then determined whether the papers we selected matched the criteria and—in case—the extent to which they did so. So, we assigned 1 if a paper fully matched the criterion, 0.5 if it partially matched the criterion, and 0 otherwise.

The criteria used for QA1 are:

  • Fully matched The objectives and research questions were explicitly stated.
  • Partially matched The goals of the paper and its research questions were sufficiently clear but could be improved.
  • Not matched No objectives were stated if the research questions were hard to determine, or if they didn’t relate to the research being carried out.

The criteria used for QA2 are:

  • Fully matched The authors of the paper provided a critical, balanced, and fair analysis of their results.
  • Partially matched The results were only partly (sufficiently) scrutinized and a comprehensive critical analysis was missing.
  • Not matched The authors did not evaluate their results.

The criteria used for QA3 are:

  • Fully matched The paper specified the methodology and the technologies used as well as the data gathered.
  • Partially matched Minor details were lacking (for example, a dataset is not readily available).
  • Not matched It was impossible to restore the sequence of actions or if other critical details (such as an algorithm or technologies used) were missing.

The criteria used for QA4 are:

  • Fully matched A comparison with other solutions offered; advantages and limitations clearly stated.
  • Partially matched The comparison was offered, but it was not comprehensively discussed.
  • Not matched No comparison was provided.

The resulting scores are shown in S2 Table , and their distribution can be found in Fig 2 .

thumbnail

Each paper was evaluated on a scale from 0 to 1 as per QA1-QA4. The bars display the number of papers with their respective quality score.

https://doi.org/10.1371/journal.pone.0283838.g002

There were many high-quality papers among those we selected for inclusion in our final log, which is demonstrated by the scores reported in Fig 2 . The average quality score was 2.9 out of 4. This confirms the reliability of the findings on which we based our SLR.

This section presents the findings gathered from the papers we included in our final reading log. More specifically, in this section, we use a series of statistical tools to cluster and organize the papers we selected in meaningful ways. Such clustering is beneficial for our readers because it provides some background for the conclusions we will draw in subsequent sections.

Preliminary clustering

We start this process of clustering by summarizing the potential advantages and disadvantages of the databases we used to perform our searches.

  • Microsoft Academic has the advantage of extensive coverage of scientific research, including patents. Its limitation is that some of the papers it lists are not peer-reviewed.
  • IEEE Xplore provides peer-reviewed publications, generally of high quality. Its limitation is that its full functionality requires a subscription, which is pricy.
  • ScienceDirect offers comprehensive coverage with tools for statistical analysis. However, it is beyond a paywall and has limitations for query building.
  • ACM provides comprehensive coverage with a particular emphasis on IT. Its major limitation is that it requires a subscription.
  • Google Scholar is one of the best database aggregators. It provides comprehensive coverage and tools for statistical analysis. However, it includes grey literature and non-peer-reviewed publications.

While not of crucial importance for the development of this work, we notice that such–complementary–information can be useful to ensure the academic integrity and scientific soundness of our approach.

The distribution of the papers included in the final reading log by databases is presented in Table 3 . Fig 3 re-elaborates the information contained in Table 3 in the form of a pie chart, which is probably more appealing for the reader. For convenience, we attributed papers to single repositories (even though some papers could be found across different databases). The attribution was subjective in character and determined by the chronological order of the searches we performed.

thumbnail

The pie chart shows the percentage of papers found in the databases we considered in this study. Acronyms used: GS—Google Scholar, IEEE—IEEE Xplore, SD—ScienceDirect. The number of papers is given in brackets.

https://doi.org/10.1371/journal.pone.0283838.g003

To give the reader a fuller picture of our results, we added information about the distribution of papers by publisher. This information can be found in Fig 4 . We note that the following journals and publishers fall under the label “others,” which accounts for about 14% of selected studies: ASTL (SERSC), CES (hikari), EISEJ, IJACSA, IJARCS, IJCNIS, IJCSE, IJOSSP, JATIT, Sensors (MDPI), and TIIS (KSII).

thumbnail

The pie chart shows the distribution (in percentages) of the papers we considered in our reading log by publishers. Acronyms used: IEEE—IEEE Xplore, WS—World Scientific.

https://doi.org/10.1371/journal.pone.0283838.g004

Studies classification

In this subsection, we present a series of statistical data that can be used to cluster our findings. Firstly, we identified 2 major topics characterizing the studies we included in our reading log: “Bug prioritization”, “Bug severity prediction”, and 2 minor topics “Issue prioritization”, and “Pull Request prioritization”. It is worth noting that even though bug severity [ 46 ] and bug priority [ 47 ] are two different theoretical entities (often treated as such even by project managers), a few works [ 12 , 48 , 49 ] demonstrated that severity can sometimes help predict priority. This is why, in this study, we consider not only papers concerned with bug priority but also those related to bug severity. Table 4 shows the distribution of publications across these topics.

thumbnail

The table shows the number (column “Quantity”) of papers devoted to a particular key topic (column “Topic”). Note: 2 papers have content for both topic 1 and 2 distribution.

https://doi.org/10.1371/journal.pone.0283838.t004

Secondly, we clustered the distribution of topics by year of publication (Figs 5 and 6 ). The dynamics of growth for the key topics underlying this study are roughly the same. This suggests that the scientific community is equally interested in both topics. As we noted above, this demonstrates their close interrelation.

thumbnail

The bars show the number of papers related to the key topic published in a particular year. Black bars show the number of papers related to “bug prioritization”. White bars show the number of papers related to “bug severity and prediction”.

https://doi.org/10.1371/journal.pone.0283838.g005

thumbnail

The bars show the number of papers related to the key topic published in a particular year. Black bars show the number of papers related to “issue prioritization”. White bars show the number of papers related to “pull request prioritization”.

https://doi.org/10.1371/journal.pone.0283838.g006

Thirdly, building and expanding on this classification, we clustered the papers we selected by the year of publication. Fig 7 shows our results. The same information is presented in Table 5 , where it is aggregated and visualized over a 4-years period.

thumbnail

The bars show the number of papers published on the topic between 2010 and 2021. No papers we found for the period 2006–2009. 2006 was the starting year for our SLR IC3.

https://doi.org/10.1371/journal.pone.0283838.g007

thumbnail

The table shows the number (column “Quantity”) and percentage (column “Percentage”) of papers for the specified period (column “Years”).

https://doi.org/10.1371/journal.pone.0283838.t005

literature review for software developers

The Pearson correlation coefficient is 0.9 with a p -value of 6.4 e − 06. This confirms our assumption that there is a significant synergy between the growth in the number of ML tools and their application to our problem domain.

Fourthly, we also collected some statistics related to the tags used in the papers we included in the final reading log. Information about this point is presented in Fig 8 .

thumbnail

The bars show the number of papers related to a specific tag. Note: several tags can be assigned to one paper.

https://doi.org/10.1371/journal.pone.0283838.g008

We note that the information presented in Fig 8 can be used to:

  • add more substance to the conclusions related to algorithm distribution we made in Further Clustering;
  • validate the relevance and the significance of our selection (the papers included in the final reading log);
  • characterize the most popular “subtopics” investigated by researchers worldwide in the selected domain.

Further clustering

We next proceed to further cluster our results and we do so along three dimensions: a) algorithms, b) datasets, and c) metrics. Fig 9 shows the algorithms used for training the models. Naive Bayes [ 51 ] is the most popular method among the models observed in the papers we reviewed.

thumbnail

The bars show the number of papers in which the specified algorithms were considered. Note: several algorithms can be considered in one paper.

https://doi.org/10.1371/journal.pone.0283838.g009

In addition, we also clustered the datasets used in the papers included in the final reading log. The most often used datasets are presented in Fig 10 .

thumbnail

The bars show the number of papers in which the specified algorithms were considered. Note: several datasets could be considered in one paper.

https://doi.org/10.1371/journal.pone.0283838.g010

Fig 10 shows the datasets most frequently used, which account for 48.7% of all datasets. The remaining datasets, accounting for 51.3% of the total, have been used only once, for example, bug repository of hdfs, etc. It is also worth noting that a single dataset can be found in many articles. The total number of dataset occurrences is calculated based on this important observation.

Finally, we collected statistics about the metrics used in the papers we included in our reading log ( Fig 11 ). We did not plot the metrics reported once. Nevertheless, we believe that such metrics are important because they might be used to create a comprehensive overview of their usage, which can be instrumental in evaluating the effectiveness of task prioritization models. These metrics include: average percentage of faults detected (APFD), normalized discounted cumulative gain (NDCG), mean squared error (MSE), Cohen’s kappa coefficient, nearest false negatives, nearest false positives, adjusted r squared, prediction time, training time, and robustness.

thumbnail

The bars show the number of papers in which the specified metrics were used. Note: several metrics could be used in one paper. Acronyms used: AUC—area under the ROC curve, MCC—Matthews Correlation Coefficient, MRR—mean reciprocal rank, MAE—mean absolute error, MAP—mean average precision, ROC—receiver operating characteristic curve.

https://doi.org/10.1371/journal.pone.0283838.g011

We note that some papers may contain multiple metrics, which might be jointly used to assess and more comprehensively evaluate the quality of a model. The f-score, as shown in Fig 11 , is the most commonly used metric in the papers we reviewed.

In this section, we contextualize and critically discuss the data presented in Results, while also highlighting their significance and relevance for the field.

RQ1. What are the existing approaches for automatic task ranking in software development?

As discussed in the Introduction, task prioritization can be divided into 3 subtopics: issues prioritization, PRs prioritization, and bugs prioritization. In this study, we treat these topics as independent issues and discuss them below in order of appearance in the literature. Bugs prioritization originated first, and it involves prioritizing the bugs based on their severity and impact on the software system [ 52 ]. Issue prioritization is the process of selecting and ranking issues based on factors such as their importance and urgency [ 53 ]. Finally, Pull Request prioritization is the process of selecting and ranking pull requests based on their impact on the software system and their relationship with other pull requests [ 54 ]. Overall trends in the task prioritization field are shown in Fig 12 .

thumbnail

The graph illustrates the chronological development of approaches to task prioritization from 2009 to 2022, specifically with respect to bugs, issues, and pull requests (PR). Each subtopic is denoted by bold text, followed by the total number of associated works. The figure is divided into three vertical fragments, each representing one of the subtopics. Green and red arrows indicate an increase or decrease in the number of publications, respectively.

https://doi.org/10.1371/journal.pone.0283838.g012

Bug prioritization

In this work, we understand bugs as reports by users and developers about program components that do not function properly. The collection of attributes used to describe the reports is usually determined by the platform on which the report was created. The platform used to detect bugs in most cases is Bugzilla (see https://www.bugzilla.org/ ). In 2004, one of the first studies by Cubranic and Murphy [ 5 ] on bug prioritization was conducted. Although this paper has been highly influential in the literature, we did not include this study in our main log because it failed to satisfy one of our inclusion criteria (namely, the third criterion).

However, since 2004, which is the year in which this paper was published, the field of bug prioritization boomed, giving raise to many profitable investigations on: prioritization on imbalanced datasets, prioritization in case of scarce datasets, and analyses concerning relevant features in datasets. Each of these areas deals with separate problems inherent to the field of bug prioritization.

Since we were unable to find precise causal relationships in the development of each of these directions, we describe them below on the basis of their popularity. The popularity of each direction is hereby determined by the number of scientific articles we found to be related to that specific direction. Fig 13 provides information about the popularity of each direction.

thumbnail

The numbers in brackets indicate the number of publications found for each branch.

https://doi.org/10.1371/journal.pone.0283838.g013

Prioritization on imbalance datasets.

According to Fig 13 , the most popular subtopic within bug prioritization is prioritization on imbalanced datasets. We believe that the popularity of this area is determined by the problem it subtends.

In brief, machine learning systems trained on imbalanced data will only perform well on samples with a dominant label. A dataset with a high number of low-priority bugs, for example, is more likely to classify subsequent bugs, even those with a high priority, as low priority. One of the researchers who also noted the importance of balancing the dataset is Thabtah [ 55 ]. Having explained the possible reasons for the popularity of this topic, we next move on to analyze its general structure as well as some of the most representative works we found that relate to it.

In light of the data we gathered, we can divide bug prioritization on imbalance datasets into two categories, based on the specific (machine learning) techniques used: those that use one predictor and those that use several predictors (what is known as the ensemble approach).

An example of work belonging to the former category is the work of Singha and Rossi [ 56 ]. The authors of this work used a modified version of Support Vector Machine (SVM) to weight classes based on the inverse occurrence of class frequencies. The results suggest that the model provides better prediction quality than standard SVM. Another example of the approach predicated by the former category is the work of Guo et al. [ 57 ]. In this study, the authors used Extreme Learning Machine (ELM) as a predictor. Several oversampling strategies were also tested.

The findings of this study are interesting for the purpose of this study because they indicate that the suggested approach can effectively balance an imbalanced dataset, which can contribute to increase the accuracy of bug prioritization.

Methods using a single predictor to deal with bug prioritization on imbalance datasets, as previously indicated, are not the only ones available. An example of work using multiple predictors (hence belonging to the latter category above-mentioned) is the work of Awad et al. [ 58 ].

The authors of this work proposed using the so-called ensemble method, in which each category of bug has its own predictor plus an additional general predictor for any type of bug. The topic of ensemble methods was described in more detail by Optiz and Maclin [ 59 ]. The peculiarity of the method is that any machine learning technique can be used as a predictor; the authors of the paper tested several techniques (such as Nave Bayes Multinomial (NBM), Random Forest (RF), and SVM). They also evaluated their proposed approach, which used both textual-based and non-textual datasets. Results showed that the proposed method can be successfully used to improve classical methods; however, this could be done only in the presence of a textual dataset.

Bug prioritization on scarce datasets.

A second subtopic we found within bug prioritization is prioritization in case of scarce data. Research in this area typically attempts to formulate methods capable of showing consistent and accurate results despite the availability of a relatively small amount of training data. The first work we found for bug prioritization in case of scarce data is the work of Sharma et al. [ 11 ], which was published in 2012. Several machine learning techniques, like SVM, Naive Bayes (NB), Neural Networks (NN), K-Nearest Neighbours (KNN), were tested to ascertain the best suitable and most accurate among them all. The authors showed that overall SVM and NN produce better results.

It is nevertheless worth noting that M. Sharma is the primary contributor to the topic, having published 5 of the 8 papers we found. We note that in this research group repeatedly utilized the same set of machine learning techniques (such as SVM, NB, NN, and KNN) [ 11 , 60 , 61 ]. We, therefore, acknowledge that this may lead to biased conclusions.

We also note that the techniques listed above are not all the techniques that are currently applied, used, or tested in the literature. For example, Zhang et al. [ 62 ] proposed using ELM, while Hernández-González et al. [ 63 ] used the Expectation Maximization (EM) Algorithm.

Analyses of relevant features in datasets.

This brings us to the discussion of the last subtopic within bug prioritization: analyses of relevant features in datasets. The goal of researchers in this field is to identify a set of features within a dataset that will yield the highest accuracy for a model trained on such data.

Although the topic is well-researched, there is still no consensus on the optimal set of attributes to be used. For example, the first publication in the domain by Alenezi and Banitaan [ 64 ] indicates that meta-data attributes are more relevant than textual description features. Sharmin et al. [ 65 ] also investigate the significance of features; however, they only compare two fields (text description and text conclusion).

Another perspective on the relevance and significance of features/attributes is offered by Sabor et al. [ 66 ]. The authors of this article proposed using stack traces as well as attributes, that were discussed by Alenezi and Banitaan [ 64 ]. More recently, a few works explored new ways for supplementing datasets with social-media information, for example, the work of Zhang et al. [ 67 ].

In light of the evidence reviewed above, we believe that the wide range of techniques and opinions developed in the literature thus far makes the task of identifying optimal qualities considerably challenging. Hence, we fear we are not in a position to make any specific recommendation with respect to this subtopic.

Issue prioritization

An issue (see https://docs.github.com/en/issues ) is an object that describes the work and the prerequisites for completing it. Any member of the open-source community can create an issue in order to enhance any given product. Issues in software development are typically found in platforms like GitHub, GitLab, or Bitbucket. Because of the novelty of these platforms, the subject has received little attention from the research community.

The main approach for dealing with issue prioritization has been that of predicting the lifetime of the issue itself. This approach was initially discussed by Murgia et al. [ 53 ]. The authors of this paper also investigate the impact of different types of developers’ activities (such as maintenance type, adding a new feature, refactoring, etc) on issue resolution time (or issue lifetime). These activities are often represented with labels. The results of this study show that fixing defects and implementing/improving new features is more effective and typically less time-consuming than other activities (such as testing or documenting).

The idea of using labels to represent activities also inspired other authors, such as Kikas et al. [ 68 ]. Subsequent work by Kallis et al. [ 69 ] confirmed the potential of this research direction and analyzed the relationship between static/dynamic features and issues’ lifetime.

Static features are those that remain consistent over time (for example, the number of issues created by the issue submitter in the three months before opening the issue). Dynamic features on the contrary are those features that change depending on when an observation is made (for example, if we look at the number of comments on an issue, we can see how it changes over time).

Another work that attempts to resolve issues prioritization by utilizing the concept of issue-lifetime prediction is the work of Dhasade et al. [ 70 ]. The authors of this work continued to use both static and dynamic attributes. They also expanded the previously developed approach by including in the model (and subsequently testing within it) various other hyperparameters (such as time and hotness). The changes implemented in the model by these researchers made the model more flexible and therefore capable of being adjusted to the needs of different teams.

As previously stated, the majority of articles predict the priority based on the expected issue lifetime. A slightly different strategy is however demonstrated by Kallis et al. [ 69 ], where labels are used by the authors to assist developers in the organization of their work (hence prioritizing their tasks). The method developed in this study can correctly and reliably anticipate one of three labels: bug, enhancement, or question.

PR prioritization

We cannot fully understand task prioritization if we do not discuss the third category that falls within it; namely, PR Prioritization. Research on PR Prioritization may be divided into two sub-topics, as shown in Fig 13 : integrator-oriented research and contributor-oriented research. An integrator is someone who is in charge of reviewing PRs, whereas a contributor is someone who creates PRs.

In the former category (integrator-oriented research), we may include the works of [ 54 , 71 – 73 ]. Van der Veen [ 71 ] offered a tool for prioritizing PRs based on static and dynamic attributes. This type of approach is quite similar to the work of Dhasade et al. [ 70 ]. In fact, Dhasade et al. [ 70 ] were inspired by this work and used it, as we have seen above, as a conceptual palette for their investigation.

A study by Yu et al. [ 54 ] proposes another approach to improving PR prioritization. The approach revolves around the idea of recommending appropriate reviewers to PRs. A description of the PR and a comment-network are two of the most crucial features used in this model. The comment network is a graph that is constructed based on the developers’ shared interests. Results from this study show that the method is capable of obtaining a 71 percent precision in predicting the appropriate reviewer.

Another method, that has similar goals, is discussed by Yu et al. [ 72 ]. The researchers developed an approach that is intended to aid in the prioritization of PRs, by forecasting their latency (i.e., evaluation time). To make such a prediction, the researchers took into consideration numerous socio-technical parameters (such as project age, team size, and total CI). The findings demonstrated that the length of the dialogue (the number of comments under the PR) had a substantial impact on its latency.

With respect to the latter category we introduced above (contributor-oriented research), we only found one relevant study by Azeem et al. [ 74 ]. In this study, the authors not only investigated the impact of each individual variable on the probability of a PR being merged, but they also formulated and developed a model capable of automatically estimating such a probability.

To obtain these results, the researchers used the XGBoost algorithm and over 50 different attributes. The mean average precision of their model for the first five recommended PRs was 95.3%, hovered at 89.6% for the first ten PRs, and eventually decreased to 79.6% for the first twenty PRs. The results show that the technique outperformed the baseline model presented by Gousios et al. [ 75 ] at all levels (for the first five, the first ten, and the first twenty PRs).

RQ2. Which methods are used in automatic task ranking models and approaches and how is their effectiveness assessed?

As we noted in the previous section, giving an exact definition of a task can be quite challenging, at least in software development. In our research, we found that the same approaches are usually employed to solve prioritization tasks for each of those different attributions. In other words, the observations we made for one meaning of the term invariably apply to the others. We speculate that the reason for this might be the presence of some sort of common fields or attributes between all these different meanings.

On these grounds and in light of the data presented in Fig 9 , we can conclude that Naive Bayes is the most frequently used technique for solving the problem of predicting bug severity and priority.

In this context, it is essential to note that, despite the growing popularity of neural network approaches in other areas of computer science, we did not entirely observe the same popularity in the studies we selected. We believe that the relative unpopularity of neural networks and deep neural networks [ 76 ] might be caused by the relatively small size of the dataset. Even though neural networks are very powerful tools, they require a lot of training to process data properly, which was not always possible, for different reasons, in the papers we reviewed.

As we pointed out in Fig 11 , the most used metrics for assessing the effectiveness of models are: f-score, precision, recall, and accuracy. This may indicate that the skewness of the datasets was relatively low [ 77 ]; however, closer scrutiny [ 60 , 64 , 78 ] reveals that this is not the case. Only nine studies employ the f-score as the only assessment metric. We can infer that in most cases additional metrics such as precision and recall are used in combination with the f-score to provide a complete and fair characterization of the model’s quality.

Based on the results shown in Further Clustering we can make another important observation. Among the most popular metrics observed in our review, we noticed that there were not any of those typically used for recommender systems [ 79 , 80 ]. Even though the question of task priority has a recommender nature, i.e., we want to know “what are the next tasks/features/bugs and how should be solved”; recommender ML techniques are not so popular. We believe that one of the reasons for that is that recommender system approaches [ 81 ] have only recently captured researchers’ attention. This speculation is reinforced by the observation concerning the number of papers published under the search query “recommender system” for the period 2010–2022 (data gathered from Scopus, Fig 14 ). The development of recommender systems can be a profitable way to guide existing works and the orientation of recommender systems can be used to gauge and support effective decision-making. This means that a model trained in this fashion cares about the ordering of output variables. So, if we have multiple entities and need only a small subset of the best of them, we most probably need to look at the solutions offered by the recommender system.

thumbnail

The bars show the number of papers on the topic published between 2010 and 2022.

https://doi.org/10.1371/journal.pone.0283838.g014

RQ3. What are the most effective and versatile models for automatic task ranking developed so far?

Giving a proper answer to the research questions proved to be more challenging due to the high variability of datasets and metrics used, and it may also depend upon the output variables selected (e.g. datasets). If we want to obtain reliable results, we should therefore compare results obtained on the same dataset [ 82 ]. Eclipse was the dataset of our choice. We chose Eclipse because it’s the most popular dataset, according to the findings we presented in our results section. We must also note that if the authors made a comparison and/or had multiple models in one of their works, we decided to consider only one model, the one with better performance and the highest results. The result of our comparisons is given in Tables 6 and 7 .

thumbnail

Priority levels are from Bugzilla [ 47 ]. The performance is described as precision/recall/f-score with the best results highlighted in bold. All data are shown as percentages.

https://doi.org/10.1371/journal.pone.0283838.t006

thumbnail

Severity levels are from Bugzilla [ 46 ]. The performance is described as precision/recall/f-score with the best results highlighted in bold. All data are shown as percentages.

https://doi.org/10.1371/journal.pone.0283838.t007

Table 6 shows a comparison between models with respect to their capacity for predicting bugs’ priority. We note that even though the number of papers using the same dataset is much higher, the comparison table has only five elements. That is because some papers either used different levels of priorities and metrics or gave only graphical information, so no accurate value for the metric could be gauged.

As Table 6 shows, the work by Pushpalatha et al. [ 85 ] has the highest amount of highlighted cells. This makes it the best approach concerning the Eclipse dataset.

Table 7 shows a severity-wise comparison among all the approaches reviewed.

We note that the highest score is attributed to the approach proposed by [ 87 ], while the second-highest result is achieved by the model proposed by [ 90 ]. This paper, based on the Naive Bayes ML algorithm, is interesting because it formulated a method capable of ensuring the most accurate result for the blocker bug, which is one of the most severe types of bugs typically found. We also note that works by [ 87 ] demonstrated better outcomes for critical bugs.

Critical review of our research question

Because the analysis we conducted above showed a limited number of research articles related to task prioritization in software development, this prompted us to make several assumptions and partially expand the scope of our study. Since the word “task” can refer to multiple concepts, we decided to consider this word in its most general meaning. This allowed us to gather more articles for our analysis. However, the fact that we only gathered a relatively limited number of scientific articles related to prioritization may indicate a significant gap in the research field. On the one hand, the presence of this gap may be taken as a sign of the relevance and novelty of this study for the research field. On the other hand, our findings raise several significant and pressing concerns. For example, why has so little research been conducted in this area? What are the pitfalls in task prioritization research? Why there was no demand for such a system? While we do not have a clear answer to all these questions, we can nevertheless assert that this SLR highlighted the need for more research in this area (task prioritization in software development) while also forming a solid basis for future progress in the field.

Analysis of RQ1: What are the existing approaches for automatic task ranking in software development?

We can make several important observations about the results we obtained. Firstly, earlier work dealt mainly with the problem of “bug” prioritization, which, albeit useful, is neither exhaustive nor comprehensive. That is so because we are interested in a broader understanding of task prioritization.

Secondly, only recently (especially in the last 5 years, as shown in Fig 12 ) researchers began to pay attention to the concept of “pull requests” prioritization and “issues” prioritization, which substantially expanded on original research conducted in the prioritization of Software Development metrics. As we discussed earlier on in this paper, this may well result in substantial growth of the literature in the near future, as it was the case with “bug prioritization” in the past.

Analysis of RQ2: Which methods are used in automatic task ranking models and approaches and how is their effectiveness assessed?

The number of methods described in this SLR for task prioritization is rather limited. The most popular method we observed is Naive Bayes. This method is important because it provides the most accurate result for the blocker bug. We also analyzed several different metrics found in the papers we reviewed, the most popular of which are: a) f-score, b) precision, c) recall, and d) accuracy.

Across the whole set of metrics, we found in the papers we reviewed for this SLR, CPU-costs related metrics are the rarest, which means that the question of computational costs has not been a priority. This may signal a new potential future research direction for task prioritization.

Based on the results presented above, we can argue that the lack of metrics commonly used in recommender systems represents an interesting research gap in the field, which is also shown in [ 93 ]. Our explanation for the existence of such a gap lies in the observation that there are still very few studies on recommender systems [ 81 ]. This is due, presumably, to the fact that recommender systems only recently attracted researchers’ attention.

Analysis of RQ3: What are the most effective and versatile models for automatic task ranking developed so far?

As we have shown above, the quality of task prioritization in software development has improved over the years as new and more accurate estimation methods have been deployed [ 87 ]. However, when it comes to prioritizing “pull requests” and “issues,” there are a lot of conflating strategies and ideas about what to prioritize [ 50 ]. Unfortunately, we have to admit that there isn’t a standardized or universally agreed approach for prioritizing such issues. Because of that, the only way to compare the approaches currently available is to fully reproduce and compare them on the same dataset and on the same set of metrics. This, however, is an extremely complicated task. Nevertheless, developing new works along this research direction may open up new vistas of vital importance for further progress in the field.

A synoptic summary

The brief synoptic summary of our results with respect to each of the research questions tackled in this study is the following:

  • RQ1 The number of articles that consider the problem of automatic task prioritization is still fairly small.
  • RQ2 Only a few articles, among those reviewed, assessed models in terms of time or CPU costs. Also, there is a sensitive lack of metrics used to analyze recommender systems.
  • RQ3 TSo far, there is no standardized approach or universal agreement on defining prioritization strategies for “pull requests” and “issues.” Presumably, this is because datasets are not marked up, meaning that there are no labels on data samples (see https://www.ibm.com/cloud/learn/data-labeling ).

Limitations, threats to validity, and review assessment

Limitations.

We start this subsection by briefly reviewing some of the obstacles that may have prevented an objective review.

In this study, we used five different databases, namely, Google Scholar, Microsoft Academic, ScienceDirect, IEEE Xplore, and ACM . A skeptical reader may point out that we should have used more databases (such as Springer Link, Web of Science, Scopus, ProQuest, Academic OneFile ). We certainly are aware of the existence of many more databases (besides those we selected) and could even agree that a larger set of databases might have broadened the scope of our work and increased the diversity of our searches. However, we picked those databases most commonly used by software engineers worldwide and that are known to aggregate the largest possible variety of papers. So, even though the list of databases used to perform our searches could have been ameliorated, we are pretty confident that our searches were scientifically sound.

Also, one may see as perhaps problematic the fact that we did not use any kind of grey literature in this work. Although there is a tendency to advocate for the usage of multivocal approaches (such as grey literature) in software engineering, we believe that such practice (given the dubious nature of such literature) should be limited to cases where there is a sensible lack of secondary sources, which was not our case.

Finally, one may rightly claim that in picking only works written in English, we somehow constrained and severely limited the scope and breadth of this research. Cross-cultural issues are emerging as vitally important in ensuring universalism in science. We agree with the importance of adding multicultural perspectives and even under-represented works in any study; however, most of the literature in the field is in English, and all the best journals only accept submissions in English. We, therefore, deem that the requirement we adopted in this SLR concerning language is pretty standard for the field and relatively unproblematic. Nevertheless, we note that our team of researchers is culturally very diverse, as it includes people from four different continents.

Threats to validity

In this subsection, we discuss a series of biases that might have affected the development and production of our review.

  • Bias towards Primary Sources —SLRs are usually performed on secondary sources. This is done to maximize objectivity. In this work, we uniquely relied on secondary sources; hence we avoided this potential bias.
  • Selection Bias —A major risk involved in any SLR is what we may call “selective outcome reporting” or “selection bias.” This typically occurs when the authors present only a selection of outcomes and/or results based on their statistical significance. We note that our reading log consists of only peer-reviewed, high-quality papers. The papers, as noted above, were published by world-leading publishers (such as Springer, Elsevier, ACM, and IEEE Xplore). In addition, we selected papers (methodologies, datasets, and metrics) from several journals as well as from reputable conference proceedings. This ensured a variety of levels of analysis and experimental protocols.
  • Bias in Synthesis —To avoid this type of bias, which can be considered as an extension of the Selection Bias above-mentioned, we carefully assessed our methodological protocol and -by extension- our findings. Thus, all the researchers involved in this study actively and consistently participated in monitoring each other’s activity to maximize objectivity and minimize mistakes (such as this bias).

Review assessment

Finally, we want to reflect on the overall quality of our work. To do so, we formulated—inspired by Kitchenham [ 94 ], a set of questions, which we critically applied to our results and findings. The questions we formulated and the answers we gave to them follow below.

  • Are the inclusion / exclusion criteria objective and reasonable? Following the best norms in our discipline, we formulated—before conducting our searches—a set of inclusion and exclusion criteria, which we subsequently applied to finalize the reading log. The criteria we formulated are congruent with those generally used in the field and are obviously relevant to the topic of our work.
  • Has there been a quality review? We developed a metric to assess the papers’ quality (Quality Assessment). We proved that the quality of the papers we included in the log was relatively high. Thus, we can confidently assert that the results that informed our work were scientifically sound and academically grounded.
  • Were the basic data / studies adequately described? We build a comprehensive literature log. The log consisted of all the relevant information we extracted from the papers we analyzed. This allowed us to process our data transparently and comprehensively. It also ensured the replicability of our findings, which is another key trait of any SLR.

This SLR investigated the problem of task prioritization in software development and focused on: a) identifying existing approaches for automatic task prioritization (RQ1), b) further investigating methods and metrics for task prioritization as developed in the literature (RQ2), and c) analyzing the effectiveness and reliability of these methods and metrics (RQ3).

Concerning RQ1, our results showed that earlier work mainly dealt with bug prioritization, and more recent work has expanded to consider prioritizing pull requests and issues. We speculate that this may lead to a substantial growth of literature in the future. RQ2 revealed that the most popular method used for task prioritization is Naive Bayes, while the most popular metrics used (in descending order) are f-score, precision, recall, and accuracy. However, there is a lack of metrics used in recommender systems, which may indicate a potential direction for future research. RQ3 showed that the quality of task prioritization in software development has improved over time; however, there is still a sensible lack of standardized approaches for prioritizing pull requests and issues.

In light of these findings, we can assert that this SLR contributed to broadening the field of research on task prioritization in software development, while also providing a solid basis for future research. Our goal in the mid-term is to develop an empirical study based on the topic of this SLR. Our aim in such a study would be to find a practical way to implement the findings of this review. To this extent, we shall consider whether it could be possible to develop algorithms for predicting task prioritization in a project using ML methods. This may well lead to novel AI-based management strategies, which could improve people’s well-being at work as well as foster moral and social good.

Nevertheless, IT practitioners should be cognizant of the relatively scarce amount of research conducted on task prioritization to date. They should also be aware of the absence of established methods for prioritizing pull requests and issues. They should therefore use the results of this SLR as a springboard for further explorations aimed at the development of such methods and tools.

Supporting information

S1 table. prisma 2020 checklist..

Template is taken from: www.prisma-statement.org/documents/PRISMA_2020_checklist.pdf .

https://doi.org/10.1371/journal.pone.0283838.s001

S2 Table. Quality scores assigned to the papers.

https://doi.org/10.1371/journal.pone.0283838.s002

  • View Article
  • Google Scholar
  • 2. IEEE Standard for Software Project Management Plans;.
  • 3. About issues; 2023. Available from: https://docs.github.com/en/issues/tracking-your-work-with-issues/about-issues .
  • 4. Anvik J. Automating Bug Report Assignment. In: Proceedings of the 28th International Conference on Software Engineering. ICSE’06. New York, NY, USA: Association for Computing Machinery; 2006. p. 937–940.
  • 7. Anvik J, Hiew L, Murphy GC. Who Should Fix This Bug? In: Proceedings of the 28th International Conference on Software Engineering. ICSE’06. New York, NY, USA: Association for Computing Machinery; 2006. p. 361–370.
  • 8. Panjer LD. Predicting Eclipse Bug Lifetimes. In: Fourth International Workshop on Mining Software Repositories (MSR’07:ICSE Workshops 2007); 2007. p. 29–29.
  • 9. Wang X, Zhang L, Xie T, Anvik J, Sun J. An approach to detecting duplicate bug reports using natural language and execution information. In: Proceedings of the 13th international conference on Software engineering—ICSE '08. ACM Press; 2008.
  • 10. Menzies T, Marcus A. Automated severity assessment of software defect reports. In: 2008 IEEE International Conference on Software Maintenance; 2008. p. 346–355.
  • 11. Sharma M, Bedi P, Chaturvedi KK, Singh VB. Predicting the priority of a reported bug using machine learning techniques and cross project validation. In: 2012 12th International Conference on Intelligent Systems Design and Applications (ISDA). IEEE; 2012.
  • 12. Tian Y, Lo D, Sun C. DRONE: Predicting Priority of Reported Bugs by Multi-factor Analysis. In: 2013 IEEE International Conference on Software Maintenance. IEEE; 2013.
  • 13. Illes-Seifert T, Herrmann A, Geisser M, Hildenbrand T. The Challenges of Distributed Software Engineering and Requirements Engineering: Results of an Online Survey. In: Proceedings : 1st Global Requirements Engineering Workshop—Grew’07 : in conjuction with the IEEE Conference on Global Software Engineering (ICGSE), Germany, Munich, 27th August 2007; 2007. p. 55–65.
  • 15. Demir K. A Survey on Challenges of Software Project Management. In: Software Engineering Research and Practice; 2009. p. 579–585.
  • 16. Gousios G, Storey MA, Bacchelli A. Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective. In: 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE); 2016. p. 285–296.
  • 17. Gousios G, Zaidman A, Storey MA, Deursen Av. Work Practices and Challenges in Pull-Based Development: The Integrator’s Perspective. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. vol. 1; 2015. p. 358–368.
  • 18. Barr ET, Bird C, Rigby PC, Hindle A, German DM, Devanbu P. Cohesive and Isolated Development with Branches. In: Fundamental Approaches to Software Engineering. Springer Berlin Heidelberg; 2012. p. 316–331.
  • PubMed/NCBI
  • 20. Wang H, Ma C, Zhou L. A Brief Review of Machine Learning and Its Application. In: 2009 International Conference on Information Engineering and Computer Science. IEEE; 2009.
  • 23. Somohano-Murrieta JCB, Ocharan-Hernandez JO, Sanchez-Garcia AJ, de los Angeles Arenas-Valdes M. Requirements Prioritization Techniques in the last decade: A Systematic Literature Review. In: 2020 8th International Conference in Software Engineering Research and Innovation (CONISOFT). IEEE; 2020.
  • 25. Sufian M, Khan Z, Rehman S, Butt WH. A Systematic Literature Review: Software Requirements Prioritization Techniques. In: 2018 International Conference on Frontiers of Information Technology (FIT). IEEE; 2018.
  • 27. Alfayez R, Alwehaibi W, Winn R, Venson E, Boehm B. A systematic literature review of technical debt prioritization. In: Proceedings of the 3rd International Conference on Technical Debt. ACM; 2020.
  • 28. Ijaz KB, Inayat I, Bukhsh FA. Non-functional Requirements Prioritization: A Systematic Literature Review. In: 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE; 2019.
  • 33. Fatima T, Azam F, Anwar MW, Rasheed Y. A Systematic Review on Software Project Scheduling and Task Assignment Approaches. In: Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence. ACM; 2020.
  • 39. Kitchenham BA, Charters S. Guidelines for performing Systematic Literature Reviews in Software Engineering. Keele University and Durham University Joint Report; 2007. Available from: https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf .
  • 45. Shoham Y, Perrault R, Brynjolfsson E, Clark J, Manyika J, Niebles JC, et al.. The AI Index 2018 Annual Report; 2018. Available from: https://hai.stanford.edu/sites/default/files/2020-10/AI_Index_2018_Annual_Report.pdf .
  • 46. Lauhakangas I, Raal R, Iversen J, Tryon R, Goncharuk L, Roczek D. QA/Bugzilla/Fields/Severity; 2021. Available from: https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Severity .
  • 47. Kanat-Alexander M, Miller D, Humphries E. Bugzilla:Priority System; 2021. Available from: https://wiki.mozilla.org/Bugzilla:Priority_System .
  • 49. Yang G, Zhang T, Lee B. Towards Semi-automatic Bug Triage and Severity Prediction Based on Topic Model and Multi-feature of Bug Reports. In: 2014 IEEE 38th Annual Computer Software and Applications Conference. IEEE; 2014.
  • 50. Freedman D, Pisani R, Purves R. Statistics (international student edition). Pisani , Purves R, 4th edn WW Norton & Company, New York. 2007;.
  • 52. Jeong G, Kim S, Zimmermann T. Improving bug triage with bug tossing graphs. In: Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering. ACM; 2009.
  • 53. Murgia A, Concas G, Tonelli R, Ortu M, Demeyer S, Marchesi M. On the influence of maintenance activity types on the issue resolution time. In: Proceedings of the 10th International Conference on Predictive Models in Software Engineering. ACM; 2014.
  • 54. Yu Y, Wang H, Yin G, Ling CX. Reviewer Recommender of Pull-Requests in GitHub. In: 2014 IEEE International Conference on Software Maintenance and Evolution. IEEE; 2014.
  • 56. Roy NKS, Rossi B. Cost-Sensitive Strategies for Data Imbalance in Bug Severity Classification: Experimental Results. In: 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE; 2017.
  • 58. Awad MA, ElNainay MY, Abougabal MS. Predicting bug severity using customized weighted majority voting algorithms. In: 2017 Japan-Africa Conference on Electronics, Communications and Computers (JAC-ECC). IEEE; 2017.
  • 61. Sharma M, Kumari M, Singh VB. Bug Priority Assessment in Cross-Project Context Using Entropy-Based Measure. In: Algorithms for Intelligent Systems. Springer Singapore; 2020. p. 113–128.
  • 64. Alenezi M, Banitaan S. Bug Reports Prioritization: Which Features and Classifier to Use? In: 2013 12th International Conference on Machine Learning and Applications. IEEE; 2013.
  • 65. Sharmin S, Aktar F, Ali AA, Khan MAH, Shoyaib M. BFSp: A feature selection method for bug severity classification. In: 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC). IEEE; 2017.
  • 67. Zhang Y, Yin G, Wang T, Yu Y, Wang H. Evaluating Bug Severity Using Crowd-based Knowledge. In: Proceedings of the 7th Asia-Pacific Symposium on Internetware. ACM; 2015.
  • 68. Kikas R, Dumas M, Pfahl D. Using dynamic and contextual features to predict issue lifetime in github projects. In: 2016 ieee/acm 13th working conference on mining software repositories (msr). IEEE; 2016. p. 291–302.
  • 70. Dhasade AB, Venigalla ASM, Chimalakonda S. Towards Prioritizing GitHub Issues. In: Proceedings of the 13th Innovations in Software Engineering Conference on Formerly known as India Software Engineering Conference. ACM; 2020.
  • 71. van der Veen E, Gousios G, Zaidman A. Automatically Prioritizing Pull Requests. In: 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE; 2015.
  • 72. Yu Y, Wang H, Filkov V, Devanbu P, Vasilescu B. Wait for It: Determinants of Pull Request Evaluation Latency on GitHub. In: 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE; 2015.
  • 73. Yu S, Xu L, Zhang Y, Wu J, Liao Z, Li Y. NBSL: A Supervised Classification Model of Pull Request in Github. In: 2018 IEEE International Conference on Communications (ICC). IEEE; 2018.
  • 74. Azeem MI, Peng Q, Wang Q. Pull Request Prioritization Algorithm based on Acceptance and Response Probability. In: 2020 IEEE 20th International Conference on Software Quality, Reliability and Security (QRS). IEEE; 2020.
  • 75. Gousios G, Pinzger M, van Deursen A. An exploratory study of the pull-based software development model. In: Proceedings of the 36th International Conference on Software Engineering. ACM; 2014.
  • 77. Jeni LA, Cohn JF, Torre FDL. Facing Imbalanced Data–Recommendations for the Use of Performance Metrics. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE; 2013.
  • 78. Zhang W, Challis C. Automatic Bug Priority Prediction Using DNN Based Regression. In: Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. Springer International Publishing; 2019. p. 333–340.
  • 81. Ricci F, Rokach L, Shapira B. Introduction to Recommender Systems Handbook. In: Recommender Systems Handbook. Springer US; 2010. p. 1–35.
  • 82. Lamkanfi A, Perez J, Demeyer S. The Eclipse and Mozilla defect tracking dataset: A genuine dataset for mining bug information. In: 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE; 2013.
  • 87. Zhang T, Yang G, Lee B, Chan ATS. Predicting severity of bug report by mining bug repository with concept profile. In: Proceedings of the 30th Annual ACM Symposium on Applied Computing. ACM; 2015.
  • 88. Tian Y, Lo D, Sun C. Information Retrieval Based Nearest Neighbor Classification for Fine-Grained Bug Severity Prediction. In: 2012 19th Working Conference on Reverse Engineering. IEEE; 2012.
  • 93. Happel HJ, Maalej W. Potentials and challenges of recommendation systems for software development. In: Proceedings of the 2008 international workshop on Recommendation systems for software engineering. ACM; 2008.

Systematic Literature Review on Global Software Development Risks in Agile Methodology

Ieee account.

  • Change Username/Password
  • Update Address

Purchase Details

  • Payment Options
  • Order History
  • View Purchased Documents

Profile Information

  • Communications Preferences
  • Profession and Education
  • Technical Interests
  • US & Canada: +1 800 678 4333
  • Worldwide: +1 732 981 0060
  • Contact & Support
  • About IEEE Xplore
  • Accessibility
  • Terms of Use
  • Nondiscrimination Policy
  • Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

AIP Publishing Logo

  • Previous Article
  • Next Article

Lean practices in software development projects: A literature review

Email: [email protected]

  • Article contents
  • Figures & tables
  • Supplementary Data
  • Peer Review
  • Reprints and Permissions
  • Cite Icon Cite
  • Search Site

RamKaran Yadav , M. L. Mittal , Rakesh Jain; Lean practices in software development projects: A literature review. AIP Conf. Proc. 3 September 2019; 2148 (1): 030044. https://doi.org/10.1063/1.5123966

Download citation file:

  • Ris (Zotero)
  • Reference Manager

Rapidly increasing customer demands, competition, continuous changing scenario and accelerating pace of technological developments have put tremendous pressure on the business organization to deliver quality products at lower cost. On the same lines the software development (SD) companies need to deliver quality codes with new features at reduced cost. This can be achieved through Lean to software development projects. As the lean has been considered in different ways and has been implemented to varying extent in different sectors of the economy this paper aims to investigate as to how “lean” is viewed in software development projects and status of implementation in software development projects. First, application of lean in different types of projects viz. construction, healthcare, aerospace, new product development and service is discussed. Secondly, application of lean to SD projects is investigated at three levels: philosophy, principles (value, value stream, flow, pull and perfection) and practices/tools. The effect of lean on performance (inventory, lead time, customer satisfaction, cost, and business value) of SD projects is also analyzed. Further, “Leagile” software development and agile dominance is explored through this paper.

Sign in via your Institution

Citing articles via, publish with us - request a quote.

literature review for software developers

Sign up for alerts

  • Online ISSN 1551-7616
  • Print ISSN 0094-243X
  • For Researchers
  • For Librarians
  • For Advertisers
  • Our Publishing Partners  
  • Physics Today
  • Conference Proceedings
  • Special Topics

pubs.aip.org

  • Privacy Policy
  • Terms of Use

Connect with AIP Publishing

This feature is available to subscribers only.

Sign In or Create an Account

Book cover

Second International Conference on Sustainable Technologies for Computational Intelligence pp 119–134 Cite as

A Systematic Literature Review on Software Development Estimation Techniques

  • Prateek Srivastava 18 ,
  • Nidhi Srivastava 18 ,
  • Rashi Agarwal 19 &
  • Pawan Singh 20  
  • Conference paper
  • First Online: 19 October 2021

388 Accesses

Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1235)

Project Failure is the key target undergoing in today’s scenario which is observed through the software project administrators. An ambiguity of the estimation is the major cause of this dilemma. According to tech project managers, project failure is the most common concern today. This issue arises as a result of measurement inaccuracy. When software's size and significance increase, its complexity makes it even more problematic to reliably estimate the cost of growth. This was the predicament of previous years. Software cost estimation (SCE) is the utmost significant subject in software design in modern times. Real calculation necessitates expense and commitment considerations when developing applications utilizing algorithmic or Artificial Intelligence (AI) techniques. In this paper, numerous predefined approaches for SCE are illuminated and their characteristics will be considered. This paper briefs numerous sessions of SCE models and methods. This study also claims that no particular procedure is superlative for all circumstances, software measures and metrics are also needed to enhance efficiency. This research paper focuses on Artificial Intelligence-based software cost estimation approaches, their implementation domain, and strategies for calculating software cost estimation. There are two kinds of SCE approaches: parametric/algorithmic models and non-parametric/non-algorithmic models. One of the most common algorithmic SCE models is the Constructive Cost Model (COCOMO). Basic COCOMO is ideal for fast, quick, and uneven estimates of the effort expected to manufacture software, but its accuracy is restricted because of the lack of variables to account for cost-driven differences. Non-algorithmic approaches for overcoming the shortcomings of algorithmic models include expert Judgment, machine learning (ML), and price to gain. In checked studies, ML models were implemented to address the shortcomings of parametric estimation models.

  • Software cost estimation (SCE)
  • Estimation techniques
  • Artificial intelligence
  • Data mining
  • Software metrics

This is a preview of subscription content, log in via an institution .

Buying options

  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
  • Available as EPUB and PDF
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

N.E. Fenton, S.L. Pfleeger, Software metrics: a rigorous and practical approach. PWS Publishing Co (1997)

Google Scholar  

V.R. Montequín, J. Villanueva Balsera, C. Alba González, G. Martínez Huerta, Software project cost estimation using AI techniques, in Proceedings of the 5th WSEAS/IASME International Conference on Systems Theory and Scientific Computation , pp. 289–293 (2005)

J.M. Minguet, J.F. Hernández, La calidad del software y sumedida (2003)

R. Tripathi, P.K. Rai, Comparative study of software cost estimation technique. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 6 (1) (2016)

D. Manikavelan, R. Ponnusamy, Software quality analysis based on cost and error using fuzzy combined COCOMO model. J. Amb. Intell. Human. Comput. 1–11 (2020)

P. Phannachitta, On an optimal analogy-based software effort estimation. Inf. Softw. Technol. 125 , 106330 (2020)

Q. Xiaotie, M. Fang, Summarization of software cost estimation. Proc. Eng. 15 , 3027–3031 (2011)

S.A. Deshmukh, S.W. Ahmad, Using classification data mining techniques for software cost estimation. 2229–5518 (2016)

V.R.Arulmozhi, B. Vijaya Nirmala, N. Deepa, A software cost estimation approach using Least Square-Support Vector Machine Technique. 2395–695X (2015)

Z.A. Khalifelu, F.S. Gharehchopogh, Comparison and evaluation of data mining techniques with algorithmic models in software cost estimation. Proc. Technol. 1 , 65–71 (2012)

S. Shivangi, U. Kumar, Review of various software cost estimation techniques. Int. J. Comput. Appl. 141 (11), 31–34 (2016)

R. Poonam, S. Jain, Enhanced software effort estimation using multi layered feed forward artificial neural network technique. Proc. Comput. Sci. 89 , 307–312 (2016)

Article   Google Scholar  

S. Densumite, P. Muenchaisri, Software size estimation using activity point, in IOP Conference Series: Materials Science and Engineering , vol. 185, no. 1, p. 012013. IOP Publishing (2017)

O.P. Sangwan, Software effort estimation using machine learning techniques, in 2017 7th International Conference on Cloud Computing, Data Science & Engineering-Confluence , pp. 92–98. IEEE (2017)

P.A. Whigham, C.A. Owen, S.G. Macdonell, A baseline model for software effort estimation. ACM Trans. Softw. Eng. Methodol. (TOSEM) 24 (3), 1–11 (2015)

F. Sarro, A. Petrozziello, M. Harman, Multi-objective software effort estimation, in 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE) , pp. 619–630. IEEE (2016)

Y. Masoudi-Sobhanzadeh, H. Motieghader, A. Masoudi-Nejad, Feature select: a software for feature selection based on machine learning approaches. BMC Bioinform. 20 (1), 170 (2019)

P. Pospieszny, B. Czarnacka-Chrobot, A. Kobylinski, An effective approach for software project effort and duration estimation with machine learning algorithms. J. Syst. Softw. 137 , 184–196 (2018)

M.M. Al Asheeri, M. Hammad, Machine learning models for software cost estimation, in 2019 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT) , pp. 1–6. IEEE (2019)

B. Baskeles, B. Turhan, A. Bener, Software effort estimation using machine learning methods, in 2007 22nd International Symposium on Computer and Information Sciences , pp. 1–6. IEEE (2007)

M.Z. Tunio, H. Luo, C. Wang, F. Zhao, A.R. Gilal, W. Shao, Task assignment model for crowdsourcing software development: TAM. J. Inf. Process. Syst. 14 (3) (2018)

M. Lawal, P. Genevès, N. Layaïda, A cost estimation technique for recursive relational algebra, in Proceedings of the 29th ACM International Conference on Information & Knowledge Management , pp. 3297–3300 (2020)

S.Z. Iqbal, M. Idrees, A.B. Sana, N. Khan, Comparative analysis of common software cost estimation modelling techniques. 33 (2017)

M. Ullah, R. Ali, M. Ahmad, T. Khan, F. UlMulk, Software cost estimation–a comparative study of COCOMO-II and Bailey-Basili Models, in 2019 International Conference on Advances in the Emerging Computing Technologies (AECT) , pp. 1–5. IEEE (2020)

L.R. Nerkar, P.M. Yawalkar, Software cost estimation using algorithmic model and non-algorithmic model a review. Int. J. Comput. Appl. 2 , 4–7 (2014)

G. Singh, D. Singh, V. Singh, A study of software metrics. IJCEM Int. J. Comput. Eng. Manage. 11 , 22–27 (2011)

Download references

Author information

Authors and affiliations.

Amity Institute of Information Technology, Amity University Uttar Pradesh, Lucknow Campus, Lucknow, India

Prateek Srivastava & Nidhi Srivastava

Department of Information Technology, University Institute of Engineering and Technology, Chhatrapati Shahu Ji Maharaj University, Kanpur, India

Rashi Agarwal

Department of Computer Science & Engineering, Amity School of Engineering and Technology, Amity University Uttar Pradesh, Lucknow Campus, Lucknow, India

Pawan Singh

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

The PNG University of Technology, Lae, Papua New Guinea

Ashish Kumar Luhach

CHRIST (Deemed to be University), Bangalore, India

Ramesh Chandra Poonia

University of Eastern Finland, Kuopio, Finland

Xiao-Zhi Gao

Namibia University of Science and Technology, Windhoek, Namibia

Dharm Singh Jat

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Srivastava, P., Srivastava, N., Agarwal, R., Singh, P. (2022). A Systematic Literature Review on Software Development Estimation Techniques. In: Luhach, A.K., Poonia, R.C., Gao, XZ., Singh Jat, D. (eds) Second International Conference on Sustainable Technologies for Computational Intelligence. Advances in Intelligent Systems and Computing, vol 1235. Springer, Singapore. https://doi.org/10.1007/978-981-16-4641-6_11

Download citation

DOI : https://doi.org/10.1007/978-981-16-4641-6_11

Published : 19 October 2021

Publisher Name : Springer, Singapore

Print ISBN : 978-981-16-4640-9

Online ISBN : 978-981-16-4641-6

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Publish with us

Policies and ethics

  • Find a journal
  • Track your research

Help | Advanced Search

Computer Science > Software Engineering

Title: systematic mapping protocol -- ux design role in software development process.

Abstract: A systematic mapping protocol is a method for conducting a literature review in a rigorous and transparent way. It aims to provide an overview of the current state of research on a specific topic, identify gaps and opportunities, and guide future work. In this document, we present a systematic mapping protocol for investigating the role of the UX designer in the software development process. We define the research questions, scope, sources, search strategy, selection criteria, data extraction, and analysis methods that we will use to conduct the mapping study. Our goal is to understand how the UX designers collaborate with other stakeholders, what methods and tools they use, what challenges they face, and what outcomes they achieve in different contexts and domains.

Submission history

Access paper:.

  • Download PDF

license icon

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

BibSonomy logo

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

  • Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

IMAGES

  1. (PDF) The Well-being of Software Developers: A Systematic Literature Review

    literature review for software developers

  2. (PDF) Benefits and limitations of automated software testing

    literature review for software developers

  3. review of literature research paper

    literature review for software developers

  4. FREE 8+ Sample Literature Review Templates in PDF

    literature review for software developers

  5. (PDF) Software Development Project Management: A Literature Review

    literature review for software developers

  6. literature review article examples Sample of research literature review

    literature review for software developers

VIDEO

  1. Writing Literature review on history 2024

  2. Literature review and its process

  3. Module 1-4 Literature Review

  4. Literature Review In ONE Day

  5. Literature Review & Technical Reading

  6. Literature Review Using Free Tools -- Pubmed

COMMENTS

  1. Systematic literature reviews in agile software development: A tertiary study

    We found 28 SLRs focusing on ten different ASD research areas: adoption, methods, practices, human and social aspects, CMMI, usability, global software engineering (GSE), organizational agility, embedded systems, and software product line engineering.

  2. Agile Global Software Development: A Systematic Literature Review

    Agile Global Software Development: A Systematic Literature Review Authors: Rafael Camara , Annelyelthon Alves , Iury Monte , Marcelo Marinho Authors Info & Claims SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software EngineeringOctober 2020Pages 31-40 https://doi.org/10.1145/3422392.3422411

  3. Skills development for software engineers: Systematic literature review

    A Systematic Literature Review was performed on six databases, resulting in 56 selected articles identifying the soft skills and the teaching methodologies desired to train Software Engineers. These were the base for the proposed framework. We proposed a grouping of soft skills found in the literature totaling 33 soft skills.

  4. Prioritizing tasks in software development: A systematic literature review

    Task prioritization is one of the most researched areas in software development. Given the huge number of papers written on the topic, it might be challenging for IT practitioners-software developers, and IT project managers-to find the most appropriate tools or methods developed to date to deal with this important issue. The main goal of this work is therefore to review the current state ...

  5. PDF Documentation Practices in Agile Software Development: A Systematic

    tools will be helpful in future research and development. Index Terms—Software Engineering; Agile Software Develop-ment; Documentation; Systematic Literature Review I. INTRODUCTION Software documentation is an integral part of software development. It works as a communication medium between developers of a team and is utilized as an ...

  6. Best Practices for Software Development: A Systematic Literature Review

    Here we report the conduction of a systematic literature review (SLR) that aims to identify (a) what best practices are, (b) what are their distinctive characteristics; (c) which methods, techniques or strategies are used to identify them, and (d) how their performance is evaluated.

  7. Software Practices For Agile Developers: A Systematic Literature Review

    Software Practices For Agile Developers: A Systematic Literature Review Abstract: Software development is one of the work practices in a company's startup, academics, and industries. Agile is a software development methodology that is currently quite popular.

  8. Software Development Analytics in Practice: A Systematic Literature Review

    This systematic literature review aims at providing an aggregate view of the relevant studies on Software Development Analytics in the past decade, with an emphasis on its application in practical settings.

  9. Best Practices for Software Development: A Systematic Literature Review

    Here we report the conduction of a systematic literature review (SLR) that aims to identify (a) what best practices are, (b) what are their distinctive characteristics; (c) which methods,...

  10. A Systematic Literature Review on Global Software Development Life

    Global software development (GSD) has now become a prominent software development paradigm. ... In this paper, we report our findings through systematic literature review that aimed at identifying the challenges faced by the globally distributed teams during various phases of software development. We have also discussed suggested best practices ...

  11. Analysing app reviews for software engineering: a systematic literature

    App reviews found in app stores can provide critically valuable information to help software engineers understand user requirements and to design, debug, and evolve software products. Over the last ten years, a vast amount of research has been produced to study what useful information might be found in app reviews, and how to mine and organise such information as efficiently as possible. This ...

  12. PDF Secure Software Development Methodologies: A Multivocal Literature Review

    Secure Software Development Methodologies: A Multivocal Literature Review Arina Kudriavtseva, Olga Gadyatskaya Abstract—In recent years, the number of cyber attacks has grown rapidly. An effective way to reduce the attack surface and protect software is adoption of methodologies that apply security at each step of the software development ...

  13. Software Development Project Management: A Literature Review

    Software Development Project Management: A Literature Review Conference: Proceedings of the 26th National ASEM Conference: Organizational Transformation: Opportunities and Challenges (pp....

  14. The Well-being of Software Developers: A Systematic Literature Review

    The Well-being of Software Developers: A Systematic Literature Review Authors: Povilas Godliauskas Mykolas Romeris University Abstract and Figures Context: Organizational studies show that...

  15. PDF Machine Learning Model Development from a Software Engineering

    A Systematic Literature Review Giuliano Lorenzoni David R. Cheriton School of Computer Science University of Waterloo (UW) Waterloo, Canada [email protected] ... procedures for software development, for example to scale it up for large systems and guarantee high-quality software with low cost software production. Indeed, SE has progressed

  16. Systematic Literature Review on Global Software Development Risks in

    Method: This review paper using the standard systematic literature review method by Kitchehamm by reviewing and analyzing the relevant state-of -art techniques and approaches in the journal libraries based on the research questions. Results: The findings show that communication in Agile Global Software development is the main risk challenge.

  17. academic writing

    1 Answer Sorted by: 1 Any source that you use in your work, be it academic literature, websites, or even tv programs, needs to be cited. Similar projects are extremely relevant to what you're doing, and therefore if your source on them is a website rather than an academic work, it should definitely be cited.

  18. PDF Democratizing Software Development: A Systematic Multivocal Literature

    A Systematic Multivocal Literature Review and Research Agenda on Citizen Development Björn Binzer1(B) and Till J. Winkler1,2 1 Chair of Information Management, University of Hagen, Hagen, Germany ... and where software developers and archi-tects are in biggest demand [5]. This IT talent gap is widening globally and is expected

  19. Systematic literature review on software architecture of educational

    The purpose of this systematic literature review (SLR) is to analyse the software architectures used in educational websites, methodologies, technological components and empirical results. The search of the SLR yielded 23 studies from the most significant academic sources in education and software engineering.

  20. Software Development Analytics in Practice: A Systematic Literature Review

    Objective: This systematic literature review aims at providing an aggregate view of the relevant studies on Software Development Analytics in the past decade, with an emphasis on its...

  21. Lean practices in software development projects: A literature review

    Lean practices in software development projects: A literature review | AIP Conference Proceedings | AIP Publishing Research Article | September 03 2019 Lean practices in software development projects: A literature review RamKaran Yadav; M. L. Mittal; Rakesh Jain Author & Article Information AIP Conf. Proc. 2148, 030044 (2019)

  22. PDF Large Language Models for Software Engineering: A Systematic Literature

    development, software testing, software maintenance, and software management. •We have summarised key challenges that using LLMs encounters within the SE field and have suggested several potential research directions for LLMs4SE. Section2presents our research questions (RQs) and elaborates on our systematic literature review (SLR) methodology.

  23. A Systematic Literature Review on Software Development Estimation

    A Systematic Literature Review on Software Development Estimation Techniques Prateek Srivastava, Nidhi Srivastava, Rashi Agarwal & Pawan Singh Conference paper First Online: 19 October 2021 386 Accesses Part of the Advances in Intelligent Systems and Computing book series (AISC,volume 1235) Abstract

  24. Systematic Mapping Protocol -- UX Design role in software development

    A systematic mapping protocol is a method for conducting a literature review in a rigorous and transparent way. It aims to provide an overview of the current state of research on a specific topic, identify gaps and opportunities, and guide future work. In this document, we present a systematic mapping protocol for investigating the role of the UX designer in the software development process ...