Choosing which tests run first — Technical https://thesis.uya.no/proposals/test-what-matters-first/#technical-track BACHELOR Final result: A test-ordering script and reproducible benchmark report. Task: Implement an established shortest-test-first order and display the resulting pytest run. Data: Flask 3.1.0 tests + 30 non-equivalent mutations (mixed) Record outcomes and durations, including survivors. finalise rankings using 10 history mutants before 20 evaluation mutants at non-overlapping mutation sites; history is artificial. Requires: Stable isolated test infrastructure, mutation checking and developers. Method: Use the pinned Flask tests and mutation fixture; measure time to first failure and check that reordering preserves results. Plan 4 participant sessions. STEPS Build and test 1. Read the starting sources and choose one established implementation method. 2. Write the requirements, data fields, system diagram and test cases. 3. Prepare Flask 3.1.0 tests + 30 non-equivalent mutations. Make the answer key and pass criteria before testing. 4. Build a working version of a test-ordering script and reproducible benchmark report. 5. Run function, integration and failure-case tests. Record each result. 6. Run the practical evaluation and list the changes the system still needs. MASTER Final result: Five-order pytest scheduler, fixed replay dataset, NAPFD/APFDc benchmark and decision dashboard. Research question: Investigate the trade-off between fast failures and incomplete assurance. Compare prioritization baselines on held-out mutation sites under equal wall-clock budgets; analyse censored failures and developers’ interpretation of partial coverage. Research result: A reproducible account of prioritization benefits and decision risks under a fixed budget. STEPS Design, build and test (DSR) 1. Read the newest papers and list the closest existing systems. 2. Write down the versions, fields, data split, case assignment, random seeds and correct answers for Flask 3.1.0 tests + 30 non-equivalent mutations. 3. Draw the user workflow, data model and system architecture. List the requirements and pass criteria. 4. Build a working version of five-order pytest scheduler, fixed replay dataset, napfd/apfdc benchmark and decision dashboard. 5. Test every function, connection and failure case. Save the failed tests as well as the passed tests. 6. Compare the system with the named alternative. Then run the user task or decision task in the assignment. 7. Report the measured result, the failed cases and the design lessons another team can reuse. CURRENT PROJECT LITERATURE Faster Feedback with AI? A Test Prioritization Study (2024, peer-reviewed): https://doi.org/10.1145/3660829.3660837 Compares modern and simple test-prioritization methods using change-based mutations. Feature-oriented test case selection and prioritization during the evolution of highly-configurable systems (2024, peer-reviewed): https://doi.org/10.1016/j.jss.2024.112157 Provides current evidence on selection overhead and failure detection under test budgets. IS THEORY STARTING POINTS Pan et al. (2021) — Test Selection and Prioritization: A Systematic Review: https://doi.org/10.1007/s10664-021-10066-6 Compare prioritization features, baselines and cost-sensitive evaluation measures. Vessey & Galletta (1991) — Cognitive Fit: https://pubsonline.informs.org/doi/10.1287/isre.2.1.63 Test whether a representation helps one kind of task more than another. Search Scopus or Web of Science and ACM Digital Library using the topic query, then follow citations to the thesis start date. Record searches and compare methods, data, findings and limitations in literature-matrix.csv. Use that review to confirm or revise the gap and choose a current comparator. The linked papers are starting points. PILOT QUESTION Can earlier failure feedback improve developers’ understanding of what remains unverified? EXAMPLE TOOLS Python, Flask 3.1.0, pytest, pandas Equivalent tools are fine. FIRST THREE ACTIONS 1. Check out Flask 3.1.0; run the unchanged suite and save the collected test order. 2. Create one mutation, measure test durations and verify which tests detect it. 3. Separate history from evaluation mutations and implement replay without reading future outcomes. LITERATURE SEARCH continuous integration test prioritisation developer feedback decisions DATA Flask 3.1.0 tests + 30 non-equivalent mutations (mixed) Record outcomes and durations, including survivors. finalise rankings using 10 history mutants before 20 evaluation mutants at non-overlapping mutation sites; history is artificial. Flask 3.1.0: https://github.com/pallets/flask/releases/tag/3.1.0 HOW TO TEST IT Verify full-run result preservation and no future-outcome leakage. Under the same 25% wall-time budget report NAPFD/APFDc, detected fraction, completed tests and right-censored time-to-first-failure. Four release-experienced developers compare dashboard/raw-log evidence and make policy-scored merge/defer decisions. FINAL RESULT Five-order pytest scheduler, fixed replay dataset, NAPFD/APFDc benchmark and decision dashboard. ACCESS OR PEOPLE Stable isolated test infrastructure, mutation checking and 4 developers. TEMPLATES https://thesis.uya.no/starters/test-what-matters-first/technical/data-plan.csv https://thesis.uya.no/starters/test-what-matters-first/technical/evaluation.csv https://thesis.uya.no/starters/test-what-matters-first/technical/literature-matrix.csv Use participant codes instead of names or email addresses.