Reviewing AI-generated code — Non-technical https://thesis.uya.no/proposals/review-with-reasons/#nontechnical-track BACHELOR Final result: A review checklist grounded in observed mistakes. Task: Identify the checking steps Python-capable final-year IS students use when reviewing AI-generated code. Data: 12 printed Python packets (planned) 6 checked-correct and 6 faulty functions; 3 faults are missed by displayed tests. Hide labels and keep code identical between evidence-present/absent versions. Requires: Recruit Python-capable final-year IS students. Reading code is required; writing software is not. Method: Use printed code/test packs in think-aloud sessions; classify missed checks and useful evidence. Plan 4 participant sessions. STEPS Study and improve 1. Read the starting sources and write the exact information problem. 2. Prepare 12 printed Python packets and a separate answer sheet or coding sheet. 3. Run one pilot and fix unclear questions. 4. Collect the named evidence with consent. 5. Group the findings with the stated categories and check the answer sheet. 6. Produce a review checklist grounded in observed mistakes. List the three most useful changes. MASTER Final result: Review checklist and findings on misplaced trust. Research question: Explain overreliance on plausible AI code among novice reviewers despite contradictory evidence. Contrast verification-cost and competence-belief explanations using task observations and follow-up interviews. Research result: A specific account of evidence use in novice AI-code review. STEPS Study, compare and explain 1. Read the newest papers and write one exact research question. 2. State which people, cases or documents you will study and what you will compare. 3. Write the selection rules, questions and analysis steps for 12 printed Python packets + 8 Python-capable final-year IS students. 4. Run one pilot. Fix unclear questions or categories, then keep the guide unchanged. 5. Collect the named interviews, cases or documents with consent. 6. Analyse them with the stated comparison or coding method. Keep disagreements and missing data. 7. Report the answer, the evidence and the practical output named in the assignment. CURRENT PROJECT LITERATURE LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluation (2024, peer-reviewed): https://doi.org/10.1109/TSE.2024.3428972 Shows that test-mediated interaction can improve evaluation of AI-generated code. Can LLM Replace Stack Overflow? A Study on Robustness and Reliability of Large Language Model Code Generation (2024, peer-reviewed): https://doi.org/10.1609/aaai.v38i19.30185 Demonstrates that executable generated code can still contain substantial API misuse. IS THEORY STARTING POINTS Buçinca et al. (2021) — To Trust or to Think: https://doi.org/10.1145/3449287 Derive a competing explanation based on verification effort and overreliance. Vasconcelos et al. (2023) — Explanations Can Reduce Overreliance: https://doi.org/10.1145/3579605 Compare verification cost with trust as explanations of observed decisions. Search Scopus or Web of Science and ACM Digital Library using the topic query, then follow citations to the thesis start date. Record searches and compare methods, data, findings and limitations in literature-matrix.csv. Use that review to confirm or revise the gap and choose a current comparator. The linked papers are starting points. PILOT QUESTION When do reviewers rely on test results despite remaining gaps in verification? EXAMPLE TOOLS LibreOffice Writer/Calc, audio recorder with consent; no programming required. Equivalent tools are fine. FIRST THREE ACTIONS 1. Prepare a pilot with 2 of the 12 printed Python packets. Recruit 8 Python-capable final-year IS students for the full study; write the task questions and independent reference answer sheet. 2. Write a recruitment message, information sheet and consent form for the participants named above. Agree privacy handling with the supervisor before contact. 3. Pilot one session after approval; revise unclear questions, finalise the task sets and coding categories, then recruit the planned sample. INTERVIEW QUESTIONS - Would you approve this function and why? - Which behaviour do the tests actually cover? - What would you check before relying on the AI output? LITERATURE SEARCH AI generated code review trust test evidence qualitative scenario study DATA 12 printed Python packets + 8 Python-capable final-year IS students (planned) 6 checked-correct and 6 faulty functions; 3 faults are missed by displayed tests. Hide labels and keep code identical between evidence-present/absent versions. HOW TO TEST IT Python-capable final-year IS students approve, reject or defer on non-overlapping packets. Compare per-person decision errors and reliance on displayed tests versus independent inspection; no model execution or interface development. FINAL RESULT Review checklist and findings on misplaced trust. ACCESS OR PEOPLE Recruit 8 Python-capable final-year IS students. Reading code is required; writing software is not. TEMPLATES https://thesis.uya.no/starters/review-with-reasons/nontechnical/study-plan.csv https://thesis.uya.no/starters/review-with-reasons/nontechnical/coding-sheet.csv https://thesis.uya.no/starters/review-with-reasons/nontechnical/literature-matrix.csv Use participant codes instead of names or email addresses.