Understanding public-service guidance — Technical https://thesis.uya.no/proposals/public-services-made-clear/#technical-track BACHELOR Final result: Search prototype, test questions and usability findings. Task: Build a BM25 paragraph search over the specified NAV page. Data: NAV guidance + 40 labelled questions (mixed) finalise expanded sections and paragraph IDs. Write 30 answerable/10 out-of-scope questions. Split by intent: development 7/3; held-out 23/7. Independently check acceptable passages. Requires: Recruit Norwegian-reading adults and an independent key reviewer; NAV staff access is unnecessary. Method: Use the fixed page and labelled questions; check retrieval hits and observe adults finding the correct source paragraph. Plan 4 participant sessions. STEPS Build and test 1. Read the starting sources and choose one established implementation method. 2. Write the requirements, data fields, system diagram and test cases. 3. Prepare NAV guidance + 40 labelled questions. Make the answer key and pass criteria before testing. 4. Build a working version of search prototype, test questions and usability findings. 5. Run function, integration and failure-case tests. Record each result. 6. Run the practical evaluation and list the changes the system still needs. MASTER Final result: Search prototype, question set and retrieval comparison. Research question: Investigate retrieval failure and abstention for Norwegian public-service questions. Compare term overlap and BM25 on answerable/unanswerable intents; study how source presentation affects successful information finding without inventing answers. Research result: A specific evaluation of retrieval and evidence-use failures, with design implications. STEPS Design, build and test (DSR) 1. Read the newest papers and list the closest existing systems. 2. Write down the versions, fields, data split, case assignment, random seeds and correct answers for NAV guidance + 40 labelled questions. 3. Draw the user workflow, data model and system architecture. List the requirements and pass criteria. 4. Build a working version of search prototype, question set and retrieval comparison. 5. Test every function, connection and failure case. Save the failed tests as well as the passed tests. 6. Compare the system with the named alternative. Then run the user task or decision task in the assignment. 7. Report the measured result, the failed cases and the design lessons another team can reuse. CURRENT PROJECT LITERATURE Design characteristics and service experience with e-government services: A public value perspective (2025, peer-reviewed): https://doi.org/10.1016/j.ijinfomgt.2024.102834 Links e-government design characteristics to user beliefs, service quality and continued use. Improving the use of public e-services through explainability (2025, peer-reviewed): https://doi.org/10.1007/s11573-024-01212-9 Provides current empirical evidence on how explainability can improve citizens’ use of public e-services. IS THEORY STARTING POINTS Gao et al. (2024 revision) — Retrieval-Augmented Generation: A Survey: https://arxiv.org/abs/2312.10997 Separate retrieval, evidence use and output evaluation; trace newer retrieval work. Vessey & Galletta (1991) — Cognitive Fit: https://pubsonline.informs.org/doi/10.1287/isre.2.1.63 Test whether a representation helps one kind of task more than another. Search Scopus or Web of Science and ACM Digital Library using the topic query, then follow citations to the thesis start date. Record searches and compare methods, data, findings and limitations in literature-matrix.csv. Use that review to confirm or revise the gap and choose a current comparator. The linked papers are starting points. PILOT QUESTION Does ranked, source-linked paragraph retrieval help readers find supported answers in NAV guidance? EXAMPLE TOOLS Python, BM25 library, Streamlit Equivalent tools are fine. FIRST THREE ACTIONS 1. Save a dated NAV page snapshot and split it into numbered paragraphs. 2. Write 5 answerable and 2 out-of-scope questions; ask a second reader to label the matching passages. 3. finalise the question split, build keyword search, then add BM25 without consulting held-out answers. LITERATURE SEARCH public service information retrieval citizen comprehension DATA NAV guidance + 40 labelled questions (mixed) finalise expanded sections and paragraph IDs. Write 30 answerable/10 out-of-scope questions. Split by intent: development 7/3; held-out 23/7. Independently check acceptable passages. NAV: Økonomisk sosialhjelp: https://www.nav.no/okonomisk-sosialhjelp HOW TO TEST IT BM25 versus term-overlap search with identical processing and interface. On the held-out set report hit@1, hit@3, MRR, answerable-query coverage, out-of-scope false acceptance and answerable false refusal separately. Twelve Norwegian-reading adults complete 8 non-overlapping tasks, 4 per mode; measure comprehension, source checking, time and unsupported reliance. FINAL RESULT Search prototype, question set and retrieval comparison. ACCESS OR PEOPLE Recruit 12 Norwegian-reading adults and an independent key reviewer; NAV staff access is unnecessary. TEMPLATES https://thesis.uya.no/starters/public-services-made-clear/technical/data-plan.csv https://thesis.uya.no/starters/public-services-made-clear/technical/evaluation.csv https://thesis.uya.no/starters/public-services-made-clear/technical/literature-matrix.csv Use participant codes instead of names or email addresses.