Problems in digital public services
Group feedback about a digital service into specific problems the service owner can fix.
Technical
Bachelor’s · applied project
Final result
A reproducible triage prototype and error report.
Material to create
200 fictional comments + 40 pilot comments
Five service steps; 180 single-topic and 20 mixed/unclear evaluation comments. Label step and actionable problem separately. Vary vocabulary, tone and length independently of labels.
What to do
- Build a TF–IDF logistic comment classifier with confidence, abstention and a manual correction log.
How it is tested
Use the reviewed synthetic comments, paraphrase-family splits and keyword/majority baselines; report held-out errors and correction effort in 4 novice-analyst sessions.
Before you start
Example tools · Python, pandas, scikit-learn, Streamlit, pytest These are suggestions; equivalent tools are fine.
Access or people · Independent label checking and IS students; all comments must be authored.
Research method · Build and test · 6 steps
- Read the starting sources and choose one established implementation method.
- Write the requirements, data fields, system diagram and test cases.
- Prepare 200 fictional comments + 40 pilot comments. Make the answer key and pass criteria before testing.
- Build a working version of a reproducible triage prototype and error report.
- Run function, integration and failure-case tests. Record each result.
- Run the practical evaluation and list the changes the system still needs.
Download the complete bachelor plan
Download bachelor instructionsMaster’s · research project
Final result
Versioned taxonomy, supervised triage service, review dashboard and held-out evaluation.
Material to create
200 fictional comments + 40 pilot comments
Five service steps; 180 single-topic and 20 mixed/unclear evaluation comments. Group paraphrases into 120/40/40 train/validation/test partitions. Label one step plus zero or more problem types: navigation, information clarity, validation/error, availability, confirmation, accessibility and other/unclear.
What to do
- Train word uni/bi-gram TF–IDF logistic models for service step and multilabel problem type; include majority and keyword-rule baselines.
- Use 40 pilot comments to finalise the taxonomy, then split 200 reviewed comments by paraphrase family into 120/40/40 train/validation/test sets.
- finalise thresholds on validation data; report macro/micro-F1, exact match, mixed-comment recall, Brier/ECE, correction recovery, introduced errors and time.
How it is tested
Tune logistic C and multilabel thresholds on validation only, then finalise. Report macro/micro-F1, exact match, mixed-comment recall, Brier/ECE and per-label confusion on test. Six novice IS analysts triage 20 non-overlapping comments in assisted versus manual views; measure correction recovery, introduced errors, missed actionable issues and time.
Before you start
Example tools · Python, pandas, scikit-learn, Streamlit, pytest These are suggestions; equivalent tools are fine.
Access or people · Independent label checking and 6 IS students; all comments must be authored.
Start with these three actions
- Write 10 comments across the five service steps; label the step and zero or more problem types separately.
- Define the multilabel taxonomy and keep every paraphrase family in one train, validation or test partition.
- Run the majority and keyword-rule baselines first, then train the two fixed logistic-regression pipelines without inspecting test labels.
Files, questions and sources
Pilot question: Does evidence-linked feedback grouping help service owners identify specific problems to fix?
Literature search: digital public service feedback analysis usability barriers
Empty CSV templates: data-plan.csv · evaluation.csv · literature-matrix.csv
Download starter instructionsResearch method · Design, build and test (DSR) · 7 steps
- Read the newest papers and list the closest existing systems.
- Write down the versions, fields, data split, case assignment, random seeds and correct answers for 200 fictional comments + 40 pilot comments.
- Draw the user workflow, data model and system architecture. List the requirements and pass criteria.
- Build a working version of versioned taxonomy, supervised triage service, review dashboard and held-out evaluation.
- Test every function, connection and failure case. Save the failed tests as well as the passed tests.
- Compare the system with the named alternative. Then run the user task or decision task in the assignment.
- Report the measured result, the failed cases and the design lessons another team can reuse.
Research question and sources
Research question: Investigate how ambiguity changes human-in-the-loop feedback triage. Compare assisted and manual triage on single- and mixed-problem comments; test whether correction recovers errors without introducing new ones.
Research result: Design principles for calibrated, reviewable public-service feedback triage under ambiguity.
Current project literature
- Prioritising national healthcare service issues from free text feedback (2024) peer-reviewed
Shows how text analytics can support explainable prioritization of service-improvement decisions. - Designing a digital citizen-centered service (2024) peer-reviewed
Positions citizen feedback and stakeholder involvement inside a digital-service design process.
IS theory starting points
- Amershi et al. (2019) — Guidelines for Human-AI Interaction — Select and justify the correction, explanation or uncertainty design being tested.
- Wang & Strong (1996) — Beyond Accuracy — Distinguish accuracy, completeness, representation and fitness for the task.
Search Scopus or Web of Science and ACM Digital Library using the topic query, then follow citations to the thesis start date. Record searches and compare methods, data, findings and limitations in literature-matrix.csv. Use that review to confirm or revise the gap and choose a current comparator. The linked papers are starting points.
Search terms: digital public service feedback analysis usability barriers
Non-technical
Bachelor’s · applied project
Final result
A versioned taxonomy, codebook and triage workflow.
Material to create
40 synthetic feedback comments
Eight comments for each of five service steps, including ten mixed/ambiguous comments; finalise an independently reviewed key before sessions.
What to do
- Manually classify 40 synthetic citizen comments and specify the triage workflow public-service staff need.
How it is tested
Four triage staff classify ten non-overlapping comments each using a fixed step/problem codebook; document ambiguities, hand-offs and escalation points.
Before you start
Example tools · LibreOffice Writer/Calc and audio recorder with consent; no programming required. These are suggestions; equivalent tools are fine.
Access or people · Recruit 4 public-service triage staff; author and independently review all comments.
Research method · Study and improve · 6 steps
- Read the starting sources and write the exact information problem.
- Prepare 40 synthetic feedback comments and a separate answer sheet or coding sheet.
- Run one pilot and fix unclear questions.
- Collect the named evidence with consent.
- Group the findings with the stated categories and check the answer sheet.
- Produce a versioned taxonomy, codebook and triage workflow. List the three most useful changes.
Download the complete bachelor plan
Download bachelor instructionsMaster’s · research project
Final result
A triage codebook, step-by-step workflow and screen requirements backed by the study results.
Material to create
48 synthetic feedback comments + 12 triage participants
Four non-overlapping packets of 12 comments across five service steps, including 12 mixed/ambiguous comments. Recruit 6 frontline staff and 6 service owners; finalise an independently reviewed problem key and grouping rationale.
What to do
- Prepare four non-overlapping 12-comment packets from the synthetic feedback set, including mixed and ambiguous comments.
- Have frontline staff and service owners manually triage one raw packet and one evidence-linked grouped packet in counterbalanced order.
How it is tested
Compare per-person problem identification, false group acceptance, correction/escalation and evidence requests across raw and grouped packets. Code completeness, interpretability, task relevance and role sensemaking; independently code at least 25% and retain disagreements.
Before you start
Example tools · LibreOffice Writer/Calc and audio recorder with consent; no programming required. These are suggestions; equivalent tools are fine.
Access or people · Recruit 6 frontline public-service staff and 6 service owners; all comments and the answer key must be authored and independently reviewed.
Start with these three actions
- Prepare a pilot with 4 comments from one of the four non-overlapping 12-comment packets. Recruit 6 frontline staff and 6 service owners for the full study; write the triage questions and finalise the reviewed problem key.
- Write a recruitment message, information sheet and consent form for the participants named above. Agree privacy handling with the supervisor before contact.
- Pilot one session after approval; revise unclear questions, finalise the task sets and coding categories, then recruit the planned sample.
Files, questions and sources
Pilot question: How do raw versus evidence-linked grouped comments change public-service staff’s triage decisions and escalation?
Literature search: digital public service feedback analysis usability barriers qualitative scenario study
Interview prompts
- Which service problem or problems does this comment describe?
- Would you accept, correct or escalate the proposed grouping, and why?
- Which missing evidence prevents a responsible owner from acting?
Empty CSV templates: study-plan.csv · coding-sheet.csv · literature-matrix.csv
Download starter instructionsResearch method · Study, compare and explain · 7 steps
- Read the newest papers and write one exact research question.
- State which people, cases or documents you will study and what you will compare.
- Write the selection rules, questions and analysis steps for 48 synthetic feedback comments + 12 triage participants.
- Run one pilot. Fix unclear questions or categories, then keep the guide unchanged.
- Collect the named interviews, cases or documents with consent.
- Analyse them with the stated comparison or coding method. Keep disagreements and missing data.
- Report the answer, the evidence and the practical output named in the assignment.
Research question and sources
Research question: Explain how frontline staff and service owners accept, correct or escalate raw versus evidence-linked feedback groups. Compare completeness, interpretability, task relevance and role sensemaking, including exception cases.
Research result: A specific triage-use framework and requirements for accountable feedback grouping.
Current project literature
- Prioritising national healthcare service issues from free text feedback (2024) peer-reviewed
Shows how text analytics can support explainable prioritization of service-improvement decisions. - Designing a digital citizen-centered service (2024) peer-reviewed
Positions citizen feedback and stakeholder involvement inside a digital-service design process.
IS theory starting points
- Vessey & Galletta (1991) — Cognitive Fit — Test whether a representation helps one kind of task more than another.
- Wang & Strong (1996) — Beyond Accuracy — Distinguish accuracy, completeness, representation and fitness for the task.
Search Scopus or Web of Science and ACM Digital Library using the topic query, then follow citations to the thesis start date. Record searches and compare methods, data, findings and limitations in literature-matrix.csv. Use that review to confirm or revise the gap and choose a current comparator. The linked papers are starting points.
Search terms: digital public service feedback analysis usability barriers qualitative scenario study
Your degree, track and shortlist are saved in this browser.