Better Interview Answers, Same Confidence? What AI-Supported Career Training Found

The findings concern short-term task quality, leaving open whether the training improves actual hiring prospects or how much AI contributed.

A graduate can finish a teaching placement with useful experience and still struggle to explain that experience in a job interview. Reading a vacancy closely, choosing examples that match an employer's needs and preparing a convincing demonstration lesson all require practice that general career advice may not provide.

A study in Scientific Reports, titled "Structured human resource development with AI support is associated with higher expert rated recruitment task quality among education graduates," examined a programme built around those challenges. Graduates who received structured training with AI support earned higher expert ratings on recruitment tasks completed without AI than those receiving conventional career guidance. The findings concern short-term task quality, leaving open whether the training improves actual hiring prospects or how much AI contributed.

Eight weeks of practice built around real recruitment tasks

The research involved 186 participants seeking education-sector jobs in Guangdong, China, including final-year students and graduates within two years of finishing their studies. Twelve existing training groups from three institutions were divided between the two programmes, with 93 participants in each.

Assignment depended on scheduling and room availability rather than random selection. Each of the six facilitators taught both programmes, and institutions and delivery schedules were balanced across the groups to reduce the influence of those differences. Both programmes offered eight weekly sessions lasting 90 minutes each. Conventional guidance covered recruitment requirements, résumé preparation, interview techniques and job-search planning through lectures, examples and discussion.

The AI-supported programme added individual practice, detailed feedback and repeated rehearsal. Participants analysed job advertisements, matched requirements to evidence from their coursework and placements, revised application materials, practised interviews and prepared teaching demonstrations. They finished by turning gaps in their experience into action plans with practice targets and checkpoints.

ChatGPT's GPT-4o model supported these activities, generating interview questions and feedback on relevance, specificity, evidence and clarity. Participants were instructed to protect personal information and use only genuine experience and qualifications. Facilitators checked the work for accuracy, authenticity and relevance, returning 16.2% of reviewed outputs for revision at least once.

Higher ratings when graduates worked without AI

Participants completed assessments before training and immediately afterwards, covering job-requirement analysis, a résumé and portfolio plan, a structured interview and a short teaching demonstration or work sample. The assessments took place under supervision with electronic devices collected and AI use prohibited. Two external experts reviewed anonymised written materials and recordings without being told the participants' programme or whether the work came from the first or second assessment.

This arrangement matters because the study examined what graduates could produce independently after training, rather than the quality of documents generated for them by a chatbot. Post-programme data were available for 168 participants, representing 90.3% of the original sample. Their average overall ratings rose from 49.92 to 60.17 out of 100 in the AI-supported programme and from 51.97 to 55.73 in conventional guidance.

After accounting for starting scores and training-group membership, the difference favoured the AI-supported programme by 5.38 points. The estimated range around that difference was 2.76 to 8.01 points, with a standardised effect size of 0.60. Eight analytical approaches produced broadly similar results, including checks addressing missing data and participants' backgrounds.

Exploratory analyses found the clearest evidence for advantages in job-requirement analysis and structured interview responses, with adjusted differences of 5.33 and 4.49 points. Estimates for résumé and portfolio planning and teaching demonstrations were smaller and less certain.

Those figures do not establish that the programme works better for some skills than others, because direct comparisons between the task effects were inconclusive. An additional analysis excluding résumé and portfolio work still found an advantage, reducing concern that the findings simply reflected participants remembering wording developed during training.

Confidence did not rise as clearly as task ratings

Participants answered questions about career confidence, their perceived fit with jobs, motivation to search for work and confidence using AI for career preparation. All four measures favoured the AI-supported programme, with smaller differences than those found in expert ratings. None provided clear evidence of an individual difference after the researchers adjusted for testing several outcomes.

The study could reliably detect only larger differences than those observed on the individual questionnaires, making modest changes difficult to establish. An additional analysis put the average questionnaire effect at 0.23 standard deviations, compared with 0.60 for expert ratings. The gap suggests that asking graduates how prepared they feel can give a different picture from examining the work they produce.

The questionnaires were newly developed and had not been validated outside this sample. The comparison between outcome types was not planned, and differences between the measures make it approximate. Possible explanations include confidence changing more slowly than task performance or expert ratings responding partly to presentation features rather than deeper capability.

What career services can take from the findings?

The largest limitation concerns the scoring itself. Experts gave overall impressions without a written scoring guide established before data collection, and agreement between them was moderate. Higher ratings cannot automatically be treated as proof of stronger professional capability.

The study compared two complete programmes that differed in practice, feedback, rehearsal and AI access. It cannot identify which ingredients produced the advantage. A comparison with equally structured training delivered without AI would be needed to establish AI's added value.

Participants volunteered, groups were not randomly assigned, and only 12 groups took part. Unmeasured differences could have influenced the results. Those missing the final assessment tended to report lower initial confidence and motivation, although analyses accounting for missing data produced similar estimates.

The same assessment materials were used twice, creating opportunities for familiarity to improve scores. Assessment staff knew which programme participants attended, and the researchers did not test whether the experts successfully remained unaware. Exact GPT-4o versions were not recorded across the delivery period.

There was no later follow-up, no measurement of interview invitations or job offers, and no evidence that the findings extend beyond this regional education-sector sample. For career services, the research supports closer examination of programmes that combine individual practice, feedback and supervised assessment. Future studies need random assignment, clear scoring guides, established questionnaires, fresh assessment materials and follow-up through actual recruitment. The practical question is whether graduates can carry what they practise into an unfamiliar interview, a classroom demonstration and the job itself.