Machine Learning Reveals Why Women’s Agency Measures Must Reflect Local Realities in Tunisia

Women’s agency is highly context-specific, with Tunisia’s evidence showing that locally validated, multidimensional measures outperform standardized indicators and imported survey questions. Machine learning can make gender measurement more efficient, but current GenAI tools cannot replace human expertise needed to guide governments, development partners and private-sector decisions.

Measuring women's empowerment may look straightforward on paper, but new research from the World Bank and the Humphrey School of Public Affairs at the University of Minnesota shows that the reality is far more complicated. A study of women's agency in Tunisia finds that survey questions that work well in one country, or even one part of a country, may perform poorly elsewhere. The findings have important consequences for governments, development partners and businesses that use gender indicators to design programmes, allocate resources and measure whether interventions are actually improving women's lives.

The research examined 387 married women with children under the age of 10, including 198 from rural areas and 189 from urban areas. Each woman completed an in-depth qualitative interview and a quantitative survey containing 89 potential questions on agency. Researchers then used machine-learning methods to identify which questions best reflected women's ability to make choices and act on their goals.

Local Context Can Change What Empowerment Looks Like

One of the strongest findings is that there is no universal set of questions that can accurately measure women's agency everywhere.

The researchers compared Tunisia with an earlier study in India. When questions selected as strong measures in India were applied in Tunisia, their correlation with the detailed qualitative benchmark was only 0.205–0.269. There was also limited overlap between the five questions identified as most useful in the two countries.

Differences appeared even inside Tunisia. Questions that worked well for urban women did not necessarily work well for rural women, and vice versa. Applying questions selected for one setting to the other produced correlations of only 0.156–0.301.

This matters for governments because standardized international indicators may be useful for comparing countries, but they may not accurately reveal women's circumstances locally. Policymakers evaluating gender programmes should therefore consider validating indicators before using them to determine whether an intervention has succeeded.

Jobs, Money and Personal Choice Matter

The Tunisian results also show that women's agency cannot be understood only by asking who makes household decisions.

Across three machine-learning approaches, the strongest question examined whether a woman's husband would be comfortable with her working in an environment where most workers were men. Other important questions covered whether women personally used bank or savings accounts, could purchase items without seeking permission, influenced financial decisions and believed they could solve difficult tasks.

This has wider economic implications. Policies aimed at empowering women may need to connect employment, financial inclusion, mobility, household bargaining and self-confidence rather than treating them as separate development issues.

The study compared two established survey approaches. Questions from the regionally relevant Labor Market Panel Surveys recorded correlations of 0.317–0.342 with the qualitative benchmark. The narrower Demographic and Health Survey decision-making questions achieved 0.213–0.256.

For development partners, this suggests that a short but multidimensional questionnaire may provide better information than a larger collection of questions concentrated on one area of women's lives.

Private-sector stakeholders also have reason to pay attention. Banks, fintech companies, employers and businesses designing products or programmes for women could benefit from better local information about financial autonomy, employment constraints and decision-making. The risk is that generic gender indicators may cause companies to misunderstand barriers or overestimate the impact of inclusion initiatives.

Better Data Does Not Always Require Bigger Surveys

Developing locally appropriate measures can be expensive, but the research provides clues about how costs could be controlled.

Using the preferred LASSO machine-learning model, the full sample of 387 women produced a correlation of 0.395 with the qualitative benchmark. Performance remained at 0.389 with 300 observations, 0.351 with 200, 0.341 with 150 and 0.322 with 100.

Below that level, performance deteriorated more rapidly, falling to 0.262 with 50 observations and 0.134 with 25. The researchers therefore suggest that approximately 100–150 observations could be a practical minimum for developing a context-specific measure using this approach.

More questions were not automatically better either. A five-question measure achieved a correlation of 0.395, while the strongest performance, 0.461, was reached with 12 questions. Adding further questions eventually reduced performance because weaker indicators introduced additional noise.

This offers governments and international development organizations an important cost-saving lesson. Instead of automatically fielding long surveys, they could invest in validating a smaller number of carefully selected questions covering several dimensions of women's agency.

AI Can Help Research, but Cannot Yet Replace Human Insight

The researchers also tested six generative AI approaches to determine whether artificial intelligence could reduce the time and cost of analysing interviews and designing surveys.

The results were weak. AI-generated qualitative scores correlated only 0.121 with manually produced scores, while another AI-generated agency measure recorded a correlation of 0.151. GenAI-based question-selection approaches produced correlations of around 0.171–0.282, compared with approximately 0.330–0.405 for supervised machine-learning models built around human-coded qualitative evidence.

The cost issue remains significant. The Tunisia research took approximately one year, with qualitative coding and scoring alone requiring nearly three months. AI could eventually reduce this burden, but the findings indicate that replacing trained researchers too quickly could weaken measurement quality.

For policymakers and development partners, the practical way forward is to combine human understanding of local conditions with carefully tested data tools. Governments should validate gender indicators before linking them to programme targets or funding decisions, while development institutions can support countries in building locally relevant measurement systems. Businesses should similarly test gender and inclusion metrics against the markets and communities in which they operate.

The wider message is simple: better policies for women require better information about women's actual choices and constraints. Machine learning can help turn detailed local evidence into shorter and more practical surveys, but Tunisia's experience shows that technology cannot remove the need to understand context.