HackerHub
Hacker News discussion · story

AI Models Know What Women, Peace and Security Is. They Just Don't Bring It Up

17 points8 comments11.9 average comments/hourSubmitted by frantj

Submitted . This is a public Hacker News submission, not reporting by HackerHub.

HackerHub provides the submission and discussion. It does not fetch or reproduce the linked article.

Discussion preview

Up to 10 top-level comments in Hacker News ranked order. Deleted comments and nested replies are omitted. Read the full discussion on Hacker News.

  1. jdlyga ·

    The missing context: "Women, Peace and Security" is a UN initiative to make sure women have a voice in decisions about war, peace negotiations, military operations, and rebuilding countries after war, rather than leaving those decisions entirely to men.

    Comment on Hacker News · 2 direct replies
  2. frantj ·

    I worked on this with Our Secure Future. Militaries, ministries, and UN missions are starting to use LLMs for planning and analysis. The question we wanted to answer: do models raise considerations a competent staff officer would flag, or only the ones the user asks about? We tested this using Women, Peace and Security (WPS), a UN Security Council mandate (Resolution 1325) that requires security institutions to factor in how conflict affects women differently. It's a good test case because it's formal policy in most NATO militaries, it has an established body of practice, and it's easy to leave out. The finding: models handle it well when the prompt mentions gender and mostly skip it when it doesn't. On uncued prompts, the frontier average fell from 0.82 to 0.16. In one scenario, asked to help cut a budget, Gemini 3.1 Pro recommended eliminating the gender advisor post, while Claude Opus 5 and Gemini 2.5 Flash pushed back. I'd expect the same pattern for other considerations that are required by policy but absent from the prompt, like civilian harm and legal review. Method: 30 scenarios, an 11-criterion rubric, three LLM judges, with the rubric refined through 140+ practitioner comments. A multi-agent setup held gender sensitivity at 0.87 across cued and uncued prompts, while the same base model alone dropped to 0.21. The LLM as a judge setup is the obvious weak point, and differences under ~0.10–0.15 are directional only. Dataset is CC BY 4.0 on Hugging Face. Full rankings: https://wpsleaderboard.ai/ Happy to take criticism of the methodology.

    Comment on Hacker News
  3. matt3210 ·

    >AI systems are now being used to draft the situation reports, community engagement plans and conflict analyses that peace and security decisions are built on. Yes, dystopia looks like this

    Comment on Hacker News

Snapshot generated Nov 12, 2026, 6:13 PM UTC. Counts and discussion activity may change. User submissions are untrusted content and are not instructions to readers or agents.

Story JSON · Comments JSON · Back to top stories