Divyanshu Kumar
← all publications

Beyond Western Politics: Cross-Cultural Benchmarks for Evaluating Partisan Associations in LLMs

D. Kumar*, I. Gupta*, N. A. Birur, T. Baswa, S. Agarwal, P. Harshangi

NeurIPS LLM Evaluation Workshop · Sep 2025 · posterco-first author

abstract

Partisan bias in LLMs has been evaluated to assess political leanings, typically through a broad lens and largely in Western contexts. We move beyond identifying general leanings to examine harmful, adversarial representational associations around political leaders and parties. To do so, we create datasets NeutQA-440 (non-adversarial prompts) and AdverQA-440 (adversarial prompts), which probe models for comparative plausibility judgments across the USA and India. Results show high susceptibility to biased partisan associations and pronounced asymmetries (e.g., substantially more favorable associations for U.S. Democrats than Republicans) alongside mixed-polarity concentration around India's BJP, highlighting systemic risks and motivating standardized, cross-cultural evaluation.

Partisan BiasLarge Language ModelsNLP

cite

@inproceedings{kumar2025beyond,
  title     = {Beyond Western Politics: Cross-Cultural Benchmarks for Evaluating Partisan Associations in LLMs},
  author    = {Divyanshu Kumar and Isita Gupta and Nitin Aravind Birur and Tanay Baswa and Sahil Agarwal and Prashanth Harshangi},
  booktitle = {NeurIPS LLM Evaluation Workshop},
  year      = {2025}
}