Itay Itzhak

I'm interested in



About

I am a PhD candidate at the Technion, co-advised by Yonatan Belinkov (Technion) and Gabriel Stanovsky (Hebrew University of Jerusalem).

My research focuses on evaluating and interpreting LLMs, particularly the reasoning and decision-making processes that shape their behavior. I combine behavioral and representational analyses to study model biases, behavioral tendencies, and how they emerge from training.

I previously interned at Meta AI, where I studied attention dynamics in translation models. I completed my M.Sc. at Tel Aviv University with Omer Levy, investigating how token-level spelling information is encoded in embedding matrices. Beyond my research, I co-organized the GEM workshop at ACL 2025, which promotes best practices for evaluating natural language generation systems across languages and tasks.

I welcome collaborations and enjoy discussing research, language models, and related ideas. Feel free to reach out via email!

News

Selected Publications

From Feelings to Metrics: Understanding and Formalizing How Users VIBE-TEST LLMs

Itay Itzhak, Eliya Habba, Gabriel Stanovsky, Yonatan Belinkov

COLM 2026
[Code] / [Webpage]

Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration

Eliya Habba, Itay Itzhak, Asaf Yehudai, Yotam Perlitz, Elron Bandel, Michal Shmueli-Scheuer, Leshem Choshen, Gabriel Stanovsky

COLM 2026
[Code]

ManagerBench: Evaluating The Safety-Pragmatism Trade-Off In Autonomous LLMs

Adi Simhi, Jonathan Herzig, Martin Tutek, Itay Itzhak, Idan Szpektor, Yonatan Belinkov

ICLR 2026
[Code] / [Webpage]

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs

Itay Itzhak, Yonatan Belinkov, Gabriel Stanovsky

CoLM 2025 - Spotlight Paper 🔦
[Code] / [Webpage] / [Talk]

DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation

Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky

ACL 2025 Findings
[Code] / [Webpage]

Trust Me, I'm Wrong: High-Certainty Hallucinations in LLMs

Adi Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, Yonatan Belinkov

EMNLP 2025 Findings
[Code]

Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias

Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, Yonatan Belinkov

TACL 2024
[Code]

The impact of tumor detection method on genomic and clinical risk and chemotherapy recommendation in early hormone receptor positive breast cancer

Yael Bar, Kfir Bar, Itay Itzhak, Chen Shitrit Niselbaum, N. Dershowitz, E. Shachar, A. Weiss-Meilik, O. Golan, I. Wolf, T. Menes, A. Sonnenblick

Breast 2021