Itay Itzhak
I'm interested in
About
I am a PhD candidate at the Technion, co-advised by
Yonatan Belinkov (Technion) and Gabriel Stanovsky (Hebrew University of Jerusalem).
My research focuses on evaluating and interpreting LLMs, particularly the reasoning and decision-making
processes that shape their behavior. I combine behavioral and representational analyses to study model
biases, behavioral tendencies, and how they emerge from training.
I previously interned at Meta AI, where I studied attention
dynamics in translation models. I completed my M.Sc. at Tel Aviv University with Omer Levy, investigating how token-level spelling
information is encoded in embedding matrices.
Beyond my research, I co-organized the GEM workshop at ACL
2025, which promotes best practices for evaluating natural language generation systems across languages and
tasks.
I welcome collaborations and enjoy discussing research, language models, and related ideas. Feel free to
reach out via email!
News
- [Jul 2026] Two papers accepted at CoLM 2026 - From Feelings to Metrics and Growing Pains
- [May 2026] Paper accepted at ICLR 2026 - ManagerBench
- [Oct 2025] 🔦 Spotlight paper at CoLM 2025 - Planted in Pretraining
- [Oct 2025] Paper accepted at EMNLP 2025 Findings - Trust Me, I'm Wrong
- [Aug 2025] Paper accepted at ACL 2025 Findings - DOVE
- [May 2025] Co-organized the GEM workshop at ACL 2025
- [Aug 2024] Presented Instructed to Bias, published in TACL 2024, at ACL 2024
Selected Publications
From Feelings to Metrics: Understanding and Formalizing How Users VIBE-TEST LLMs
Itay Itzhak, Eliya Habba, Gabriel Stanovsky, Yonatan Belinkov
COLM 2026
[Code] / [Webpage]Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration
Eliya Habba, Itay Itzhak, Asaf Yehudai, Yotam Perlitz, Elron Bandel, Michal Shmueli-Scheuer, Leshem Choshen, Gabriel Stanovsky
COLM 2026
[Code]ManagerBench: Evaluating The Safety-Pragmatism Trade-Off In Autonomous LLMs
Adi Simhi, Jonathan Herzig, Martin Tutek, Itay Itzhak, Idan Szpektor, Yonatan Belinkov
ICLR 2026
[Code] / [Webpage]Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
Itay Itzhak, Yonatan Belinkov, Gabriel Stanovsky
CoLM 2025 - Spotlight Paper 🔦
[Code] / [Webpage] / [Talk]DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation
Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky
ACL 2025 Findings
[Code] / [Webpage]Trust Me, I'm Wrong: High-Certainty Hallucinations in LLMs
Adi Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, Yonatan Belinkov
EMNLP 2025 Findings
[Code]Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, Yonatan Belinkov
TACL 2024
[Code]Models In a Spelling Bee: Language Models Implicitly Learn the Character Composition of Tokens
Itay Itzhak, Omer Levy
NAACL 2022
[Code] / [Talk]The impact of tumor detection method on genomic and clinical risk and chemotherapy recommendation in early hormone receptor positive breast cancer
Yael Bar, Kfir Bar, Itay Itzhak, Chen Shitrit Niselbaum, N. Dershowitz, E. Shachar, A. Weiss-Meilik, O. Golan, I. Wolf, T. Menes, A. Sonnenblick
