Hey I’m Juan!

I am a research scientist at Arb Research, where I evaluate language models empirically and study their internals, to make sense of what they appear to do.

I am interested in the following research areas:

• generalization and compositionality: what are the mechanisms behind what we call generalization, is it confounded by semantic duplicates (shallow generalization), and what is the role of compositionality in all of it?
• agent reward hacking: what kind of environments or model organisms are best suited to continue researching it, and how is it related to generalization and emergence?
• mechanistic interpretability: activation engineering, steering vectors and methods for natural-language explanations of activations. Can we come up with ways to faithfully explain what is going on in the residual stream?
• evaluations and evaluation awareness: how can we (and now AIs) run evaluations with construct validity given what the model has seen in training; when models appear evaluation aware, is it a real phenomenon and does it matter in deployment?
• collusion and coordination: how can we best study multi-agent systems when coordination turns into collusion against other parties in the environment?

In the past I have worked on urban planning and quantum material dynamics research. Before that I did a theoretical physics undergrad at Trinity College Dublin. Other stints include: bartender, fast food chain worker, observatory guide, electronics shop assistant, government advisor, teacher.

You’ll find me in most places as Juan J Vazquez.

• google scholar
• arb research
• linkedin
• x formerly known as twitter
• hi at juan.app