Large Language Models

We propose to adopt a causal framework for preference learning to define and address challenges like causal misidentification, preference heterogeneity, and crucially, confounding due to user-specific objectives.

We train LLM agents as Language-conditioned policies without requiring expensive labeled data or online experimentation. The framework leverages LLMs to enable the use of unlabeled datasets and improve generalization to unseen goals and states.

This paper formalizes task ambiguity in tasks specified in natural language and frames task disambiguation through Bayesian Experimental Design, leading to more effective strategies for LLMs to pose clarifying questions.