publications

2024

1. Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel

   Brendan King, and Jeffrey Flanigan

   In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Nov 2024

   Abs

   Training task-oriented dialogue systems typically requires turn-level annotations for interacting with their APIs: e.g. a dialogue state and the system actions taken at each step. These annotations can be costly to produce, error-prone, and require both domain and annotation expertise. With advances in LLMs, we hypothesize that unlabeled data and a schema definition are sufficient for building a working task-oriented dialogue system, completely unsupervised. We consider a novel unsupervised setting of only (1) a well-defined API schema (2) a set of unlabeled dialogues between a user and agent. We propose an innovative approach using expectation-maximization (EM) that infers turn-level annotations as latent variables using a noisy channel model to build an end-to-end dialogue agent. Evaluating our approach on the MultiWOZ benchmark, our method more than doubles the dialogue success rate of a strong GPT-3.5 baseline.

2023

1. Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking

   Brendan King, and Jeffrey Flanigan

   In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023

   Abs DOI

   There has been significant interest in zero and few-shot learning for dialogue state tracking (DST) due to the high cost of collecting and annotating task-oriented dialogues. Recent work has demonstrated that in-context learning requires very little data and zero parameter updates, and even outperforms trained methods in the few-shot setting. We propose RefPyDST, which advances the state of the art with three advancements to in-context learning for DST.First, we formulate DST as a Python programming task, explicitly modeling language coreference as variable reference in Python. Second, since in-context learning depends highly on the context examples, we propose a method to retrieve a diverse set of relevant examples to improve performance. Finally, we introduce a novel re-weighting method during decoding that takes into account probabilities of competing surface forms, and produces a more accurate dialogue state prediction. We evaluate our approach using MultiWOZ and achieve state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings.

2. Dependency Dialogue Acts — Annotation Scheme and Case Study

   Jon Cai, Brendan King, Margaret Perkoff, and 9 more authors

   In The 13th International Workshop on Spoken Dialogue Systems Technology, IWSDS 2023, Los Angeles, United States, Feb 2023

   PDF

2018

1. ProbAnnoWeb and ProbAnnoPy: probabilistic annotation and gap-filling of metabolic reconstructions

   Brendan King, Terry Farrah, Matthew A. Richards, and 3 more authors

   Bioinformatics (Oxford, England), May 2018

   Abs DOI

   SUMMARY: Gap-filling is a necessary step to produce quality genome-scale metabolic reconstructions capable of flux-balance simulation. Most available gap-filling tools use an organism-agnostic approach, where reactions are selected from a database to fill gaps without consideration of the target organism. Conversely, our likelihood based gap-filling with probabilistic annotations selects candidate reactions based on a likelihood score derived specifically from the target organism’s genome. Here, we present two new implementations of probabilistic annotation and likelihood based gap-filling: a web service called ProbAnnoWeb, and a standalone python package called ProbAnnoPy. AVAILABILITY AND IMPLEMENTATION: Our tools are available as a web service with no installation needed (ProbAnnoWeb) at probannoweb.systemsbiology.net, and as a local python package implementation (ProbAnnoPy) at github.com/PriceLab/probannopy. CONTACT: evangelos.simeonidis@systemsbiology.org or nathan.price@systemsbiology.org. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

2016

1. Mighty morphing metabolic models: leveraging manual curations for automatic metabolic reconstruction of clades

   Matthew A. Richards, Brendan King, Evangelos Simeonidis, and 1 more author

   F1000Research, May 2016