Dynamic Vocabularies for Conditioning a Language Model for Transforming Natural Language to a Logical Form

Abstract. Techniques are disclosed herein for generating dynamic vocabularies for conditioning a language model. A dynamic vocabulary is constructed from an input prompt, database schema information for a database to be queried, and programming language information for a programming language to be used for querying the database to condition the language model to predict an output statement in the programming language. The dynamic vocabulary can be included in prompt information that is provided to the language model. The number of tokens in the dynamic vocabulary can be different than a number of tokens included in a vocabulary of the language model. By utilizing a dynamic vocabulary, the language model can be conditioned to predict tokens for the output statement that are contextually consistent with the tokens included the dynamic vocabulary.

Links: Patent