John Nay (Stanford, NYU; Google Scholar), David Karamardian (M.S. 2023, Stanford), Sarah B. Lawsky (Northwestern; Google Scholar), Wenting Tao (Unbiased), Meghana Bhat (Unbiased), Raghav Jain (Unbiased), Aaron Travis Lee (Unbiased), Jonathan H. Choi (Minnesota; Google Scholar) & Jungo Kasai (Ph.D. Candidate 2023, Washington), Giant Language Fashions as Tax Attorneys: A Case Examine in Authorized Capabilities Emergence:

Higher understanding of Giant Language Fashions’ (LLMs) authorized evaluation skills can contribute to enhancing the effectivity of authorized providers, governing synthetic intelligence, and leveraging LLMs to determine inconsistencies in regulation. This paper explores LLM capabilities in making use of tax regulation. We select this space of regulation as a result of it has a construction that enables us to arrange automated validation pipelines throughout 1000’s of examples, requires logical reasoning and maths abilities, and permits us to check LLM capabilities in a fashion related to real-world financial lives of residents and corporations. Our experiments exhibit rising authorized understanding capabilities, with improved efficiency in every subsequent OpenAI mannequin launch.