Loading Events

« All Events

Nicholas Rogers’ MS in Applied Mathematics Project Presentation “A Mathematical Description of Large Language Models and Analysis of ChatGPT’s Responses”

Monday, Jul 22nd @ 10 a.m. - 12 p.m. Free

Join us for Nicholas Rogers’ MS in Applied Mathematics Project Presentation “A Mathematical Description of Large Language Models and Analysis of ChatGPT’s Responses” on Monday July 22nd, from 10am-12pm in Student Commons 4113 or Zoom. Email mathstats-staff@ucdenver.edu for the link.

 

Title: A Mathematical Description of Large Language Models and Analysis of ChatGPT’s Responses

Abstract: Large language models (LLMs) are machine learning models that are designed to perform Natural Language Processing (NLP) tasks, which include part of speech identification, language translation, and text generation, among others. The most famous LLM today is ChatGPT, which has become a common source for coding questions. However, LLMs require a large amount of high-quality training data to provide accurate responses. One such source of training data is StackExchange.

In this paper, we first present a rigorous mathematical background on LLMs. We then discuss an analysis of data collected from StackExchange in the time around ChatGPT’s release, which shows a decrease in the amount of available input data since ChatGPT’s release. Third, we discuss the results of an experiment we performed using ChatGPT’s most recent model, GPT-4o, to determine if we can see the effects of reduced amounts of available data in recent years on ChatGPT’s responses to questions related to newer data. This is done using packages in R’s Bioconductor library.

We find that ChatGPT struggles to answer questions that solely test its ability to recall information about the packages but performs much better when asked to identify and resolve common bugs and dependency issues relating to packages in the Bioconductor library.