Detection of Numerical Bugs with Large Language

**Detection of Numerical Bugs with Large Language Model(LLM)**:

- Réf **ABG-126594**
- Stage master 2 / Ingénieur- Durée 6 mois- Salaire net mensuel 630€- 31/10/2024- David Defour- Lieu de travail- Perpignan Occitanie France- Champs scientifiques- Informatique
- Mathématiques
- Mots clés- IEEE754, floating-point computation, LLM-
- 01/02/2025**Établissement recruteur**:
**Site web**:
The University of Perpignan Via Domitia is one of the oldest universities in Europe, located at the center of the Toulouse-Montpellier-Barcelona triangle. It houses the Laboratory for Mathematics, Multidisciplinary Research, and Simulation (LAMPS), which focuses on Mathematical and Numerical Modeling for Mechanics, Physics of Complex Systems, Characterization of Digital and Discrete Systems, as well as Nonlinear Analysis and Optimization.

**Description**:
Introduction:
The advent of LLMs (Large Language Models) is generating a great interest in the research community for their potential to enhance productivity in numerous sciences and engineering. For example, GitHub Copilot, a model trained on natural language and a wide range of programs, helps developers write code faster and more efficiently by using automatic code completion and answering complex directives in natural language (prompt). However, this potential remains to be explored/demonstrated in many challenging areas, such as writing programs that manipulate floating-point numbers due to the specificity of these numbers and the issues related to their use.

The objective of this internship is to assess the usefulness of LLMs for static detection (without executing the program) of numerical bugs during code writing. It is worth noting that this internship is part of a significant project funded for the period 2025-2029 by the National Research Agency (ANR project Floating-Point Transformer 4), bringing together academic and industrial partners (University of Perpignan Via Domitia, University of Versailles - Paris-Saclay, Sorbonne, EDF, CEA, ANEO, Intel). A continuation through a PhD is possible.

Floating-point arithmetic:
The IEEE-754 [6] standard is the cornerstone of floating-point number representation in modern computer systems, ensuring consistency and precision in numerical calculations across various hardware and software platforms. Universally adopted, this standard defines several formats for representing real numbers, the most used being single precision (32 bits) and double precision (64 bits). These numbers are represented by three fields: the sign, which determines whether the number is positive or negative, the biased exponent, which represents the range of possible values, and the mantissa, which represents the fractional part of the number.

Today, these issues can be detected at the end of the process through static or dynamic code analysis. Static analysis involves examining the source code without requiring its execution and focuses on identifying errors or coding standard violations. It allows for early problem detection, reducing development time and costs. However, this type of analysis may generate false positives or false negatives, affecting the quality of the analysis. On the other hand, dynamic analysis requires code execution and evaluates its behavior in real-time on one or more test data sets, but it is often incomplete (examples of tools: CADNA[2], FPCheckerErreur Source du renvoi introuvable., Verificarlo[3], FramaC[5]).

LLM:
Several methods exist to improve the relevance of the results produced by these tools. For example, prompt engineering involves designing precise and relevant textual instructions, fine-tuning adjusts a pre-trained model with new data for a specific domain, and RAG (Retrieval-Augmented Generation) enriches queries with information retrieved from external sources. Each of these solutions has its advantages and disadvantages. Fine-tuning is generally more suitable for stylistic adjustments but has some drawbacks (e.g., susceptibility to forgetting due to overtraining or altering the importance of information located at the core of its context window), while RAG is better suited for knowledge enhancement but comes at the cost of more complex prompts, leading to higher processing costs or scalability issues. Lastly integrating one or multiple Llm in an agentic system, such as langchain, and enabling reflection and reasoning strategies like chain of thought or self-iterating, have proven to bring significant progress in complex task at the expense of time complexity, in math and coding challenges.

Methodology:
- Set up a software infrastructure based on Ollama to evaluate LLMs on these programs, first on a test machine and then on a production machine (e.g., Jean-Zay).
- Make the necessary modifications to InterFLOPBench to automate the analysis of responses (e.g., adding metadata and scripts).
- Propose metrics to evaluate the quality of the LLMs' responses.

Assess the value of different solutions for imp