果冻传媒

Citation

Saemundsdottir GB & Haraldsson SO (2024) Large Language Models as All-in-one Operators for Genetic Improvement. In: volume 33. GECCO '24 Companion: Genetic and Evolutionary Computation Conference Companion, Melbourne VIC Australia, 14.07.2024-18.07.2024. ACM, pp. 727-730. https://doi.org/10.1145/3638530.3654408

Abstract
Due to their versatility and increasing popularity, Large Language Models (LLMs) can offer exciting new research avenues in most fields of science and technology. This paper describes a proof-of-concept experiment on the applicability of LLMs as a recombination operator for Genetic Improvement (GI). A simple GI search was applied to two faulty Python functions that are generally considered appropriate exercises for beginner programmers. The initial generation of programs was seeded with a set of faulty versions and then GPT-3.5 turbo from OpenAI was used with minimal prompting, as the only recombination operator to generate subsequent generations. Several interesting observations were made that will guide further investigations into the use of LLMs in GI. Key among those was that it seems LLMs will need substantial and highly optimised prompting to be effective standalone GI operators. The LLM did not recognise logic errors but when the search was seeded with syntax errors then the performance was significantly better. An important feature of generative AI that was observed and should be exploited in future work is its ability to synthesise mutations even if the prompt did not ask for them.

Keywords
genetic improvement; program repair; search based software engineering; large language models; recombination operators