(Note: This was literally a shower thought – so I haven’t thought this through very deeply.)

Introduction

I have been working with a (to me) novel database schema over the last few weeks and have been using an LLM to generate some queries based on the schema and some examples. While the success rate is far from 100%, the outputs have still been good starting points, and I would have spent a lot more time on the (often throwaway) queries if I had to write them by hand.

At the same time, the queries that are actually put into production will need to be very efficient – perhaps at the cost of readability.

And here’s where my idea comes in: Given a good test suite and a method of evaluating performance, an evolutionary algorithm could likely be used to optimize solutions by using an LLM to reformulate – and possibly recombine – the solutions.

In my case, this would be queries and a test suite that ensures correctness and measures performance. I would probably also want to include the schema and some example data as context.

Background

An evolutionary algorithm (EA) is a class of optimization algorithms that mimic the process of natural selection to evolve a population of candidate solutions to a problem. The algorithm iteratively selects the best solutions from the population, applies genetic operators (mutation, crossover) to create new solutions, and evaluates the fitness of the new solutions. It then repeats this process until some criteria are met or a pre-defined number of iterations have been performed. (There is a simple example in the documentation for my SimpleEA package. and an explanation in my Master’s thesis, starting on page 14.)

Idea

Given an optimization problem and a way to evaluate how good a solution is (“fitness” in EA parlance), an LLM could be used to generate new solutions by mutating existing solutions (“mutation”) and possibly also by combining two or more solutions (“crossover”). These operations can probably be done with a high degree of entropy (i.e. a high “temperature” for the LLM) as wrong solutions will be heavily penalized by the fitness function.

Closing thoughts

There are probably a lot of devils hiding in a lot of details here. Just off the top of my head, for my example of optimizing SQL queries:

• One would probably also need to allow the LLM to modify indexes
• It might be necessary to come up with some notion of “simplicity” to avoid ending up with something unmaintainable

I still, however, think this is a really interesting idea and hope I can find some time to experiment with this. Or maybe you already have?. Let me know!