I’m picturing a technique where you can see what an LLM is likely to respond with, which updates in real time.
It’s a bit trippy, but it’s like GitHub Copilot, in that there is predicted text while you type, but it’s predicting what an LLM would say in response.
It updates its prediction for any change in word of your current input.
So if I just type the word “The”, the prediction might show the average of a bunch of repeated LLM responses, maybe “I’m sorry, I didn’t catch that, could you please ask again?” or something.
Like Google Translate, maybe there are 3 main variations on what answers you’re expecting, so there’s the main one and the alternatives listed below it.
Now when you continue typing, to “The book”, the predicted responses update.
However, it shows in relative terms what changed the most by the addition of the token “book”, so you can see / get a sense for specifically how that token changes the model’s “understanding”, in that context.
For example, you could color-code or size-code words to show which words appear in the predicted output which are more unique or distinctive. Like by adding “book” into the input context window, the output might indicate the presence of highly unique tokens in the predicted output, like “library”, “read”, etc.
There is a lot you could do with this.
I see it as maybe being based on a kind of constructed “calculus” of LLM inputs and outputs.
You have to find a way to define the average of any N strings. If the average of two strings is judged highly similar, they can be treated (in that context) as synonyms; if the average is highly different, they are two different variations of output you might expect.
The average helps you determine which tokens are “unique” - because they are or are not in some average of N output strings.
Can anyone suggest how you could construct that average function?
One way would involve adding and subtracting common words between two sentences.
Another would be using an LLM to judge how similar two sentences are; ie an LLM-native approach where the LLM provides all the functions for assessing the LLMs own output.
