Denny at 19:46, 12 June 2024

2024-06-12T19:46:49Z

Denny: Created page with "{{pubdate|{{subst:CURRENTDAY}}|{{subst:CURRENTMONTHNAME}}|{{subst:CURRENTYEAR}}}} Here’s an experiment I would love to do if I had the resources. Just to start gaining some..."

2024-06-12T19:34:18Z

Created page with "{{pubdate|{{subst:CURRENTDAY}}|{{subst:CURRENTMONTHNAME}}|{{subst:CURRENTYEAR}}}} Here’s an experiment I would love to do if I had the resources. Just to start gaining some..."

New page

{{pubdate|12|June|2024}}

Here’s an experiment I would love to do if I had the resources. Just to start gaining some more understanding of how LLMs work.

# Train an LLM Z on a lot of English text.
# Ensure that the LLM in its response uses correctly the past tense of “go”, “went”, in its responses.
# Ask the LLM directly what the past tense of “to go” is, and expect “went”.
# Remove all sentences / texts from the corpus that contain the word “went”. Add more text to the corpus to make it roughly the same size again.
# Train an LLM A on that corpus.
# Use the same prompts to see what the LLM uses instead of “went”.
# Ask the LLM directly what the past tense of “to go” is. I expect “goed”?
# How many example sentences / texts containing the text “went” does one need to add to the corpus of LLM A and retrain in order for the resulting LLM to get it right. Is one enough? Ten? A thousand?
# Add an explicit sentence ‘The past tense of “to go” is “went”’. to the corpus of LLM A and retrain instead of the implicit training data. Did the trained LLM now get it right? Does it use it right? Does it answer the explicit question correctly?
# Add an explicit sentence to the prompt of LLM A, instead of retraining it. Does it use the word right? Does it answer the explicit question correctly?

If there is some similar work to this out there, or if anyone has some work like this, I’d be very curious for pointers.

{{tag|Simia}}
<noinclude>{{simiapost|english}}</noinclude>

@@ Line 16: / Line 16: @@
 If there is some similar work to this out there, or if anyone has some work like this, I’d be very curious for pointers.
 {{tag|Simia}}
 <noinclude>{{simiapost|english}}</noinclude>

Experiment to understand LLMs better - Revision history

Denny at 19:46, 12 June 2024

Denny: Created page with "{{pubdate|{{subst:CURRENTDAY}}|{{subst:CURRENTMONTHNAME}}|{{subst:CURRENTYEAR}}}} Here’s an experiment I would love to do if I had the resources. Just to start gaining some..."