LLMs, in fact, have slop profiles (aka overused tokens/phrases) common to the family/company, often from “inbreeding” by training on their own output.
Sometimes you can tell if new model “stole” output from another company this way. For instance, Deepseek R1 is suspiciously similar to Google Gemini, heh.
This longform writing benchmark tries to test/measure this (click the I on each model for infographics):
LLMs, in fact, have slop profiles (aka overused tokens/phrases) common to the family/company, often from “inbreeding” by training on their own output.
Sometimes you can tell if new model “stole” output from another company this way. For instance, Deepseek R1 is suspiciously similar to Google Gemini, heh.
This longform writing benchmark tries to test/measure this (click the I on each model for infographics):
https://eqbench.com/creative_writing_longform.html
As well as some some disparate attempts on GitHub (actually all from the eqbench dev): https://github.com/sam-paech/slop-forensics
https://github.com/sam-paech/antislop-vllm