This hits a sweet spot I think for conversations too.
I've been playing (for quite a while) on trying to encapsulate long running conversations.
You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc.
Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a general trend over the longer term.
When trying to reconstruct a conversation you really need to find the right balance without pulling in everything that has ever been discussed.
Can you say more about how this applies to long-running conversations? I've been thinking about them as well, but can't write wrap my head around how this would be better than (or even different to) standard compaction.
Forgive me if I'm being naive, but can't you just tweak the compaction prompt to differentiate? Presumably that's what you would do in the separate prompt anyway, right?
I tried to do that for very long translations, I had a sliding window, I had a memory for the important things to keep it consistent, a loop for repairs etc. https://jeena.net/loop-engineering
But for some reason the local models I used back then that was almost 2 years ago) weren't good enough so none of my optimizations did anything good for the translation quality.
You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc.
Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a general trend over the longer term.
When trying to reconstruct a conversation you really need to find the right balance without pulling in everything that has ever been discussed.
This definitely is worth further investigation.