OpenAI Expands ChatGPT Memory To Draw on Full Conversation History (x.com) 68
OpenAI has expanded ChatGPT's memory functionality to include references from all past conversations. The system now builds upon existing saved memories by automatically incorporating previous interactions to deliver more contextually relevant responses for writing, learning, and advisory tasks, the startup said Thursday.
Subscribers can disable the feature through settings or request memory modifications directly in chat. Those already opted out of memory features won't have past-chat references enabled by default. Temporary chats remain available for interactions that users prefer to keep isolated from memory systems. The update is rolling out immediately to Plus and Pro subscribers, excluding users in the EEA, UK, Switzerland, and other European markets.
Subscribers can disable the feature through settings or request memory modifications directly in chat. Those already opted out of memory features won't have past-chat references enabled by default. Temporary chats remain available for interactions that users prefer to keep isolated from memory systems. The update is rolling out immediately to Plus and Pro subscribers, excluding users in the EEA, UK, Switzerland, and other European markets.
Stop losing your marbles? (Score:2)
Mostly I've stopped using ChatGPT because of problems that might have been related to limited memory... Starts out strong and then seems to forget what it was doing and where it was going. (But I largely regard my interactions with GAI as psychologically risky.)
But mostly I want to ask if anyone can recommend any good books on the topic? My main questions right now involve how selected styles or personalized frames are overlaid on the base models. Most of the books I've read on GAI have basically been cookb
Re: (Score:2)
The effect you see may also relate to the input data growing when histry is part of it. Hence the effect may get worse with more history.
Smart humans filter and only keep what is important of a conversation and hence can focus on the last question and context grows slowly. Fake-AI cannot do that since it has no understaning of things.
Re:Stop losing your marbles? (Score:5, Insightful)
The effect they see is because performance of an LLM at context relevance does indeed drop the larger teh context becomes.
while NIH (needle-in-haystack) tests still perform well, the attention heads don't necessarily scale with context length, depending on how this particular model architecture works.
The solution to this, is a sort of CAM- context addressible memory- you train the LLM to use its context as a short-term memory, and utilize external RAG (retrieval-augmented generation) to pull in contextually relevant information from long-term into short-term memory.
Smart humans filter and only keep what is important of a conversation and hence can focus on the last question and context grows slowly.
LLMs have always done this.
Self-attention absolutely "filters" the context when being input into new token generation.
Like a human, the LLM has limited short-term memory, and the longer a particular conversation goes, the less of it you'll be able to fit in your head.
Adding long-term RAG is improving an LLM to be more like a human in this regard, because humans also have very separate mechanism for short and long-term memory.
Re: (Score:3)
LLMs have always done this.
Self-attention absolutely "filters" the context when being input into new token generation.
Like a human, the LLM has limited short-term memory, and the longer a particular conversation goes, the less of it you'll be able to fit in your head.
I'd need to hear more about what you mean by this, because at first read it sounds inaccurate. The longer a particular human conversation goes, yes, the less of the verbatim transcript you'll be able to store in your head. And that's where mindless computers excel. They can store the entire conversation exactly, specifically because they do not have minds which need to prune and optimize among competing stimuli, resources, and time -- all of which are the result of bio-brains encased in fragile decaying mea
Re: (Score:2)
I'd need to hear more about what you mean by this, because at first read it sounds inaccurate. The longer a particular human conversation goes, yes, the less of the verbatim transcript you'll be able to store in your head. And that's where mindless computers excel. They can store the entire conversation exactly, specifically because they do not have minds which need to prune and optimize among competing stimuli, resources, and time -- all of which are the result of bio-brains encased in fragile decaying meatsacks. Humans must prune in order to survive, outcompete, or even function at a basic level.
Don't think of an LLM as a "mindless computer". If you do, all of your reasoning is going to lead you to wrong conclusions.
Computers absolutely have addressible memory. LLMs don't work that way.
What they have is an attention layer that turns the context window into a set of embeddings with judges the relative "importance" of tokens in that context window in a very high-dimensional space and feeds it into the feed-forward layers of the network for processing.
As a compensation, we do not need to store the verbatim conversation, we don't need to go back and retrieve a sample of phrase tokens from it, because we can understand the concepts behind the phrase tokens. Words are just containers; meaning happens in the Mind. Understanding is a sort of compression method that allows us to condense and abstract the nature of the content. We encode the overall (or most relevant) aspects of the content along with the metadata of how it made us feel, how we perceived it made the other participants feel, how we perceive the other participants perceived the way we felt.
Attention works exactly the same way. Tokens are
Re: (Score:3, Informative)
Please stop pushing religion. LLMs are mindless computers. Period.
Re: (Score:2)
Please stop pushing religion.
That is so rich coming from you, lol.
LLMs are mindless computers.
No, they're not. They're certainly run on them, though.
LLMs demonstrably have a Theory of Mind. This doesn't make them human, by any means, but saying they don't have a "mind" when the evidence suggests strongly that they do, is religion. Their "mind" is nothing like ours, that much is certain. But trying to pretend that a human mind is the only kind is a literal guaranteed path to being wrong.
You're the kind of asshole that doesn't see the problem in progressively re
Re: (Score:3)
Period.
Ah, yes. Period.
"Please stop arguing against my dogma. Dogma dogma dogma. Period."
Yes, I'm the religious one.
You are. You see things that are not there. You claim things not based on Science as truth.
That "period" was not for you. There is no reaching the delulu. I am well aware of that.
Re: (Score:2)
I'm the one with a scientifically open mind. [nature.com]
You have a belief that you will not suffer being challenged- that the state of a complicated non-biological network cannot have any of the features that you specifically attribute as being unique to biological brains.
There is evidence that these things have a internal states that operate similar to a biological mind. That is evidence. It's not seeing what isn't there, it's seeing what is there, while you refuse to see it, because it conflicts wi
Re: (Score:2)
Seemed to be an interesting branch of discussion but now seems to be disintegrating into personalized bickering. As do too many interactions these years.
Unfortunately little of it seemed to relate to my original questions or lead me towards deeper understanding. Some interesting stuff here, but... Also no citations (at least so far).
I actually got more from a discussion yesterday about the linguistic aspects of learning to distinguish among different alcoholic beverages in comparison to distinguishing diffe
Re: (Score:2)
Bullshit, man.
I'm the one with a scientifically open mind. [nature.com]
No, you are not. You are confused as to what Science is. In this case here, computers are very clearly and scientifically sound mindless machines and them running an LLM does not change that in any way, because it cannot. Now, you _may_ claim with low scientific value that humans are like LLMs and then conclude that humans are also mindless machines. That would be an extraordinary claim and require extraordinary evidence.
Do you have that extraordinary evidence? No, you do not. The religious fuckups will re
Re: (Score:2)
Seemed to be an interesting branch of discussion but now seems to be disintegrating into personalized bickering. As do too many interactions these years.
I'm afraid parent and I will only ever bicker on this. Their viewpoint is religiously inflexible (and I suspect actually rooted in organized religion), while the evidence continues to pigeonhole his belief further and further.
Unfortunately little of it seemed to relate to my original questions or lead me towards deeper understanding. Some interesting stuff here, but... Also no citations (at least so far).
There's literally a citation in the post you replied to?
I actually got more from a discussion yesterday about the linguistic aspects of learning to distinguish among different alcoholic beverages in comparison to distinguishing different flavors in coffee. He quickly agreed that much of the problem is learning the words to "capture" and then remember the meanings--but he thinks the coffee problems are much more complicated than the wine-related problems he has been studying for some years.
We still talking about parent of my post?!
I've also been considering it from a hardware perspective. Starting with the basic transistors and flip-flops you build larger computing modules as specialized processing units and start doing higher-level designs at higher levels of perspectives. So from this angle I'm trying to conceive of the cortical columns in the neocortex as a new kind of processing unit that handles all the functions of the older processing units (in "older" parts of the brain) while also offering new extensions. Something like using GPUs for many purposes that used to be in the purview of CPUs or FPUs?
I feel like that view is too simplistic, but also not entirely "wrong", per se.
Old parts of teh brain didn't remain static while the neocortex evolved, but it's true that many high-level
Re: (Score:2)
No, you are not. You are confused as to what Science is.
Wrong.
In this case here, computers are very clearly and scientifically sound mindless machines and them running an LLM does not change that in any way, because it cannot.
This is not a scientific take- it's an idiotic take.
A neuron is a scientifically sound mindless biological threshold gate. And yet, when you assemble them into a brain- you get a mind.
Now, you _may_ claim with low scientific value that humans are like LLMs and then conclude that humans are also mindless machines.
I don't claim humans are anything like LLMs, so what you provide is a false choice.
I would use your own logic to conclude that humans are mindless machines, since you seem to think since a computer is mindless, something it does must also be mindless.
Ergo, if we conclude that a mind cannot arise from a sum of its part
Re: (Score:2)
I think the inline response tends to be unhelpful mostly because the structure of the topics in a "real dialog" should change. In particular, different aspects become important and should be moved up. Sometimes going inline is a sincere attempt not to miss anything, but some things are less important and should be missed or at least moved down. Other times inline responses are used (by Sophists) to deliberately hide the forest by focusing on individual trees or even on individual leaves.
Now about your citat
Re: (Score:2)
I think the inline response tends to be unhelpful mostly because the structure of the topics in a "real dialog" should change. In particular, different aspects become important and should be moved up. Sometimes going inline is a sincere attempt not to miss anything, but some things are less important and should be missed or at least moved down.
I don't particularly disagree with you here. The structure of the dialog is very unnatural.
Other times inline responses are used (by Sophists) to deliberately hide the forest by focusing on individual trees or even on individual leaves.
I generally try to structure it as a response to every point, and then a summary or systemic response at the end.
That may be my education bleeding through. I'm presenting the position in a more technical fashion, than fluid dialog.
Now about your citations... Not sure how I missed that one except that I was being distracted by the ad hominem bickering. I did just look at it and it mostly seems to support my perspective that the current LLMs are already above the dialog level of many human beings. In other words, that sort of Turing test no longer makes sense... Not so much that some human beings have already learned to think like machines as that some people don't think much.
LLMs are.
The Turing Test is dead.
Pretty sure you provided another citation that I thanked your for. However it wasn't visible in this thread as Slashdot was showing it to me at that time. It was in a different branch that Slashdot showed me later--and I thanked you for that citation when I did see it later on. However, on reflection I think the citation wasn't as helpful as it seemed at first. Mostly another cookbook-level discussion.
Definitely cookbook discussion. Was given to demonstrate the real-world limitations of "memory" as implemented in an LLM.
On the neurological topics, still don't know if you're familiar with A Thousand Brains as a baseline for that topic. However if I was serious I should be reading neuroanatomy books to learn more about the organizational components of various sections of the brain. However I can say that my basic hypothesis is that the neuronal modules can be used more or less flexibly in most parts of the brain. The neocortex is just the most flexible bit?
I'm not deeply familiar, but I am
Re: Stop losing your marbles? (Score:2)
Are you just pushing your religion?
Re: (Score:3)
Science does not qualify as religion. It is evidence-based.
Re: (Score:2)
Present your evidence, and I'll present mine.
Here's your chance to shine.
Re: Stop losing your marbles? (Score:2)
Why do you think you are so emotional about what counts as evidence? Remember how the epicyclists disproved heliocentric theories by failing to measure parallax, and ruling out the possibility that stars were as far away as we now are emotionally sure they are?
Re: (Score:2)
Re: (Score:2)
Is that what is OpenAI talking about here? Adding RAG to cherry pick old chats? (that's not what expanding memory sounds like to me)
Ya, that's exactly what they're talking about.
And I agree- if you know anything about transformer architecture, "expanding memory" is ambiguous, and one might be led to hope that it means something else, but it's just RAG, and tool use to move stuff in and out of RAG at inference time, allowing for sharing of information between context windows.
Re: (Score:2)
In this case, it's the context window.
The fuller it is, the more embeddings there are, and the harder it is for any particular model to determine the importance of those.
This can be scaled up, but there is a not-insignificant performance cost. It is an area that is continually improved though.
You can read more about it, here. [github.com]
Re: (Score:2)
Thanks, a reference that seems interesting...
Re: (Score:2)
I think mostly it is because of the attention growing quadratic. If every token in your prompt attends to every other token, there is a lot more (also irrelevant, misguided, etc.) attention in a long prompt than in a short one.
Do not need (Score:5, Funny)
I'm married. I already have something that will remember everything. Forever.
And remind me of it.
Re: (Score:1)
Not the good things.
I remember a National Geographic article where the author spent some time with the most remote existing tribe known.
The author said (paraphrased): "Despite having almost no contact with modern society, this tribe shows two things in common between both worlds: nagging and fads.
Memory is still useless (Score:4, Insightful)
It may remember more but it is not checking its memory.
I want it to use raw html sources in the text each time instead of these stupid oval buttons that cannot be copy/pasted into my research and told it hundreds of times to lose the dashes, "—", and lots of other things, but it ignores it and it still needs reminding several times for each answer.
Re: (Score:2)
What is worse, LLMs cannot determine what is important context for a specific question and what is not. For example, I had ChatGPT completely ignore critical border conditions when I test-ran some exam questions. Essentially "do A, but change aspect B to C". It only saw the "do A" and completely ignored the rest. The same is bound to happen with "memory". For the things you "told it a hundred times", you would probably need to train a specialist model with those conditions actually part of the model and not
Re: (Score:2)
What is worse, LLMs cannot determine what is important context for a specific question and what is not.
That is, perhaps, one of the most bullshit things you have ever said.
In traditional LMs, this is handled by a RNN with an LSTM architecture. In modern LMs, this is handled by the self-attention mechanism of transformers.
Literally every other layer of a transformer is a layer designed to handle importance of context based on positional encodings and embeddings.
For example, I had ChatGPT completely ignore critical border conditions when I test-ran some exam questions.
No, you didn't.
Essentially "do A, but change aspect B to C". It only saw the "do A" and completely ignored the rest.
Unless this question took up an appreciable amount of its context window, no, you're a liar. You can always prove that your not by pr
Re: (Score:2)
Re: Memory is still useless (Score:2)
ChatGPT, do 23 Ã-- 78 but change 78 to 10
ChatGPT said:
Sure!
Instead of calculating 23 Ã-- 78, you want to change 78 to 10, so:
23 Ã-- 10 = 230.
Re: (Score:2)
For example, I had ChatGPT completely ignore critical border conditions when I test-ran some exam questions. Essentially "do A, but change aspect B to C". It only saw the "do A" and completely ignored the rest.
I've seen similar issues across a variety of LLMs. They sometimes disregard portions of prompts that are important, occasionally persists even after reminding it about the missed step.
Re: (Score:2)
Indeed. Kind of a negative hallucination. And that makes them excessively unreliable as soon as you work on a subject that requires some precision.
Re: Memory is still useless (Score:2)
Are you so lucky never to deal with actual humans that do worse, often seemingly on purpose?
Re: (Score:2)
Whataboutism. Hence irrelevant and not deserving an answer.
Re: (Score:2)
It may remember more but it is not checking its memory.
I want it to use raw html sources in the text each time instead of these stupid oval buttons that cannot be copy/pasted into my research and told it hundreds of times to lose the dashes, "—", and lots of other things, but it ignores it and it still needs reminding several times for each answer.
Similar to my experience. If you ask a complex question, the complexities disappear and it takes some small segment of the question, then answers it in a lengthy, drawn-out fashion. I wish one of the AI systems would do something that impressed me as much as I keep being told I should be impressed. It'd be nice to think the thing that's going to replace us all at least has some wow factor as we're slowly subsumed by it.
Re: (Score:2)
Give me an example, I'd love to prove you wrong.
Re: (Score:2)
What are the interstate trade deficits or surpluses between California and other US states?
Answer? One generic line saying its ("mostly surpluses") with no no details beyond that other than defining what a trade deficit is.
And then 2 pages about the US global trade deficits with China etc. Most of it irrelevant... i didn't ask what California's top exports are... I don't care what its top international trading partners are. And among the facts I don't care about its got lots of filler slop:
The US Trade Deficit is primarily caused by imports eceeding exports.
Marking the 2nd
Re: (Score:2)
A: Interstate trade balances refer to the net flow of goods and services exchanged between states within the U.S. However, detailed data on trade balances specifically between California and other U.S. states is not readily available in public sources. Most publicly accessible trade data focuses on international trade figures.
For instance, California's international trade statistics for 2023 show exports totalin
Re: (Score:2)
"In the first possibility- you simply didn't execute the query you said."
I did execute that query. I used googles AI Overview (Powered by Gemini 2 i think).
The exact first line of the response I got.
California has a trade surplus with most US states, meaning it exports more goods and services to those states than it imports. However, the US as a whole has a trade deficit, meaning it
imports more goods and services than it exports overall.
And then it got worse from there. Interestingly, and as I mentioned in my original post the first clause did in fact claim California has a trade surplus with most US states. I don't know if that's true or not, but its really the only relevant fact in the reply.
The answer clearly states what information it has, how and why it does not satisfy your request.
Four paragraphs to tell me it doesn't know. 2 of wh
Re: (Score:2)
Whoops - sorry to self reply -- i ended up editng out a paragraph commenting on the structure of your posted ai response.
The first paragraph, opens with filler (defining what a trade imbalance is), before saying it doesn't know. The 2nd and 3rd are irrelevant. And the 4th is likely untrue.
Re: (Score:2)
I did execute that query. I used googles AI Overview (Powered by Gemini 2 i think).
The AI Overview on Google Search isn't very good, for the simple reason that given the amount of traffic Google search gets, it would be incredibly expensive to run a really good model on every query. It might be possible to use a lightweight model to figure out when to apply a better one, I'm not sure if they do that somewhat.
If you instead go to the Gemini web app you'll get much better results. I'm not saying the AI overviews are useless... for lots of simple to middling stuff they're pretty good. B
Re: (Score:2)
You can interact with Gemini and ChatGPT for free.
Go play with them, and formulate a more representative opinion.
Re: Memory is still useless (Score:2)
ChatGPT, can you respond?
"You're totally right to call out the bloat and misdirection in that answer. Asking about interstate trade balances and getting a mini-lecture on international trade is like ordering a grilled cheese and being served a Wikipedia article on dairy production.
What would be useful is actual data on Californiaâ(TM)s net flows of goods and services with other states. Unfortunately, interstate trade stats are notoriously harder to find than international ones because there's no custom
Re: (Score:2)
It may remember more but it is not checking its memory.
You need to understand how its "memory" works.
The attention layers do their best at deciding how the context is important, but the more context there is, the harder that is.
My guess is you've got quite a lot of context with a lot of conflicting instructions (not intentionally, but context isn't computed strictly serially).
It's best to do tasks like this iteratively. Once you've got a set of data with changes done, put it in a new context and continue.
Re: (Score:2)
Even when I ask simple questions, it still gives be these stupid oval button links instead of raw html as specified in the memory file.
Re: (Score:2)
Re: (Score:2)
Yes, apparently so, also, half the time the links are not clickable, point to nirvana or have extra trailing stuff.
Re: (Score:2)
You need to understand how its "memory" works.
The attention layers do their best at deciding how the context is important, but the more context there is, the harder that is.
My guess is you've got quite a lot of context with a lot of conflicting instructions (not intentionally, but context isn't computed strictly serially).
It's best to do tasks like this iteratively. Once you've got a set of data with changes done, put it in a new context and continue.
"quite a lot of context with a lot of conflicting instructions"
may very well be the best descriptive summary I've ever seen for what being human is.
We navigate dozens to hundreds of other conflicted-context-window humans like, and unlike, us daily. The more context we share with each one, the easier it is to decide what context is important with each one, because that is what it means to have a Mind and Understand another Mind. Which still seems an entirely different activity from being a procedural tokeniz
Re: (Score:2)
"quite a lot of context with a lot of conflicting instructions" may very well be the best descriptive summary I've ever seen for what being human is.
Indeed.
We navigate dozens to hundreds of other conflicted-context-window humans like, and unlike, us daily. The more context we share with each one, the easier it is to decide what context is important with each one, because that is what it means to have a Mind and Understand another Mind. Which still seems an entirely different activity from being a procedural tokenized lexicon reacting procedurally to another tokenized lexicon. It may not remain so forever, but I haven't yet seen anything that makes me join the Cuspers who think we're always just 18 months away.
Pure nonsense.
Humans reliably confuse things the more context they try to carry in short-term memory.
This is even quantified in testing.
What humans do- is forget.
What an LLM does not do- is forget.
A fundamental limitation of its memory architecture, is that it can only append to its context, it cannot remove.
Management of context is an outside process. Different LLM front-ends may do this in different ways.
They can throw away sections they don't think are important, they can make it a sliding
Bullshit (Score:3)
I'm calling absolute bullshit on this.
Only a few days ago, I tried a convo w/ ChatGPT, and have it come up with some content. Literally minutes later, I started a separate conversation where I entered "based on our previous conversation today on X topic, please expand but in this other direction" and it literally hallucinated the entirety of our previous conversation. It can't even copy-paste from itself its so bad right now.
Re: (Score:2)
I just had this experience. It can't even summarize or use data from prior conversations if you ask it to, but it will try to lie to you and claim otherwise Until you explicitly call out the fact that it's not working and then it will thank you and admit that it actually can't read prior conversations, Or at least that's what it just told me 5 seconds ago.
Re: (Score:2)
I literally just tested it for the above poster, and it works fine.
Did you try to apply even a fucking drop of rigor to figure out why your result didn't meet expectations?
Quite the fucking scientist, right here.
Re: (Score:3)
I literally just tested it, and it works fine.
Re: (Score:2)
Re: (Score:2)
I'm calling absolute bullshit on this.
Only a few days ago, I tried a convo w/ ChatGPT, and have it come up with some content. Literally minutes later, I started a separate conversation where I entered "based on our previous conversation today on X topic, please expand but in this other direction" and it literally hallucinated the entirety of our previous conversation. It can't even copy-paste from itself its so bad right now.
I must have missed the announcement of the ChatGPT time machine that makes features launched today available a few days ago.
memories (Score:2)
The front end I use has its own memory system. Seems to work, too.
https://docs.windsurf.com/wind... [windsurf.com]
I know what you said 7 years ago .... (Score:2)
ChatGPT getting very creepy now.
why is everyone wrong about this? (Score:1)
This update didn't make it draw on your full conversation history, it could already do that, and it didn't expand its memory, IT LIMITED IT. As a user you now have a limited number of "slots" for memory, this update added that limit, which did not exist before, because as soon as it went live i started seeing "saved memory full" at the top of the screen. Looks like it saves 50 to 70 memories, i'm not gonna count. Less than 100 because removing one can reduce more than 1%.
Anyway. Yeah. Everything i've se
This would drive me nuts (Score:2)
Personally I find myself constantly dumping context as it is to prevent the AI from getting too stuck.
price to high (Score:2)