Soundslice Adds ASCII Tab Support After ChatGPT Hallucinates Feature 39
After discovering that ChatGPT was falsely telling users that Soundslice could convert ASCII tablature into playable music, founder Adrian Holovaty decided to actually build the feature -- even though the app was never designed to support that format. TechCrunch reports: Soundslice is an app for teaching music, used by students and teachers. It's known for its video player synchronized to the music notations that guide users on how the notes should be played. It also offers a feature called "sheet music scanner" that allows users to upload an image of paper sheet music and, using AI, will automatically turn that into an interactive sheet, complete with notations. [Adrian Holovaty, founder of music-teaching platform Soundslice] carefully watches this feature's error logs to see what problems occur, where to add improvements, he said. That's where he started seeing the uploaded ChatGPT sessions.
They were creating a bunch of error logs. Instead of images of sheet music, these were images of words and a box of symbols known as ASCII tablature. That's a basic text-based system used for guitar notations that uses a regular keyboard. (There's no treble key, for instance, on your standard QWERTY keyboard.) The volume of these ChatGPT session images was not so onerous that it was costing his company money to store them and crushing his app's bandwidth, Holovaty said. He was baffled, he wrote in a blog post about the situation.
"Our scanning system wasn't intended to support this style of notation. Why, then, were we being bombarded with so many ASCII tab ChatGPT screenshots? I was mystified for weeks -- until I messed around with ChatGPT myself." That's how he saw ChatGPT telling people they could hear this music by opening a Soundslice account and uploading the image of the chat session. Only, they couldn't. Uploading those images wouldn't translate the ASCII tab into audio notes. He was struck with a new problem. "The main cost was reputational: New Soundslice users were going in with a false expectation. They'd been confidently told we would do something that we don't actually do," he described to TechCrunch.
He and his team discussed their options: Slap disclaimers all over the site about it -- "No, we can't turn a ChatGPT session into hearable music" -- or build that feature into the scanner, even though he had never before considered supporting that offbeat musical notation system. He opted to build the feature. "My feelings on this are conflicted. I'm happy to add a tool that helps people. But I feel like our hand was forced in a weird way. Should we really be developing features in response to misinformation?" he wrote.
They were creating a bunch of error logs. Instead of images of sheet music, these were images of words and a box of symbols known as ASCII tablature. That's a basic text-based system used for guitar notations that uses a regular keyboard. (There's no treble key, for instance, on your standard QWERTY keyboard.) The volume of these ChatGPT session images was not so onerous that it was costing his company money to store them and crushing his app's bandwidth, Holovaty said. He was baffled, he wrote in a blog post about the situation.
"Our scanning system wasn't intended to support this style of notation. Why, then, were we being bombarded with so many ASCII tab ChatGPT screenshots? I was mystified for weeks -- until I messed around with ChatGPT myself." That's how he saw ChatGPT telling people they could hear this music by opening a Soundslice account and uploading the image of the chat session. Only, they couldn't. Uploading those images wouldn't translate the ASCII tab into audio notes. He was struck with a new problem. "The main cost was reputational: New Soundslice users were going in with a false expectation. They'd been confidently told we would do something that we don't actually do," he described to TechCrunch.
He and his team discussed their options: Slap disclaimers all over the site about it -- "No, we can't turn a ChatGPT session into hearable music" -- or build that feature into the scanner, even though he had never before considered supporting that offbeat musical notation system. He opted to build the feature. "My feelings on this are conflicted. I'm happy to add a tool that helps people. But I feel like our hand was forced in a weird way. Should we really be developing features in response to misinformation?" he wrote.
CEO gets free marketing and complains about it (Score:2, Informative)
News at 11
Re:CEO gets free marketing and complains about it (Score:5, Insightful)
It wasn't free marketing, either the company suffer reputational damage for not supporting something a stupid AI said they could which will cost them money in lost customers or they develop a new future which will cost them money. Your definition of definition is very strange.
Re: (Score:2)
Your definition of definition is very strange.
And that'll teach me to proofread, it should read Your definition of free is very strange.
Re: (Score:2)
or they develop a new future which will cost them money
Potentially a problem in the general case, but this was a trivial feature to implement compared to what the product already does. The developer was not incompetent so this story has a happy ending. Should we protect incompetence?
Re: (Score:2)
Should we protect incompetence?
Why did you go to the trouble of inventing a strawmen argument?
Re: (Score:2)
So, we have your vote, and it is yes.
I wonder if intelligent has an opinion on this.
Re: (Score:2)
So we went from the false argument "its free advertising" to "should we protect incompetence".
You sure are competent in mental gymnastics at least.
Re: CEO gets free marketing and complains about it (Score:3)
The complaining is also free marketing, can't blame the guy.
The AIs made me write a new intuitive feature for my app!
Next week: AI helped write hot new feature and ship it over a weekend.
Shareable ascii tab rickroll link / covert viral ad campaign in 3.. 2..
Just a taste (Score:1)
of how AI is taking over. Not in any kind of superintelligence way, but in permeating our online cultural conversation, to the point where the line is blurred between hallucinations and reality.
Orwellian result (Score:1)
ChatGpt says this works, so we better obey our AI mastersand make this work.
Hm, maybe I can convince ChatGpt to tell everyone that short, fat guys with bad knees are attractive.
Ignore this. (Score:2)
Just posting to undo accidental moderation.
ChatGPT's next hallucination (Score:2, Funny)
No dupes on Slashdot.
So the extreme hallucinations are still not fixed? (Score:2, Funny)
While hallucinations cannot really be fixed in LLMs, I had thought they would at least have gotten the more extreme ones under control. Apparently that cannot be done either. Such an impressive "knowledge" tool!
Re:So the extreme hallucinations are still not fix (Score:5, Informative)
LLMs literally "hallucinate" every answer, it's just that those hallucinations coincide with reality a surprisingly high percentage of the time.
If your photos app is able to "erase" people from a picture, it has no idea what's behind that person. It hallucinates what might be behind that person, and gets it right a surprisingly high percentage of the time. But the pixels it generates do NOT represent what was actually behind that person, they are completely and fully generated from the "imagination" of the AI.
LLMs do the same, just with words.
Re: (Score:1)
LLMs literally "hallucinate" every answer, it's just that those hallucinations coincide with reality a surprisingly high percentage of the time.
Strictly speaking this is true, but aren't you trivializing the situation? Everybody understands (or rather, I believe) that the fundamental data generation from LLMs is working fine, but data integrity/consistency checking is lacking, either at the time of output or during training. But you're kind of missing the forest for the trees.
Re: (Score:2, Interesting)
Everybody understands (or rather, I believe) that the fundamental data generation from LLMs is working fine, but data integrity/consistency checking is lacking
The fundamental data generation from LLMs is working fine if you're happy with neither you nor the tool knowing whether the output is any good unless you a) understand the subject and b) scrutinize 100% of the output.
you're kind of missing the forest for the trees
Your standards are unexplainably low.
Re: (Score:2)
What? No, I find them mostly useless for the reason you mention. But explaining how they work in lieu of discussing how to make them better seems wrongheaded.
Re: (Score:1)
explaining how they work in lieu of discussing how to make them better seems wrongheaded.
If anyone knew how to make them better, they would be doing it. There are thousands of developers working on that right now. This is a fundamental flaw which suggests that LLMs are only going to be one small piece of a functional AI which you can actually trust. You can't do trustworthy fact checking with more hallucination, so you can only make incremental improvements by having a LLM check itself, or even using a different LLM to check output.
Re: (Score:1)
And that is my point. They had several years now and there are foundations and tools for automated deduction, there are fact-bases, yet they still have not gotten even the most obvious hallucinations fixed. That implies they cannot with what is known and at the very least quite a bit of fundamental research is needed. And that means decades or longer until this can be fixed. If it can be fixed.
Re: So the extreme hallucinations are still not fi (Score:2)
I agree with all of the parts except the time estimate. I'm not saying it won't take that long, only that it's not predictable.
Re: (Score:2)
Well, time estimates are tricky. But I have had a look into tech history in the last few years, because I was puzzled why IT and IT security issues well known when I got my first degree (about 35 years ago) are still not fixed or even close to be fixed today. What I realized is that tech progress is much, much slower than it appears to be and that IT is in no way special compared to other disciplines. In fact, it seems to be one of the slower ones due to higher complexity. I do think I have a reasonably goo
Re: (Score:3)
I'm actually fine with the results possibly being "hallucinated"... but that's because I'm mostly using LLM to help me flesh out details in the Dungeons and Dragons game that I run. Doesn't matter much if facts are inaccurate if they're about something that was imaginary in the first place, as long as I make sure they're used in a consistent way and they help me make the game fun.
Re: (Score:2)
It's always been my comment that:
AI is great, as long as it doesn't have to be correct.
If you just want a picture, it can do that. If you want a picture of a specific person, it'll likely be wrong.
If you want it to write a song, it can do that. If you want it to write the sheet music for Beethoven, I wouldn't trust it.
That's the problem I have with AI at work. They keep asking us to integrate it into our Business Intelligence work, but I wouldn't trust it to analyze my business at all. Those answers a
Re: (Score:2)
Doesn't matter much if facts are inaccurate if they're about something that was imaginary in the first place, as long as I make sure they're used in a consistent way
That's part of my point. Although the penalty for failure is low in your use case, the AI won't be able to hallucinate consistently because that's not how hallucinations work. Therefore the output won't be consistent even to itself, and you will still have to doctor it. As you say, this is not a big problem for you, but it does illustrate the fundamental limitations of the technology.
Re: (Score:3)
I know that people are missing AI mistakes, because I've caught myself doing it. Things like trusting a Google AI summary, only to find out that it got a key detail wrong.
For example, I searched for "How many students graduated from Klein High School this year?" It confidently spit out 4,500. But looking at the sources, that number came from the total for the entire school district. I almost ran with that number, thinking it got it right.
I suspect lots of those kind of mistakes happen.
AI accuracy can be imp
Re: (Score:2)
Almost Google results summary I've paid attention to has been wrong. Though I only read it when I feel qualified to catch the mistakes.
Having a second agent validate the output is reasonable, but unimaginative. Imagine it had to assess each training datum (like a prompt) before it was trained, then it would train on that datum along with its assessment. So it would start to notice--when it sees all those marketing sites say a certain product has all these great uses--and when it sees a forum of the target a
Re: (Score:3)
With regards to cross-checks, there are some use cases that are already happening.
For example, there are a number of code security validation AI tools commercially available. They specifically look for security risks. Such AI tools aren't likely to fall into the same hallucinations that the original code generator did.
A second approach is what Microsoft Copilot does with math. If you ask it a question that requires calculations, the LLM spits out an answer and a rationale, and a separate math-focused AI doe
Re: (Score:2)
I suspect lots of those kind of mistakes happen.
With most people just copying what they "found on some website" before? Yes. Only now we do not even have that flawed website anymore.
AI accuracy can be improved, by using a second, error-checking AI. This is very similar to a human proofreader. Two AIs with two different focuses, aren't likely to make the same exact mistake. But of course, that kind of cross-checking costs more, so some will skip it.
Unlikely. If that was actually possible to a worthwhile degree, it would be done. But it is not. And that means attempts at it failed, because this will have been tried. There was more than enough time for it now.
Re: (Score:2)
With most people just copying what they "found on some website" before? Yes. Only now we do not even have that flawed website anymore.
AI summaries on Google and Bing both do have links to websites from which the summary is drawn.
If that was actually possible to a worthwhile degree, it would be done. But it is not. And that means attempts at it failed, because this will have been tried. There was more than enough time for it now.
This *is* happening already.
For code generators, AI tools like Mend do look for things like security vulnerabilities created by generated code.
Copilot LLM delegates math problems to a separate math-focused AI, resulting in better quality when the response involves math.
It doesn't seem far fetched that a secondary AI could cross-check reference citations, or other common problems with current AI chatbots.
All these
Re: (Score:2)
Actually, it seems more and more far-fetched as time progresses and the prediction of a golden age get more desperate and less based on facts.
My take is in the coding-space, we will get cheap, easy to replace "prompt engineers" and expensive, hard to replace actual coders that occasionally will let AI help them but do most things themselves because writing good code is faster than fixing code written by AI. Yes, I am well aware that all those that find AI very helpful will go into the "cheap" class and that
Re: (Score:2)
Count me among those who are in denial then.
Re: (Score:2)
Then good luck to you, you are going to need it. Might take a few years, but I recommend you prepare.
Re: (Score:2)
To me, "being prepared" means educating myself thoroughly on this new technology, so that I know exactly what it can and cannot do, what it's good for and not good for, and how to leverage it to my own advantage. Looking away and pretending it doesn't exist, is not any kind of "preparation."
Re: (Score:2)
That is oversimplified to a degree that it is not true anymore. What you probably mean to say is that LLMs do not verify their answers and only have a probability of it being correct.
Re: (Score:2)
LLMs literally "hallucinate" every answer, it's just that those hallucinations coincide with reality a surprisingly high percentage of the time.
This is exactly how I understand the situation. In other words, instead of the certainty with which each answer is given, the response should start with: "I don't quite recall from where, but I once heard the following..."
I like this guy's creativity (Score:3)
If you have an ice cream shop, and people are talking up a flavor you don't actually have, you'd be smart to add it! It's free publicity.
If you're an app developer, and AI is attributing things to your software that it doesn't have, well why not add it! It just might draw more people to your app.
Or ChatGPT can see the future (Score:3)
It knew this feature would be part of the app, so it reported it as being there, because like many psychics, it finds it difficult to tell what's in the present and what's in the future... ;)
Good response (Score:3)
The only part of this story I have a problem with is "decided to actually build the feature -- even though the app was never designed to support that format". Yeah, no shit, if the app was designed to support that format, he wouldn't have to build the feature because it would already be there. That's not an "even though", it's literally a requirement for it to happen. Author, editor, or whoever completely fails not only at basic English, but at logic, too.
But, this is also a hilariously simple feature. It's a format conversion! What the app already does is orders of magnitude more difficult, and the new feature can almost certainly use existing code to do most of the job. The output part already exists, this is only a new input part, and immensely simpler than the existing input part. This doesn't make it not news, of course.
Finally, the answer to the dev's question "Should we really be developing features in response to misinformation?" is "why not"? There is not a single solitary reason not to do so. And, to close the circle, he could not have developed features which people said did exist, because they would already have been developed. But even more accurately, he did not develop the feature in response to misinformation. He did it in response to demand, or put another way, the customer response to misinformation.