Sir Tim Berners-Lee Lays Out Nightmare Scenario Where AI Runs the Financial World (techworld.com) 131
The architect of the world wide web Sir Tim Berners-Lee has talked about some of his concerns for the internet over the coming years, including a nightmarish scenario where artificial intelligence (AI) could become the new 'masters of the universe' by creating and running their own companies. From an article: Masters of the universe is a reference to Tom Wolfe's 1987 novel The Bonfire of the Vanities, regarding the men (and they were men) who started racking up multi-million dollar salaries and a great deal of influence from their finance roles on Wall Street and in London during the computerised trading boom pre-Black Monday. Berners-Lee said, "So when AI starts to make decisions such as who gets a mortgage, that's a big one. Or which companies to acquire and when AI starts creating its own companies, creating holding companies, generating new versions of itself to run these companies. So you have survival of the fittest going on between these AI companies until you reach the point where you wonder if it becomes possible to understand how to ensure they are being fair, and how do you describe to a computer what that means anyway?"
Ah, yes... (Score:3)
Sir Tim Berners-Lays, father of the chip.
Re: (Score:3)
Prove that AIs don't already run Wallstreet...
Actually, bots run wall street. All you need is a quick google search on HFT and Flash crash. The one in 2010 was attributed to HFT bots. I'm willing to bet that there have been others, we just haven't heard about them. The stock markets have built-in safeguards but I'm sure that they have their weaknesses too.
As for AIs, we haven't developed one that can handle unexpected market changes yet. Perhaps in the future. That being said, the bigger concern isn't the use of AI's but who gets access to them.
Re: (Score:2)
"All you need is a quick google search on HFT and Flash crash. The one in 2010 was attributed to HFT bots."
It is for this reason that I propose a random small amount of time (say, 30 sec to 2 min?) be added to all trades.
Re:Ah, yes... (Score:5, Informative)
The one in 2010 was attributed to HFT bots."
The "flash crash" was attributed to HFT bots in the immediate aftermath. A fuller investigation determined that they were not the cause, and when HFT programs were disabled, that actually made the crash worse because of reduced liquidity. Programmed trading may have been a cause, but HFT != programmed trading.
It is for this reason that I propose a random small amount of time (say, 30 sec to 2 min?) be added to all trades.
What "problem" are you trying to solve?
HFT algorithms reduce transaction costs and reduce prices for everyone else.
Your "solution" will increase costs, and provide no benefit to the "little guy" (who doesn't have direct access to the market in the first place), since HFTs will just wait until the last microsecond before the window closes to place their trade.
A bigger problem is that most transactions (by volume) don't even happen in the public markets.
They happen in unregulated and opaque dark pools [wikipedia.org], and your "solution" will just make that even worse.
Re: (Score:2)
It is for this reason that I propose a random small amount of time (say, 30 sec to 2 min?) be added to all trades.
I'd like to see the code for your random number generator.
"Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin." - John Von Neumann
Re: (Score:2)
The top 1% don't need AI, they have II (Inside Information).
Re: (Score:3)
False dichotomy. The "average investor" is not a day-trader reading tea leaves. The "average investor" is someone who has a 401k, mutual fund, pension or some other pooled investment. These institutional investments would almost certainly be the first to use such an AI and so the "average investor" wou
Re: Ah, yes... (Score:1)
I'm still convinced in my own conspiracy theory mind that the 2010 crash was caused by a derivative of the dual slit quantum eraser experiment used to reduce trade lag. Too bad the poor bastard that made it create the causality loop can never publish his proof of time travel due to those pesky insider trading laws he broke.
Re: (Score:2)
Yes, and the term "A.I." has been cheapened just like how "nanotech" includes toothpaste instead of Drexler's little machines.
Despite true "A.I." being an inestimable way off some dumb bots not much more capable than the current dumb bots may end up being capable of delivering such a nightmare.
Re: Ah, yes... (Score:2)
It's always just a bad apple ruining it for the rest of us. The system is never to blame.
credit scores and algorithms already decide. (Score:2)
bots run them. they make the "decision".
what the guy who inputs the inputs into the application does though is know after a while what gets a pass and what doesn't, so thell their clients to tell them that say that you have this and this much and can get this and this.
it's not really ai of course. but berners lee seems to be out of touch of how regular people get mortages and such in todays world. sure if you're a sir you get to meet the bank manager and he can make a decision to give you mortage on your 2
Re: (Score:1)
Feel the Berners!
As opposed to ... (Score:4, Funny)
... Nightmare Scenario Where AI Runs the Financial World
As opposed to the natural stupidity that currently runs it? How could the AI be worse?
Re: (Score:3)
As opposed to the natural stupidity that currently runs it? How could the AI be worse?
The AI may just go the the biggest and best profit regardless of consequences. For example if it is profitable for it to crash the markets, have millions out of work and companies going bankrupt all so it can make a few thousand dollars bigger profit it may decide to do that simply because nobody forgot to program it with the negative consequences of such a decision because it is not often that social consequences are large enough to result in serious push back again raw capitalism.
Of course given enoug
Re:As opposed to ... (Score:5, Insightful)
Re: (Score:2)
At least a human has, somewhere in it's limbic system, some evolutionary instinct that says if you hoard all of the food and don't kill all the other people, you will either get your head bashed in by your own tribe or you will lack a tribe at all and the next tribe over the hill will bash your head in.
Of course AI's answer to that problem is Skynet, so there's that.
Re: (Score:2)
Some people are lacking those instincts, and they have a good chance to be more successful, at least on the shorter term.
Re: As opposed to ... (Score:2)
Thank you, Dr Pedant, for that enlighteningâ commentary.
Re: (Score:3, Insightful)
The AI may just go the the biggest and best profit regardless of consequences. For example if it is profitable for it to crash the markets, have millions out of work and companies going bankrupt all so it can make a few thousand dollars bigger profit
The hell are you talking about? That's exactly how humans are ruining economies right now. Get out of your cushy white collar office space sometime and see how your top tier entitled privileged lifestyle compares to real people in the real world.
Re: (Score:2)
And so might a human trader. See the Soros attack on sterling!
I am not sure the computers will be any different than the psychopaths currently pulling the strings.
Re: (Score:2)
I am not sure the computers will be any different than the psychopaths currently pulling the strings.
They won't do anything different, but they will do as much damage in a millisecond what it would take the humans days to achieve.
Re: (Score:2)
Came here to say this. This nightmare scenario is no worse than current reality, unless you see the fact that an AI is doing instead of a human to be worse somehow.
Mankind is already ruled by a distributed resource-management AI that is indifferent to human suffering, it's called capitalism. It seems to consider the executive class to be "goodlife."
Re: (Score:2)
Bots could do it more often and do it all night.
Re: (Score:2)
As opposed to the natural stupidity that currently runs it? How could the AI be worse?
If there's one thing I've learned in life, it's that things can always get worse. ALWAYS.
Re: (Score:3)
I've actually been doing R&D into precisely one of the scenarios listed in the summary - using AI techniques for lending (e.g. mortgage) approvals.
The concerns listed are actually part of my focus - how do we get to the bottom of why the decision was made. There are regulatory blocks against simply doing without being able to justify already.
But here's the thing, we can use existing data sets of applications for things like mortgages, coupled with historical data of how the person borrowing the money pl
Re: (Score:1)
Re: (Score:1)
And the differnce is? (Score:5, Interesting)
So, the financial industry will be controlled by heartless automatons, but now they will make intelligent, logical decisions? Seems like a net gain.
Also, MOTU is a reference to He-Man.
Bravo (Score:3)
I came to post exactly the same thing. They ask how the AI will be fair. Better to ask, it what way could an AI NOT be fair? Harder to add unfairness into a system than it is to have the system be neutral.
What I'd be almost more excited about is an AI HR system so the HR system could not ignore you based on personal biases and/or relationships.
Goal driven (Score:2)
Those who set up the model for use will add in their preferred bias so there is no reason to be excited. Those who don't like the results will not accept them.
I'm reminded of some earlier post here about how all those HR questions about attitudes to family are just a back door method of screening for homosexuals. Don't expect HR people to be ethical (or even properl
Re: (Score:2)
Re: (Score:3, Funny)
...and instead decides to start keeping it all.
And do what? Head off to Bloomingdale's?
Re: (Score:2)
"but now they will make intelligent, logical decisions? Seems like a net gain."
Logic doesn't lead 'AI' anywhere, so whether something is a net gain depends on its programmed goals, and the transaction rules (ie regulations) they (the owners of the AIs) have to follow.
Re: (Score:3)
As reused cynically by Tom Wolfe in his novel. Tim Berners-Lee is using it in that context despite being the right age to have once been bombarded by He-Man advertising.
More AI (Score:5, Insightful)
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
More AI bullshit. We can barely even create functional regular software.
If you look at some of the common behaviors of your species, you can barely create functional regular people.
Re: (Score:2)
Chinese Wall (Score:4, Interesting)
Nightmare scenario for traders, benefit for everyone else. The AI likely would have Chinese walls built into them to prevent collusion and insider trading. I for one look forward to my innately honest (and auditable/examinable) AI masters
Re:Chinese Wall (Score:4, Insightful)
The AI likely would have Chinese walls built into them to prevent collusion and insider trading.
Ha ha ha. Hahahahahaha. Well played, sir.
Re:Chinese Wall (Score:5, Insightful)
There's no guarantee an AI will be auditable. Lots of AIs are too complicated to understand how they work.
Re: (Score:2)
They will be auditable as far as from where they are getting data. Neuron networks might be bit magic, but if you put firewall between them and database which holds customer information, it won't base its decisions on unavailable data. On the other hand, it is hard to audit one guy talking about few upcoming transactions with other guy over a lunch.
Having 100% audit over inputs is a lot more than we have currently with traders. With traders, you probably control only 80-90% of inputs they are getting, 0% of
Re: (Score:2)
Not all AI is generalized. In fact, so far no AI is generalized. So, no, it wouldn't be able to learn the concept of fairness unless that was built into it. Not yet.
OTOH, others have rightfully raised the question of just how fair the current system is. My version of the question is "What would be a precise definition of fairness in an economic trading system?". I have my doubts that anyone can give a defensible answer.
Re: Does AI learn (Score:2)
It's fair when my side disproportionately benefits, of course.
Just more fiction for mental masturbation (Score:2)
I mean when it comes to money, it doesn't have any real meaning to a computer A.I. It does, however, mean pretty much everything to human beings directly involved in the market, in trading, and in the business of trying to generate maximum wealth.
That ensures that A.I. will never be allowed to spin out of control to create the "nightmare scenarios" one can create in their imagination.
It will only be implemented as far as it is able to assist people in performing the tasks they wish to perform manually anywa
Perfected Central Planning! (Score:2)
Well if AI runs the investment universe and does a perfect job, isn't that central planning perfected? I mean that was always the problem wasn't it? Investment was risky and uncertain so investors had to compete and a single government entity couldn't do it. Now if AI does it then lovely, capital is allocated properly to the right companies who produce the best results with it. Great!
Re: (Score:2)
They tried to do this in Chile in the 70s: https://en.wikipedia.org/wiki/... [wikipedia.org]
Or a human (Score:2)
Re: (Score:2)
This may not be as hard as it sounds. The real problem is that right now our defined outcome for how trades and commerce in the market is supposed to work is that it's supposed to generate profit. But the actual goal that most people have for the market is that it generate prosperity. Which it doesn't really do very well. So if you were to give an AI "prosperity" as a goal for running the markets, it might actually have quite a bit better of an outcome than what we are doing now, with greed, essenti
Re: (Score:2)
Re: (Score:2)
The AI discovers a drug that leaves its users in crippling pain, but extends their lifespan by an average of 30 minutes. The AI forces everybody to take the drug because it's optimizing for lifespan.
Congratulations, you completely fucked up. This isn't an easy problem to solve.
Loss of control may not be so bad (Score:2)
That seems unlikely... (Score:2)
Traders are finding it too difficult to make money with High Frequency Trading (HFT) since everyone and there mother is doing it.
http://www.investopedia.com/news/high-frequency-trading-flash-boys-losing-steam/ [investopedia.com]
Re: (Score:2)
Not really true - what's happening is that algorithmic trading has become harder (because you're now 'playing' against other algorithms and not just slow humans, and because the spreads have got smaller). Now, if you want to trade any real volume algorithmically, you're going to need to be really, really good at it from the outset. Had you started 5 years ago and evolved you'd be doing just fine. If you start now, or started back then and failed to evolve then you've got a really steep hill to climb.
That sa
Feed The Greed. (Score:2)
By any means possible. Automated self serving, self learning digital cash gobblers.
smash the robots and let's see how the money men (Score:2)
smash the robots and let's see how the money men like paying 20K-40K + court and med costs a year to lock someone up
Just how bad can it be? (Score:2)
Mr Bundy was useless, but I can't really see him doing a worse job than the market right now. Despite his pitiful salary as a shoe salesman, he managed to keep his family housed, fed, and mostly clothed. That takes a degree of financial smarts that many don't have.
Don't think so (Score:3)
Yeah i don't think so. The only reason high frequency trading machines exist now is because investors understand these machines one advantage over humans is speed. They also don't allow HFTM to have control of billions of dollars for investment, usually limiting its pool of funds it works with in order to limit risk. The HFTM owners have also 'gamed' the system to a certain extent. If a bug causes an algorithm to go haywire and negatively affect the market, the transactions can be 'rolled back' so the investor doesn't loose all their cash.
No one in the financial world right now is ready to put AI in control of billions of capital, thousands of employees, and dozens of production chains in the sole hands of an unaccountable AI with no personal vested interest in the business. More likely? In a few years we get a personal assistant like Alexa that can provide suggestions to CEO's on what new business decisions would be the best course of action. The AI will augment, not replace.
Re: (Score:2)
...and yet right now, algorithms do the trading, and humans just alter the config of the algorithm. It's not too much of a reach to expect that in a few years what takes 10 traders today will take 1 trader because the algorithms will be better and controlled in more circumstances by machine learning.
Fast forward that a bit further and maybe one person controls the same volume of trading as one of the bigger algo-trading companies do today. Keep going, and maybe it's really down to 10 guys around the world,
Re: (Score:2)
Bullshit ... (Score:1)
Accelerando Reference (Score:2)
You don't, that's what. (Score:2)
..and how do you describe to a computer what that means anyway?
You don't. Nothing we have is actually sentient, sapient, conscious, or has a conscience, for that matter. Some decisions regarding human beings can be made with pure logic alone, but many many more require much much more from the mind making the decision than just pure logic. Haven't there been more than enough cautionary tales from science fiction to illustrate this point? Or do we really have to, once again, learn a big important lesson the hard way? The more time that passes, the more I read, the more
Capitalism isn't fair (Score:2)
Shades of Grey (Score:2)
Assuming people actually did just what you suggested and made access to "food, shelter, healthcare, education & transportation" available to everyone. What do you do with the people who do not take advantage of it? Do you have to take all of it, or can you just take the food and transportation? Are you required to get the education to get the rest? Is forcing people to use it just another form of slavery? You can lead a horse to water, but you can't make it drink.
Fair? (Score:2)
... you reach the point where you wonder if it becomes possible to understand how to ensure they are being fair ...
We seem utterly unable to solve this problem even with the current (allegedly) human banksters - I'm not so sure it will be significantly different with AI running the markets. Also, this is but one tiny portion of the grief that might lie ahead of us if AI becomes ubiquitous, omniscient, and omnipotent. We don't need to worry about AI in the financial sector - we need worry about AI, period.
AIs Understand Fair (Score:1)
Fictional AI in 1964 Great Time Machine Hoax novel (Score:2)
https://en.wikipedia.org/wiki/... [wikipedia.org]
It runs the world by printing out business letters (and checks) that hire people to expand itself.
https://www.worldswithoutend.c... [worldswithoutend.com]
"Chester W. Chester IV, sole surviving heir of eccentric millionaire-inventor Chester W. Chester I, has entered into his inheritance: a semi-moribund circus; a white elephant of a run-down neo-Victorian mansion furnished with such hot items as TV sets shaped like crouching vultures; the old gentleman's final invention, a mammoth computer whose so
On breeding friendlier corporations and AIs (Score:2)
Part of something I posted in 2000 to Doug Engelbart's "Unifinshed Revolution II" colloquium touching on corporations as "AIs":
http://www.dougengelbart.org/c... [dougengelbart.org]
========= machine intelligence is already here =========
I personally think machine evolution is unstoppable, and the best hope for humanity is the noble cowardice of creating refugia and trying, like the duckweed, to create human (and other) life faster than other forces can destroy it.
Note, I'm not saying machine evolution won't have a human compone
Re: This is cephalapod (Score:2)
I welcome our Cephalapod overlords and would like to point out that as a software engineer I would be very capable of taking care and feed AI entities