TikTok Preparing a US Copy of the App's Core Algorithm (reuters.com) 57
An anonymous reader quotes a report from Reuters: TikTok is working on a clone of its recommendation algorithm for its 170 million U.S. users that may result in a version that operates independently of its Chinese parent and be more palatable to American lawmakers who want to ban it, according to sources with direct knowledge of the efforts. The work on splitting the source code ordered by TikTok's Chinese parent ByteDance late last year predated a bill to force a sale of TikTok's U.S. operations that began gaining steam in Congress this year. The bill was signed into law in April. The sources, who were granted anonymity because they are not authorized to speak publicly about the short-form video sharing app, said that once the code is split, it could lay the groundwork for a divestiture of the U.S. assets, although there are no current plans to do so. The company has previously said it had no plans to sell the U.S. assets and such a move would be impossible. [...]
In the past few months, hundreds of ByteDance and TikTok engineers in both the U.S. and China were ordered to begin separating millions of lines of code, sifting through the company's algorithm that pairs users with videos to their liking. The engineers' mission is to create a separate code base that is independent of systems used by ByteDance's Chinese version of TikTok, Douyin, while eliminating any information linking to Chinese users, two sources with direct knowledge of the project told Reuters. [...] The complexity of the task that the sources described to Reuters as tedious "dirty work" underscores the difficulty of splitting the underlying code that binds TikTok's U.S. operations to its Chinese parent. The work is expected to take over a year to complete, these sources said. [...] At one point, TikTok executives considered open sourcing some of TikTok's algorithm, or making it available to others to access and modify, to demonstrate technological transparency, the sources said.
Executives have communicated plans and provided updates on the code-splitting project during a team all-hands, in internal planning documents and on its internal communications system, called Lark, according to one of the sources who attended the meeting and another source who has viewed the messages. Compliance and legal issues involved with determining what parts of the code can be carried over to TikTok are complicating the work, according to one source. Each line of code has to be reviewed to determine if it can go into the separate code base, the sources added. The goal is to create a new source code repository for a recommendation algorithm serving only TikTok U.S. Once completed, TikTok U.S. will run and maintain its recommendation algorithm independent of TikTok apps in other regions and its Chinese version Douyin. That move would cut it off from the massive engineering development power of its parent company in Beijing, the sources said. If TikTok completes the work to split the recommendation engine from its Chinese counterpart, TikTok management is aware of the risk that TikTok U.S. may not be able to deliver the same level of performance as the existing TikTok because it is heavily reliant on ByteDance's engineers in China to update and maintain the code base to maximize user engagement, sources added.
In the past few months, hundreds of ByteDance and TikTok engineers in both the U.S. and China were ordered to begin separating millions of lines of code, sifting through the company's algorithm that pairs users with videos to their liking. The engineers' mission is to create a separate code base that is independent of systems used by ByteDance's Chinese version of TikTok, Douyin, while eliminating any information linking to Chinese users, two sources with direct knowledge of the project told Reuters. [...] The complexity of the task that the sources described to Reuters as tedious "dirty work" underscores the difficulty of splitting the underlying code that binds TikTok's U.S. operations to its Chinese parent. The work is expected to take over a year to complete, these sources said. [...] At one point, TikTok executives considered open sourcing some of TikTok's algorithm, or making it available to others to access and modify, to demonstrate technological transparency, the sources said.
Executives have communicated plans and provided updates on the code-splitting project during a team all-hands, in internal planning documents and on its internal communications system, called Lark, according to one of the sources who attended the meeting and another source who has viewed the messages. Compliance and legal issues involved with determining what parts of the code can be carried over to TikTok are complicating the work, according to one source. Each line of code has to be reviewed to determine if it can go into the separate code base, the sources added. The goal is to create a new source code repository for a recommendation algorithm serving only TikTok U.S. Once completed, TikTok U.S. will run and maintain its recommendation algorithm independent of TikTok apps in other regions and its Chinese version Douyin. That move would cut it off from the massive engineering development power of its parent company in Beijing, the sources said. If TikTok completes the work to split the recommendation engine from its Chinese counterpart, TikTok management is aware of the risk that TikTok U.S. may not be able to deliver the same level of performance as the existing TikTok because it is heavily reliant on ByteDance's engineers in China to update and maintain the code base to maximize user engagement, sources added.
Sounds costly. (Score:4, Insightful)
Kinda sounds like a lot of unnecessary work... unless it's not about money.
Re:Sounds costly. (Score:4, Insightful)
Re:Sounds costly. (Score:5, Insightful)
Sounds like *they* don't know what they're doing nor why. The recommendation algorithm isn't the problem, it's the hoovering up of personal information and surveillance that's the problem. This project shouldn't be any more difficult than forking the repo(s) and changing a few configuration files to point at a new US-based hosting environment with Azure/AWS/GCP/pick-your-poison.
A new hosting environment would not help, either. TikTok is already hosted in the USA, on servers owned by TikTok, Inc -a Los Angeles, CA -USA- company.
The issue is the Chinese company (ByteDance) that owns TikTok has unrestricted access and control. Even tho TikTok has stated that this is not the case, and that they would not allow unauthorized or inappropriate access... they got caught-in-the-act with ByteDance, China employees accessing location data from posts by journalist-types critical of China and (allegedly) using it to identify and threaten the journalists and their families -without needing any permission or help from TikTok, USA employees.
It happened. They promised steps would be taken to prevent future happenings. But (some of) the code is still written and updated from China... so it certainly CAN happen.
Fool me once... Meh. Fool me twice? No.
As long as ByteDance owns TikTok, you cannot ensure that rules will be followed.
Re:Sounds costly. (Score:5, Informative)
December 2022
China’s ByteDance Admits Using TikTok Data to Track Journalists [securityweek.com]
Re: (Score:2)
Thank you for adding the citation to my statement.
Re: (Score:3, Insightful)
it's the hoovering up of personal information and surveillance that's the problem.
Nobody serious thinks that's the problem.
The problem is deciding what gets promoted. Think propaganda. The recommendation algorithm is everything.
China don't want the information to get out about how they are doing this so effectively, so they are trying to recreate a "more open version" that hides what they are doing in the secret version.
Re: (Score:2)
Tik Tok sends less information back to the mother ship than Farcebook, but the US government doesn't seem to think that the latter is an issue.
Re: (Score:2)
You mean Farcebook and Google? Both are infested with 'former' US and UK intel agency staff, and neither's algorithms will allow their users to diverge far from the Official Brainwashington Narrative. Tik Tok is just the one that they don't control yet. Twitter made a rather lackluster attempt to go its own way, but was quite quickly brought back into the fold. Telegram is still too small and diffuse to go after yet, but if it gets more popular in the NATO+ countries it too will be in the crosshairs.
Re: (Score:2)
Re:Sounds costly. (Score:5, Insightful)
The recommendation algorithm isn't the problem, it's the hoovering up of personal information and surveillance that's the problem.
I disagree. Hoovering up information of personal information isn't much of a problem, I reckon, outside of the potential to blackmail key figures.
I think the megascale data, and the recommendation algorithm, ARE the problem. If you can run ads then you can influence public opinion in a small way and the influence can be traced to you. But if you own the recommendation algorithm then you can make subtle tweaks to the recommendations, ones that are completely invisible to anyone -- to the general public, to the regulatory bodies, to legislators, to the press. And if you have megascale data then you'll be able to make those tweaks much more effectively.
Thought experiment. Imagine bytedance is instructed to swing the US presidential vote by 0.5% in a swing district, maybe to a candidate they believed would be more anti-Taiwan or more likely to lead to US destabliziation. They have a huge amount of data about their userbase, and can infer from it reasonably good approximations to a whole load of further variables (which way they're likely to vote, how likely they are to go and vote). Now they plug into their ML training model all of these factors, and next train their model to optimize video recommendations that are more likely to elicit a "like" about a get-out-and-vote video by the right people and less likely for the wrong people.
Result: thanks to megascale data, and thanks to owning the recommendation algorithm, they've influenced maybe 10,000 people to vote who otherwise wouldn't have. Moreover their influence is COMPLETELY UNDETECTABLE. It's not like ads where there's a paper trail of who bought the ad. It's not possible to look at the pattern of recommendations and establish a pattern. No one will ever know that the influence happened.
How easy would it be for a Bytedance employee to stick in those training parameters? Really easy. Folks in the ML community don't generally practice rigorous source code checkins. A data scientist assembles a dataset, kicks off a training job, waits a day, and then uploads the result. There's rarely an audit trail of what they've done. And because the training things are terabytes big, they don't get retained longer than a few weeks.
Re: (Score:3)
Substitute Meta (Farcebook) or Alphabet (Google) for Bytedance and you're observation is equally valid.
Re: (Score:2)
Get caught? Who are you kidding? You really think that the PTB which benefit from the narrative control that this access gives them are going to allow any such thing? Come back after you graduate high school and we'll talk about how the world really is.
Re: (Score:2)
Ironically the issue is that young people are using TikTok to influence public opinion in ways that politicians don't like. TikTok is a very progressive platform, with a lot of popular posts about socialism, anti-Zionism, and other topics that the US ruling class don't like.
I suppose you could imagine a conspiracy where it's all part of the CCP's evil plan to destabilize the US with good ideas, but the same movement exists on Twitter as well. The key difference is that Musk made Twitter's algorithm tuned to
Re: Sounds costly. (Score:3)
If China is involved anywhere you can rest assured that they will be harvesting data or using the system to breach other systems.
Re: (Score:1)
i understand this as: "if you absolutely want to have this, you can have a watered down version, but you will wait not 2 months but one year and good luck with it, no guarantees, you get 150 million users anyway. it's that or we leave, and you explain to those 150 million american users why the whole world can enjoy tiktok but "free america" can't, and hope they still vote for you. fuck you."
Re: (Score:2)
Kinda sounds like a lot of unnecessary work... unless it's not about money.
There’s 170 million customers at risk. Of course it’s about the money. TikTok content isn’t exactly designed to be destined for the hallowed halls of preserved proud history. It’s more like that shit you delete first when you get the storage full/out of memory error.
Re: (Score:2)
It's just ctrl-c ctrl-v on the code and pay a lawyer to draw up the contract transferring ownership.
It's absolutely about the money. The algo is extremely lucrative.
Re: Sounds costly. (Score:1)
git clone is not costly, what is costly is hiding all the crap before opening it. Twitter opened its source code and within hours people found the knobs that they had lied about prior. The COVID platforms for a few countries opened source code and people instantly found how the supposed anonymous system leaked identifying data to the government. NSA key registry?
Trust (Score:4, Insightful)
Don't trust a word that comes from a CCP backed company. You can be damn sure no matter how "separate" Bytedance claims it will be ... it will not be.
Re: (Score:2)
Yeah, absolutely no one is gonna be fooled by this, except those who are doing the fooling.
Re: (Score:2)
Don't trust a word that comes from a CCP backed company. You can be damn sure no matter how "separate" Bytedance claims it will be ... it will not be.
No kidding, right? Shit, I’ll bet they even brag about “anonymizing” data too. Like anyone would be dumb enough to fall for..
Re: (Score:2)
So just the same as every other tech company then. Or are you saying you trust Google, Apple, Facebook, Twitter, Microsoft etc?
I'm not singling you out, it's just interesting that every time there is a story about a US tech company doing something the usual comments about them lying and secretly harvesting all your data and selling it off to the lowest bidder get made. Yet somehow it's notable that Chinese tech companies are equally untrustworthy, and what's more it's because they are backed by the CCP rath
Re: (Score:2)
China wants to burn the free world, the others you mention want to profit from it. We can put regulations and controls in effect to control the profiteers. Maniacs that want to burn the world on the other hand is a whole different kind of sickness.
Re: (Score:1)
Do they though? That just sounds like deranged propaganda to me.
Re: (Score:2)
Stop shilling for the CCP. You sound stupid.
Re: (Score:1)
I'm just looking at who invaded other countries, who supported Israeli genocide, who demands everyone adopt their economic and political models... It's not China.
Re: How funny... (Score:1)
Re: (Score:2)
If I had mod points, I'd mod you up.
Unfortunately, there's one caveat: Neither unrestricted access to guns nor unrestricted free speech will work in a large society with rapid communication and rapid transportation. So what both sides are arguing about is what are the reasonable limits. And neither side is composed of people who will argue honestly.
Re: (Score:2)
I own firearms, have a concealed carry permit, and still I would like to see more controls on gun purchase, licensing and ownership because school shootings being what they are, and things like Uvalde proving beyond a shadow of a doubt that we cannot trust the police to even protect children. Uvalde wasn't the only time that that happened, it is just the one that came to mind first.
And I used to be an NRA member, until they opposed any sort of new controls, then I told them to shove it up their
Re: (Score:2)
One line of code (Score:4, Insightful)
In the past few months, hundreds of ByteDance and TikTok engineers in both the U.S. and China were ordered to begin separating millions of lines of code, sifting through the company's algorithm that pairs users with videos to their liking. The engineers' mission is to create a separate code base that is independent of systems used by ByteDance's Chinese version of TikTok, Douyin
Hasn't China heard of the "copy" command? OK, so two lines of code, since they have to do a search/replace for the server info too.
while eliminating any information linking to Chinese users
OK, so they'd have to filter their userbase and re-run the algorithm too.
But I bet there's a bunch of user manipulation stuff they need to remove too.
Re: (Score:1)
TikTok management is aware of the risk that TikTok U.S. may not be able to deliver the same level of performance as the existing TikTok because it is heavily reliant on ByteDance's engineers in China to update and maintain the code base to maximize user engagement
I'm not sure who's worse, CCP or Tiktok? Only Chinese engineers can write code apparently.
Re: (Score:2)
Also, we should leave "maximize user engagement" for the drug dealers.
Re: (Score:2)
Also, we should leave "maximize user engagement" for the drug dealers.
They are the drug dealers.
Re:One line of code (Score:5, Informative)
It's not about that, it's about exposing the assumed fact that China, through ByteDance uses the algorthm to steer peoples feeds to topics that align with their interests and suppress topics that they don't. Nobody has the algorithm so it can't be directly confirmed but evidence is there:
A Tik-Tok-ing Timebomb: How TikTok's Global [networkcontagion.us]
Platform Anomalies Align with the Chinese Communist Party's Geostrategic Objectives
Given the research above, we assess a strong possibility that content on TikTok is either amplified or suppressed based on its alignment with the interests of the Chinese Government. Future research should aim towards a more comprehensive analysis to determine the potential influence of TikTok on popular public narratives. This research should determine if and how TikTok might be utilized for furthering national/regional or international objectives of the Chinese Government.
Re: (Score:2)
America as a nation doesn't have "to do" a TikTok because the lack of any real centralized control over our media system means it will organically do it for us.
TikTok is doing it's thing now but America, in a media and culture war? Long term victory, all day long. Fox maybe did some brainwashing for sure, but that was American brainwashing, the best kind.
Re: (Score:2)
Takes one to know one Mr. AC. Beep boop.
Re: (Score:2)
Everyone that controls the media does "brainwashing" to promote their own interests, "The power of the press belongs to the man who owns one." I'm not sure who I'm quoting. I used to think it was Benjamin Franklin, but Google doesn't even find a source.
The thing is, in China all businesses are (conceptually) largely owned by the state. (I say "conceptually", as I don't know the legal status of that claim.) In the US the media are owned by...well, it used to be the oligarchs (see. e.g., William Randoph H
Re: (Score:2)
Hasn't China heard of the "copy" command? OK, so two lines of code, since they have to do a search/replace for the server info too.
They need to remove all the code which scans your phone and reports people for non-aligned opinions.
There is obviously quite a lot of that.
Re: (Score:2)
Simple solution (Score:2)
2) Force Apple/Google app stores to remove the apps. (I know doesn't stop the apps from being distributed on like APK sites).
3) DDoS the TikTok servers. I mean IBM now has those fancy quantum computers.
The fact that they need to work on a clone (Score:2)
That's not going to help (Score:2)
Tiktok abdicated a lot of control to the US years ago.
The effort against it now seems to come from a variety of sources, including competitors and of course pro-Israel sources because a lot of young people are clustering there and escaping the narrative.