X Open Sources Its Ranking and Filtering Algorithms (techcrunch.com) 57
An anonymous reader shared this report from TechCrunch:
X is significantly expanding its open source codebase, which includes the app's "For You" algorithm and its core ranking engine, and adding a feature that will let users see if their account or posts have been impacted by any of its ranking systems, the social network said on Thursday. The company is making the source code for the "For You" timeline, the default feed you see when you open the app, available on GitHub under the Apache v2 license. It's also expanding its previous efforts to open source parts of its codebase to add more detail, including the model configuration, filter, and core ranking system details. That means it includes the parameters used to weight different signals — key to understanding which posts are actually displayed. This also makes the codebase roughly 10 to 15 times larger than it was before.
"You'll get the core ranking code that pulls posts and ranks them for any given user and assembles the feed," X's VP of Product Keith Coleman told TechCrunch in an interview ahead of the announcement. "You can see the systems that filter out potentially problematic, rule-violating content...And some of those systems, like the ranker and the score, you can even run yourself outside the company."
"This is the kind of thing that I think people will be fairly shocked that we are releasing," he added.
In addition to the repository, X is providing tools that will let users see for themselves if and how X's ranking systems have impacted their account or posts. A new transparency tool is rolling out to an "Under the Hood" page in the app's settings, which will let users who have posted 10 or more times over the past month download their aggregate stats as a JSON file. The file will show if any labels have been applied to their account or posts over the past calendar month.
X's VP of Product told TechCrunch that X engineers will consider pull requests. "That would be amazing to have people submitting code that improves the algorithm...I mean, how cool would it be for the X algorithm to be not just visible to the public, but also, like, by the public?"
"You'll get the core ranking code that pulls posts and ranks them for any given user and assembles the feed," X's VP of Product Keith Coleman told TechCrunch in an interview ahead of the announcement. "You can see the systems that filter out potentially problematic, rule-violating content...And some of those systems, like the ranker and the score, you can even run yourself outside the company."
"This is the kind of thing that I think people will be fairly shocked that we are releasing," he added.
In addition to the repository, X is providing tools that will let users see for themselves if and how X's ranking systems have impacted their account or posts. A new transparency tool is rolling out to an "Under the Hood" page in the app's settings, which will let users who have posted 10 or more times over the past month download their aggregate stats as a JSON file. The file will show if any labels have been applied to their account or posts over the past calendar month.
X's VP of Product told TechCrunch that X engineers will consider pull requests. "That would be amazing to have people submitting code that improves the algorithm...I mean, how cool would it be for the X algorithm to be not just visible to the public, but also, like, by the public?"
What's the point if you can't actually change it? (Score:1, Insightful)
The one thing that defines utility of a social media platform is the userbase. Literally nothing else matters if you don't have the users. That means that forking it is, unlike most open source software, a complete non-starter. You can't just decide you don't like the algo and that you'll modify it and use a different one; if Twitter doesn't expose the actual users and their content to whatever selection you wish to make from it, showing the alleged code to the selection algo is worthless.
It's some form
Re:What's the point if you can't actually change i (Score:5, Insightful)
They said they would consider pull requests.
And if they didn't, I think a greater level of transparency is worthwhile in its own right.
Re:What's the point if you can't actually change i (Score:5, Interesting)
Can we be sure that the open source code is what they are running on their website though? Because it feels like e.g. Elon Musk's posted get boosted more than others, even if his own Grok AI usually debunks them.
Re: (Score:3)
Because it feels like e.g. Elon Musk's posted get boosted more than others
There's no "feels" about it. Not only has there been enough leaked evidence to show that Elon Musk specifically requested his engineers find a way to boost his specific engagement, public analysis of his posts also clearly show a dramatic step change around the same time as those leaked reports came out: https://www.qut.edu.au/news/re... [qut.edu.au]
Re: (Score:2)
Yeah, I'm extremely dubious that the open source version will "sync" with the operational version. I expect the general codebase to be truly represented, but not the "levers" that can be pushed on an operational system.
Re: (Score:2)
In short, that analysis has the causal arrows backwards. That was plainly obvious
Re: (Score:2)
Re: (Score:1, Informative)
So yeah it would be funny to actually see the algorithm with all its warts and all.
But that said the algorithm without the data isn't a complete picture. Especially because we know that the owner of Twitter routinely has the data altered artificially in order to fit with whatever silly little agenda he has this week.
Re: (Score:3, Interesting)
They are open-sourcing the platform. It is good news. Even if it cannot be changed, the insight is worth it.
Take, for example, Slashot algorithm. It had some success but has tendency to create echo chambers in its own right. For example, your post has little substance on the topic except some paranoia, self-promotion and your political views, but yet Slashdot moderates it up.
Perhaps someone will create something better by analyzing both
Re: (Score:2)
You know that you are the "Slashdot algorithm," right?
Re: (Score:1)
You know that you are the "Slashdot algorithm," right?
The problem is that second “you”. In comparison to previous decades, one-sided brigading politicized downvotes now often dominate.
I suspect this is why YouTube and X have largely flipped to an upvote-only system.
As a basic solution, I’d prefer changing to an upvote-only system or making downvotes more expensive for raters to use.
Or simply supply a personal option to let each account choose a the value of a downvote - for example, a user could choose to have the scores they see calculated i
Re: (Score:2)
Peer review sucks.
Negative mods are an emotional hit to the uninitiated, but anyone who's been around the Internet for more than a couple of days knows that it just means someone who doesn't have a reply was annoyed enough to waste mods from their sock puppet army.
I tell grad students that if they're not pissing off someone they're probably not saying anything of consequence.
Re: (Score:1)
Ha! Reminds me of a pedagogy study - it turned out that the most “hated” math teachers were also objectively the most effective.
Re: (Score:2)
There are a bunch of studies that show you learn most effectively when you're pushed right to, but not beyond, your ability. It should be a frustrating struggle, but ultimately achievable. Lots of people don't enjoy anything with "struggle" in it, regardless of the payoff.
Re: (Score:2)
>"I'd not trust them to actually give you the complete real deal and there's no way whatsoever to validate that the code they release is really it."
Oh, you mean kinda like Chrome?
You know, that popular browser by Google.... the one that is binary and you don't really know what they are doing. They release the code as Chromium. And don't really let the community steer it in any meaningful way, they have 100% control over the code. Then they take it and do mysterious stuff on/in it and produce a binary
Re: (Score:2)
I mean, on any decent social media platform you should be able to? Bluesky has had this feature for years. Not just algo choice, but the ability to outright create your own algos and share them with other people.
It's IMHO a red flag when a for-profit company wants to force everyone to use the same algo.
Algorithm (Score:2)
Like (Score:5, Funny)
Re: Like (Score:5, Funny)
Cry for help? (Score:3)
Re: Cry for help? (Score:2)
Re: (Score:3)
What is the reason for this?
A thin veiled attempt at transparency by a company that fired its entire transparency department.
Bamned by AI for no reason (Score:2)
FIX YOUR SYSTEMS, MUSK.
Re: (Score:2)
USAF Vet here.
I am not an enemy of the US, far from it.
I just had my account there permanently, and unfairly, suspended by some algorithm for "inauthentic behavior. " Attempts to appeal are instantly denied.
In the 15 years that I have had that account, I have never once done anything that they classify as "inauthentic behavior. "
Re: (Score:2)
Maybe the only behavior that is considered authentic by X comes from AI?
Elon doesn't want dissenting voices. Did you disagree with (or fail to support) the preferred Oligarch-first narrative?
X11 (Score:2)
I read the headline and thought it was something about screen information and video filtering and thought, why? Since X11 has always been open source.
But I see that it's about twitter and that's something I've never used and have never made any effort to understand.
I guess I'm an old coot...
Re: (Score:2)
More important is what it does. (Score:5, Informative)
X algorithm changed in July [2024] to boost right-wing accounts, study finds [cybernews.com]
but there is more to it [stateofsurveillance.org]
Research reveals systematic changes to X's algorithm under Elon Musk. After his Trump endorsement on July 13, 2024, Musk's own post views increased 138% and retweets jumped 238%. PLOS One found hate speech is 50% higher than pre-acquisition, spanning racism, homophobia, and transphobia. In January 2025, Musk's Islamophobic posts accumulated 1.19 billion views in 30 days. Academic audits show the algorithm amplifies emotionally hostile, antidemocratic content, and that right-leaning accounts had higher baseline visibility before Musk even endorsed Trump. Cambridge research confirms rude, disrespectful, and false content sees greater amplification. The platform isn't just allowing extremism. It's designed to promote it.
Shitty twitter needs to be flushed.
Re:More important is what it does. (Score:5, Interesting)
leon/elmo bad
orange man also bad
How did it become controversial to think Nazis and child rapists were bad? When this is the state of political discourse and debate in the USA, we deserve for our nation to fail, as it is in the process of doing.
Re: (Score:1)
Nazis... you keep using that word. I do not think it means what you think it means.
Re: More important is what it does. (Score:1)
Ok Nazi.
Re: (Score:2)
Yes. Oh sorry was that you just stating facts based on the acts of the people, or were you trying to stupidly mock this position while presenting no counter argument of your own?
Re: (Score:2)
Cambridge research confirms rude, disrespectful, and false content sees greater amplification.
Give the people what they want. And people want rage bait. This isn't a twitter problem by the way, not only do all platforms amplify rage bait, so does the media, with entire publications predicated on the principle of enraging the readership to drive engagement.
People seem to crave being angry.
Re: (Score:3)
>"X algorithm changed in July [2024] to boost right-wing accounts, study finds [cybernews.com]"
How quickly we forget the down-ranking and bans on right-wing accounts. Or did you agree with that?
https://nypost.com/2022/12/08/... [nypost.com]
>"Cambridge research confirms rude, disrespectful, and false content sees greater amplification."
I don't participate in social media. But what I do see is just as much rude and disrespect from the "left" as the "right". And "false content" is often based on the eye of the beh
Re: (Score:2)
If they were lies, then I agree with down-ranking them.
If they were just opinions, then I only agree with avoiding them.
OTOH, I don't use X/Twitter. Slashdot is as close as I come to social media.
Re: (Score:2)
And it's bad enough as it is.
What gets modded insightful or Troll around here is also very much a matter of trends.
I've been here long enough to be on both sides twice. What went for humor once becomes a troll or iinsightful and back.
It is what it is when people are involved.
Re: (Score:2)
How quickly we forget the down-ranking and bans on right-wing accounts.
LOLOLOLOOLLLOLLLLL
Or did you agree with that?
Yes, those specific accounts were banned for actual reasons, of course sensible people agreed with that. Meanwhile multiple studies showed that during that same period, right wing speech was amplified more than left wing speech [theguardian.com]. You fucking pedo loving maggots always cry persecution while it's provably not happening to you.
Re: (Score:2)
The Paper was funded by Australian Research Council (ARC)
March 2023: Xitter restrict free academic API access. Introduces new paid API pricing tiers.
June 23, 2023: The dedicated Academic Research API access was fully removed from remaining accounts.
November 2024: Study published: A computational analysis of potential algorithmic bias on platform X during the 2024 US election.
Re: (Score:2)
And those links are silly. Their evidence for a change is that Musk said something that made a huge splash and there was a huge jump in the number of accounts paying attention to the things he was saying. That didn't require a change in the algorithm, it simply required him to start posting things that made a big political splash.
The rest can be dismissed with, "the Left calls anything they disagree with 'hate speech', and X stoppe
State ordered censorship (Score:1)
Part of the explicit points of this, is that anything they've been forced to do becomes visible as well. Hopefully civil rights groups with the FSF will be all over this.
Re: (Score:2)
This requires that you believe they're being honest, with no backing evidence. I can see possible evidence of consistency, but that's a much weaker claim.
Does this include the ad algorithm? (Score:4, Interesting)
Does all this open source stuff include enough information to know how X decides which ads to show you?
The next weapon (Score:5, Insightful)
What in the upcoming US elections, gives Musk so much power that he will abandon (or re-sell) one of the greatest tools for destruction, ever built? I speak of Twitter/X like it is a weapon. It is but one that destroyed trust, empathy and identity, not bridges and cannons and neighbours.
What will be the next weapon that Musk aims at the USA: Will it too be mistaken for ordinary or useful to everyday living?
Re: (Score:3)
I suspect most of the amplification of right wing noise has something to do with the left wingishness of the complainers.
My stance comes from seeing it from the other side. I'm comfortably right of center and my previous experience with Twitter (x is a stupid name) is that it absolutely was a left wing echo chamber. It indeed was rather calmer because of the aggressive pruning of any conservative voices.*
I believe the algo serves you aggressively oppositional views to spur contrary posting and raise engag
Re: (Score:2)
>"I speak of Twitter/X like it is a weapon. It is but one that destroyed trust, empathy and identity"
Twitter was a weapon, destroying those long before Musk bought it. A lot of the stuff going on just wasn't noticed (at least by the main-stream).
None of these companies have any great morality. Their motivation is primarily "engagement" and nothing engages more than stirring up conflict.
X sucks worse than old Twitter (Score:2)
At least old twitter basically worked.
I just had my account there permanently, and unfairly, suspended by some algorithm for "inauthentic behavior. " Attempts to appeal are instantly denied.
In the 15 years that I have had that account, I have never once done anything that they classify as "inauthentic behavior. "
I have found that I am not alone. Thousands of users are making similar complaints.
Re: (Score:2)
For all his talk, Musk has dropped this ball.
YOUR ROBOTS ARE LOCKING OUT YOUR USERS !