![Google Google](http://a.fsdn.com/sd/topics/google_64.png)
Google Now Defaults To Not Indexing Your Content 66
An anonymous reader a report:Google is no longer trying to index the entire web. In fact, it's become extremely selective, refusing to index most content. This isn't about content creators failing to meet some arbitrary standard of quality. Rather, it's a fundamental change in how Google approaches its role as a search engine.
From my experience, Google now seems to operate on a "default to not index" basis. It only includes content in its index when it perceives a genuine need. This decision appears to be based on various factors:
Extreme content uniqueness: It's not enough to write about something that isn't extensively covered. Google seems to require content to be genuinely novel or fill a significant gap in its index.
Perceived authority: Sites that Google considers highly authoritative in their niche may have more content indexed, but even then, it's not guaranteed.
Brand recognition: Well-known brands often see most of their content indexed, while small or unknown bloggers face much stricter selectivity.
Temporary indexing and de-indexing: In practice, Google often indexes new content quite quickly, likely to avoid missing out on breaking news or important updates. Soon after, Google may de-index the content, and it remains de-indexed thereafter. So getting initially indexed isn't necessarily a sign that Google considers your content valuable.
From my experience, Google now seems to operate on a "default to not index" basis. It only includes content in its index when it perceives a genuine need. This decision appears to be based on various factors:
Extreme content uniqueness: It's not enough to write about something that isn't extensively covered. Google seems to require content to be genuinely novel or fill a significant gap in its index.
Perceived authority: Sites that Google considers highly authoritative in their niche may have more content indexed, but even then, it's not guaranteed.
Brand recognition: Well-known brands often see most of their content indexed, while small or unknown bloggers face much stricter selectivity.
Temporary indexing and de-indexing: In practice, Google often indexes new content quite quickly, likely to avoid missing out on breaking news or important updates. Soon after, Google may de-index the content, and it remains de-indexed thereafter. So getting initially indexed isn't necessarily a sign that Google considers your content valuable.
Coming soon (Score:5, Insightful)
- Google announces a program where site owners can pay a fee that indicates their website should be considered highly authoritative and unique.
the search engine that Gotham deserves (Score:5, Insightful)
Re: (Score:3)
I suspect in this case, you either die a best-in-kind search engine or live long enough to see a combination of SEO gaming and generative AI render your product obsolete.
Let me know when someone figures out how to do seamless product placement within LLM-generated answers and I'll show you the next Google.
Re: (Score:2)
Re: (Score:3, Funny)
Get bought by Google.
Re: (Score:2)
Could we resurrect Altavista? It was awesome to actually find what you're looking for. I've been playing around on the internet for nearly 30 years now. Not once have I ever used Yahoo to look for anything. On occasion I would play billiards with someone or use the chat program but never the search.
Re: (Score:3)
That's not how I recall it. Altavista seemed to have a very rudimentary algorithm based on literal text matching, and crafting queries to get what you wanted was often tough. They had also started piling on annoying features to monetize their site.
I still remember when Google showed up: Back then it returned far superior results quickly and without clutter (I'm not claiming that this is still the case, BTW). I and most everyone I knew switched to exclusively using Google almost instantly.
Re:the search engine that Gotham deserves (Score:4, Insightful)
Google results did suck in 2010. It was the end of the '90s to the mid '00s when Google was "good".
Re: (Score:3)
Re-launching AltaVista on the modern web would be like sending the Zero up against an F35.
Re: (Score:2)
Re: (Score:2)
Re: the search engine that Gotham deserves (Score:2)
The best modern analogs for Altavista are general-type dark web search engines. They are not particularly effective or useful. A funny alternative to Altavista was the warez search site Astalavista. A pretty sure way to get infected too.
Google was next level and changed the game completely. The early years of Google with dead simple usability and magic-like functionality would be nice, so somewhere in the late 90s to 00s.
Re: (Score:2)
- Google announces a program where site owners can pay a fee that indicates their website should be considered highly authoritative and unique.
That simply dovetails with any business practice that they MAY HAVE for site owners to pay Google to improve their page ranking or even pop to the top.
Re: (Score:2)
- Google announces a program where site owners can pay a fee that indicates their website should be considered highly authoritative and unique.
The absolute Truth of your statement hit me like a ton of bricks. In my mild disorientation, while "coming back to Reality", my mind was open enough to see a parallel:
Ever watch a child grow up from infancy to death? When they were a baby, they uncritically took EVERYTHING in as Truth. Then, they run across a lie and become more circumspect in what they ingest. Eventually, they become bitter and the only things you can get them to ingest are things that directly benefit them. Eventually, they become resista
Their only option (Score:3)
Re:Their only option (Score:5, Insightful)
You must be asking pretty dumb questions if "chatgpt" can answer them.
Re: (Score:1, Insightful)
I don't think it is either, I think they're trying to increase engagement on the actual search engine itself, which comes at the expense of relevance.
Re: (Score:2)
I also find ChatGPT more useful for finding answers to things - because I get an answer directly back with usually fairly relevant information, instead of a page of summaries to websites that have been padded out massively with someone's life story about the subject in an attempt to get ranked.
For example, it used to be simple to get say, a generic muffin recipe, using Google. Now the results that come back are full of fluff, probably a link to a video on youtube and if you are lucky, the actual ingredients
Makes sense (Score:5, Interesting)
Google is not your memory. It is a service to find information that you are, or may be interested in.
If this change means that Google starts to rid its index of content farms and other not particularly useful sites, then it can only make Google better.
Also a shot across the bow for anyone who wants to demand money from Google for indexing their website.
Re:Makes sense (Score:4, Insightful)
Problem here is that Google has become "the web" for most people. It's a de facto monopoly in web search, which makes it a de facto gatekeeper of the web.
That means that as it moves to be more selective, it effectively renders freedom of expression that enabled internet as a revolution of being able to find so much niche information into a cable TV-style highly selected, highly curated and highly constrained source of knowledge.
The question then becomes, are people going to switch? Or are LLMs replacing web search entirely anyway?
Re: (Score:3)
... It's a de facto monopoly in web search, which makes it a de facto gatekeeper of the web.
No! Bing is every bit as capable as Google!!! /s
Re:Makes sense (Score:5, Interesting)
Hilariously, my search kit includes three things.
Startpage to give me Google results. It's generally best for finding things overall.
DDG to give me Bing results. It tends to be better than Google in certain specialist things but worse overall.
Yandex to find things that are censored on the other two. It's worse than other two for everything else.
Problem is that most people are not this meticulous in searching for something, because they have better things to do than hunt a specific thing. A general answer is good enough in most cases for most people. This is why Google changing in the way it's being suggested is going to cause problems.
Re: (Score:1)
Re: (Score:2)
What I'm finding more frequently nowadays is that DuckDuckGo is returning msn.com results for "current event" queries. Since I do a lot of my browsing using a little known browser in Private Mode - and msn.com can't handle that combination - DDG is becoming less useable for me. Independently of that problem, DDG (says) it does not track its users' queries, but when the results point to msn.com that non-tracking starts to become pretty useless.
Re: (Score:3)
Re: (Score:2, Insightful)
Let me correct the actual concern:
10 generate malinformation in web page format
20 publish on web
30 goto 10
Remember, there are three things of concern on the internet according to DHS report that defined those. And the main concern isn't misinformation, but malinformation. That is defined as factually correct information that is against the mainstream narrative.
Let me give you an example of such malinformation, so you understand why it's by far the worst one.
"If we didn't count African American violent crime
Re: (Score:3)
Google is not your memory. It is a service to find information that you are, or may be interested in.
If this change means that Google starts to rid its index of content farms and other not particularly useful sites, then it can only make Google better.
Also a shot across the bow for anyone who wants to demand money from Google for indexing their website.
If it's only link farms and the likes I'm OK with this change as long as that's all it is and not completely destroying the ability to search on obscure subjects.The most valuable searches are generally the esoteric ones. There is a whole string of search engines that I stopped using because Google gave me good results on all manner of esoteric searches but the other search engines didn't. I don't really care if a search engine gives me good results on: "Taylor Swift", what I value is good results on esoter
Re:Makes sense (Score:4, Interesting)
Or the current bane, content farms that are AI generated.
Content farms are bad enough, but with AI, they are exponentially worse .
Re: (Score:2)
You're right but i think there's also a pushback. People don't like being shoved into the basement while there's a party upstairs. That's why I left twitter, for example. Now in 1 year threads is already half the size of twitter. Not that threads isn't shoving people into the outer darkness too, but they aren't as obvious about it, demanding money for the privlage of being heard.
Contents Page not Index (Score:3)
Re:Contents Page not Index (Score:4, Insightful)
Re: (Score:2)
how did you notice
I guess you didn't read this part "all it takes is a few competitive searches". So, in addition to being a coward, you're a moron, too.
Re: (Score:2, Redundant)
Who is the author? (Score:1)
This is just some rando dude's thoughts with nothing supporting it. It may be true, may not be true, who can tell?
Re: (Score:1)
This is just some guy's thoughts. There is no official statement, in the article, from Google. This is all happening on the internet.
Re: Who is the author? (Score:2)
10 years experience and entrepreneur, in building applications as a freelancer, no less.
Cute.
Re: (Score:2)
Re: (Score:1)
See what Google says for "rhetorical question".
Google continues to shoot own foot with Uzi (Score:2)
Re: (Score:2)
if websites want to be indexed (Score:2)
Noble goal, but... (Score:3)
I love the idea of eliminating AI crap and SEO
Unfortunately, this will hurt a lot of small creators like me who have a real small business
It's kinda like wikipedia that deletes useful stuff because they judge that it's "not notable"
Google has been getting increasingly useless. This change will eventually mean that the only search results that are presented are ads and content from the famous
No longer a search engine (Score:3)
Re: (Score:2)
While most people think of AOL as a walled garden (and understandably so), it had its roots in the Quantum Link (QLink) days of the Commodore era and in fact a lot of the AOL communication protocol was derived from that (FDO tokens and so forth). Once you got past the "Welcome" screen there was a lot you could do, and it enabled a lot of small-time software developers to host their shareware programs for others to try out. I remember getting a lot of early Macintosh games that way (TaskMaker, for example)
Google is an advertising company (Score:2)
Google (Score:1)
Google is a reddit search engine with ads
What I think this translates to: (Score:2)
Time to stop using Google (Score:3)
They have now completely gone off the rails.
google needs to take a few steps back (Score:1)
google needs to take a few steps back and reverse the changes they have done in the last few years to their search algorithm. The search result accuracy has been steadily declining. With results being either dated, or completely out of context from what the search was for.
Is this new? (Score:2)
Around a decade ago I used to play with things on my personal blog just to see how it would impact search rankings, and traffic from google. One thing I noticed as a very clear trend, was that if I had Google Analytics on my blog, and/or Google Ads, I'd get a lot better SEO. I never believed that was coincidence. Google has ALWAYS been a business which sells its users to advertisers.
Just the latest fundamental change in Google (Score:3)
Originally, Google was a search engine. It crawled the web and generated an index.
People who were using the web knew how to type, and a little bit of how computers work.
They typed in text to search for, and Google returned links to pages containing that text.
But over the following Septembers, the behavior of the users changed.
The people at Google realized that more and more people weren't using the search the way it was written. A lot of them were asking questions, trying to use Google to get information, instead of to look for web pages.
So the behavior of Google changed, both to give those people more useful results, and to give Google more opportunities to sell advertising.
That's why Google hasn't worked as a straight search engine for a long time. Oh, how I miss that search engine.
Since it's not a search engine anymore, indexing the whole web no longer makes sense for Google.
Re: (Score:2)
That's why Google hasn't worked as a straight search engine for a long time. Oh, how I miss that search engine.
Whenever this topic comes up, I see a lot of comments like this. If it's really possible, and desirable, to have a search engine in the style of Google circa 2000, why isn't anyone running one? Surely it would be wildly successful?
What I see out there is Bing, which does all the same things Google does, but not quite as well, and DDG, which is Bing plus a privacy shield, which makes it a bit worse than Bing -- but still very much trying to compete directly with Google as it is now, rather than offering
Article written by an SEO guy (Score:3)
Right there at the very top of the headline in the linked article, "SEO."
This guy is upset because his SEO tactics aren't working as well any more.
The only ones scummier than Google, are the SEO companies. I'm glad Google seems to be winning that battle.
No actual knowledge, just a guy's observations (Score:5, Insightful)
This guy is complaining because HIS sites aren't getting indexed as quickly as they used to. Maybe his content is just SEO BS, a type of website Google explicitly combats.
I personally run a genealogy website that has several hundred active users. I didn't have to do a thing to get Google to index it, they just picked it up on their own. But then, this site has actual content that people want to find, not made-up content that an SEO company wants you to find, instead of what you're actually looking for.
Re: (Score:2)
I kind of agree with you, but my own personal experience matches his, unfortunately. I revamped my personal website at the end of November 2022, and since then, none of my pages have been reindexed - and some of the pages of the old website have been deindexed, even with the 301 redirects I explicitly put there for helping people accessing the website.
At some point I thought that was because I didn't put the Google Analytics spy in my pages and did not register to the Webmaster Tools, for ethical reasons. I
Sucks to be a webcrawler (Score:3)
Most websites are behind a cloud hosted "I'm not a robot" captcha, and all the content is javascript generated.
You have to jump through their hoops now. (Score:1)