All those who doubted me when I said that Bingʼs index was in the 1 to 2 milliards …
This is where Inktomi was over two decades ago, and itʼs a fraction of the size of Mojeekʼs.
#AI#GenerativeAI#Search#SearchEngines: "There seem to be clear indications of a novelty factor at work. And while novelty in and of itself is not a bad thing, if it isn’t followed with a consistent behavior change, we can’t really call it a trend.
Take the above Bing.com numbers for instance. If we credit the inclusion of AI search tools on the platform as the cause of the unique user bump, it would seemingly serve to solidify the predicted 25% drop. Yet when we consulted our panel data further, we found that only between 4% and 9% of users used Bing Chat (their AI agent) in any given month during 2023. What’s more, of those that did use it, only two to four searches were conducted over the ensuing month.
#AI#GenerativeAI#SEO#Reddit#Google#Search#SearchEngines: "For years, people who have found Google search frustrating have been adding “Reddit” to the end of their search queries. This practice is so common that Google even acknowledged the phenomenon in a post announcing that it will be scraping Reddit posts to train its AI. And so, naturally, there are now services that will poison Reddit threads with AI-generated posts designed to promote products.
A service called ReplyGuy advertises itself as “the AI that plugs your product on Reddit” and which automatically “mentions your product in conversations naturally.” Examples on the site show two different Redditors being controlled by AI posting plugs for a text-to-voice product called “AnySpeech” and a bot writing a long comment about a debt consolidation program called Debt Freedom Now." https://www.404media.co/ai-is-poisoning-reddit-to-promote-products-and-game-google-with-parasite-seo/
#AI#Search#SearchEngines#Google: "Google is considering charging for new “premium” features powered by generative artificial intelligence, in what would be the biggest ever shake-up of its search business.
The proposed revamp to its cash cow search engine would mark the first time the company has put any of its core product behind a paywall, and shows it is still grappling with a technology that threatens its advertising business, almost a year and a half after the debut of ChatGPT.
Google is looking at options including adding certain AI-powered search features to its premium subscription services, which already offer access to its new Gemini AI assistant in Gmail and Docs, according to three people with knowledge of its plans.
#AI#GenerativeAI#Search#SearchEngines#LLMs#Hallucinations: "A couple of days ago, Wharton professor Ethan Mollick, who studies the effects of AI and often writes about his own uses of it, summarized (on X) something that has become clear over the past year: “To most users, it isn't clear that LLMs don't work like search engines. This can lead to real issues when using them for vital, changing information. Frontier models make less mistakes, but they still make them. Companies need to do more to address users being misled by LLMs.”
If you're running a business, you can either invest at being good at your business, or good at Google SEO. Choose the former and your customers will love you – but they won't be able to find you, thanks to the people who choose the latter. And if you're going to invest in top-notch SEO, why bother investing in quality at all? https://pluralistic.net/2024/03/05/the-map-is-not-the-territory/
"That's why the only serious competitor to Google is Bing, another Big Tech company (Bing is also the primary source of results on Duckduckgo, which is why DDG sometimes makes exceptions for Microsoft's privacy-invading tracking)."
In an age of LLMs, is it time to reconsider human-edited web directories?
Back in the early-to-mid '90s, one of the main ways of finding anything on the web was to browse through a web directory.
These directories generally had a list of categories on their front page. News/Sport/Entertainment/Arts/Technology/Fashion/etc.
Each of those categories had subcategories, and sub-subcategories that you clicked through until you got to a list of websites. These lists were maintained by actual humans.
Typically, these directories also had a limited web search that would crawl through the pages of websites listed in the directory.
Lycos, Excite, and of course Yahoo all offered web directories of this sort.
(EDIT: I initially also mentioned AltaVista. It did offer a web directory by the late '90s, but this was something it tacked on much later.)
By the late '90s, the standard narrative goes, the web got too big to index websites manually.
Google promised the world its algorithms would weed out the spam automatically.
And for a time, it worked.
But then SEO and SEM became a multi-billion-dollar industry. The spambots proliferated. Google itself began promoting its own content and advertisers above search results.
And now with LLMs, the industrial-scale spamming of the web is likely to grow exponentially.
My question is, if a lot of the web is turning to crap, do we even want to search the entire web anymore?
Do we really want to search every single website on the web?
Or just those that aren't filled with LLM-generated SEO spam?
Or just those that don't feature 200 tracking scripts, and passive-aggressive privacy warnings, and paywalls, and popovers, and newsletters, and increasingly obnoxious banner ads, and dark patterns to prevent you cancelling your "free trial" subscription?
At some point, does it become more desirable to go back to search engines that only crawl pages on human-curated lists of trustworthy, quality websites?
And is it time to begin considering what a modern version of those early web directories might look like?
#Media#News#Journalism#SEO#AdTech#Search#SearchEngines: "[F]ew network effects have damaged the news more than Search Engine Optimization, where the allure of traffic from search engines like Google has led publishers to create content not with the goal of serving their audience, but attracting the spurious traffic that one might get from those searching "when does the Super Bowl start."
The result is a media industry in crisis. Desperate executives and disconnected editors twist their reporters' coverage to please Google's algorithms as a means of improving traffic to please advertisers' algorithms, creating content that looks and sounds the same as other outlets, which in turn leads to layoffs as profits fail to increase, which in turn normalizes and weakens the content created by the outlet. This is largely a result of those in power not actually consuming or producing any of the product that makes the outlet money, only understanding the business as a series of symbols that at some point create revenue, ostensibly from the written word and video.
When you make decisions for a website or company that produces words that it sells for money based not on the writing, but on how to twist that writing to make it "more profitable," the conclusion is always inevitable — the creation of identical-looking slop that people only read by accident, and the slow asphyxiation of journalism and culture.
It almost always leads to overstaffing and mismanagement, too. Any form of creative media requires an understanding that building an audience takes time and money, and that one cannot just spend a bunch of money to make that happen. But these craven idiots are as rotten as the rest of the economy (...) The media is being run by people that do not see value in people or the things that they create, but the metrics that come as a result."
#AI#GenerativeAI#Web#Search#SearchEngines#Chatbots: "The Browser Company’s new app lets you ask semantic questions to a chatbot, which then summarizes live internet results in a simulation of a conversation. Which is great, in theory, as long as you don’t have any concerns about whether what it’s saying is accurate, don’t care where that information is coming from or who wrote it, and don’t think through the long-term feasibility of a product like this even a little bit. Or, as Dash put it, “It’s the parasite that kills the host.”
The base logic of something like Arc’s AI search doesn’t even really make sense. As Engadget recently asked in their excellent teardown of Arc’s AI search pivot, “Who makes money when AI reads the internet for us?” But let’s take a step even further here. Why even bother making new websites if no one’s going to see them? At least with the Web3 hype cycle, there were vague platitudes about ownership and financial freedom for content creators. To even entertain the idea of building AI-powered search engines means, in some sense, that you are comfortable with eventually being the reason those creators no longer exist. It is an undeniably apocalyptic project, but not just for the web as we know it, but also your own product."
#Media#News#Journalism#SEO#Google#Search#SearchEngines: "In our experience, each rollout of the Products Review Update has shaken things up, generally benefitting sites and writers who actually dedicated time, effort, and money to test products before they would recommend them to the world.
That said, most searches for specific product models don’t just magically start with users searching for specific devices off the top of their heads. There is an immediate step before this: the hours of research reading through lists of product recommendations.
If you have been reading HouseFresh for a while, your first encounter with us was likely a list like this one or this one recommending the best devices for a specific issue you were trying to solve. That is how most of our readers find us.
Unfortunately, we’re getting less and less traffic from those pages, and it’s endangering the future of our site.
#Search#SearchEngines#OpenSource#FLOSS: "Using Google has started to feel worse over the last few years, as results are seemingly taken over by SEO'd content, AI-generated results, and websites with tons of affiliate links and ads. As a response to this state of affairs, a single coder has launched a new, open-source search engine in part as a response to internet’s overwhelmingly corporatized and homogenous search ecosystem. The new search engine, called Stract, is running on a server in the basement of its developer’s office, is highly customizable and, based on feedback from users in the project’s Discord, is rapidly improving.
The project grew out of founder Mikkel Denker’s master’s thesis at the Technical University of Denmark, which was focused on helping people search their own files and documents, he told me in an online chat. He is set to finish that master's next week and will then pivot his attention to Stract fulltime.
“Most of our searches go through the same handful of entities (Google, Bing, Yandex),” Denker told me. “Even other search engines such as DuckDuckGo use Bing for their results. I found it very weird that there essentially is no way to browse the web in an open manner. So that's what I am trying to build.”"
"A Russia-based company has become the legal owner of tech giant Yandex as it prepares to separate from its Dutch parent company, the state-run Interfax news agency reported Tuesday."
Remember web directories? Back in the mists of Internet time, it was clever to know that they existed and know which ones were relevant to the subjects you were interested in. That was when the web was still small enough to imagine we could catalog and curate content.
Indeed: web directories! Given the collapse of effective search through Google &co, we will have to return to those directories. Human-controlled, intelligent lists of access points to real, validated information.