First, there was the thunder, then the wonder, followed immediately by the blunder. The tongue-in-cheek and hyperbole aside, this encapsulates the story of generative AI since the launch of ChatGPT, Gemini et al. The initial hype was like a clap of thunder, causing the masses to freeze in awe and wonder, while AI companies were scraping the internet, including copyrighted content, to train their large language models (LLMs).
In several court cases, most notably New York Times v. OpenAI, the AI companies have argued that the “fair use” provision of copyright law doesn’t obligate them to compensate content publishers. AI companies are using this defense as the basis for accessing content without permission, attribution or remuneration.
Not only are AI companies defending themselves in court and engaging in general disputes with content publishers, but financial market analysts are starting to warn that the AI expansion could trigger a bubble similar to the dot.com bust in October 2002.
Creating a fair and equitable playing field for content publishers and AI companies is a problem in search of a better solution than litigation. In 2024, OpenAI and several large media companies, including Time, News Corp. and The Associated Press, signed individual licensing agreements, allowing access to their content and archives. A licensing agreement is a solution, but a messy one, and fewer of these agreements are occurring in 2025.
Publishers’ ad revenues at risk
Bubbling below the surface is a more significant problem for content publishers: the loss of advertising revenue and, by inference, visibility and audience engagement. If, as has been predicted, AI prompts replace internet searches, people will simply be given answers in whatever AI app they’re using. Why use a web browser? With no pages of links based on their searches, users won’t click through to content publishers’ sites and view the ads that appear there.
This nightmarish scenario became apparent to Toshit Panigrahi when ChatGPT-4 was released in the spring of 2023, while he was an employee at Toast, the restaurant management software company. He envisioned a future when AI could book restaurant tables and order patrons’ food, resulting in no one viewing Toast’s ads. Panigrahi watched the drama of The New York Times suit and statements from the News/Media Alliance and realized the news/media industry was facing the same problem as Toast.
“The world was, until recently, very focused on copyright, LLM training and fair use. What we realized running the ad business at Toast is that it's a monetization problem. If AI agents autonomously took these actions on behalf of consumers, then advertising could be disrupted. The entire unit economics of the web would change. You couldn’t reliably monetize through ads anymore. This is a problem coming for everyone on the internet, but publishers are the first to feel it due to AI search, and we must start to solve it there,” Panigrahi said.
The future he saw prompted him to co-found TollBit, as CEO, with Olivia Joslin, COO. It is one of several companies offering what may be a better solution: an AI marketplace.
AI marketplaces: New revenues for publishers, content access for AI companies
According to Panigrahi, TollBit helps publishers monitor, manage and monetize. It shares data with publishers that shows them being scraped tens of thousands of times a day. He says it is a revelation for publishers, shifting the conversation from a copyright or licensing issue.
“We show publishers how this world is evolving. They realize a new visitor is emerging on the internet that will far exceed human visitors, and there is no way to have a value exchange with this new visitor. Publishers must change their mindset and think not just about humans, but about those non-human autonomous readers trying to access content,” Panigrahi said.
ProRata and Cloudflare are two other companies in this broadly defined AI marketplace ecosystem. According to Annelies Jansen, chief business officer at ProRata, TollBit and Cloudflare are considered potential partners, sharing the common opportunity to find a solution. Like Panigraphi, ProRata identified the lack of an economic model to compensate content creators for the use of their content in generative AI.
Jansen added that ProRata doesn’t position itself as an AI marketplace, because a marketplace needs a buyer and a seller. It has created a product aimed at end users called Gist.ai. It is an AI search tool that only provides answers from content publishers licensed with ProRata.
“Attribution is the foundation of the company, and the only way to build attribution is if you have access to licensed content. That was phase one, and from it we’ve developed a set of guidelines for the industry as phase two. The next phase is based on requirements to test the high-quality content from our partners. Several tools in the market can identify whether content was written by a human or by AI. We only want to ingest content written by humans,” Jansen said.
By comparison, Cloudflare is a well-established internet infrastructure company, founded in 2009 and powering approximately 20% of the web. According to Matthew Prince, co-founder, CEO and chair of the board of directors, Cloudflare started as a cybersecurity company battling hackers from North Korea, Russia, Iran or China.
“Several startups tried to create AI marketplaces, but the first challenge was that markets require scarcity. For many years, publishers treated robots and humans the same. The second challenge is that they lacked the necessary resources to block the bots. It won’t work unless you block the bots first, so publishers came to us because we’re good at stopping bots,” Prince said.
At first, Prince wasn’t totally convinced that generative AI would be a threat to the existing business model of the web. Once he was educated about the danger, however, Cloudflare’s first step in September 2024 was to create scarcity in the market by training publishers and giving them the tools to block bots. Then, on July 1, 2025, blocking AI blots became a default capability for all new Cloudflare customers. Having the tools and conditions to create the necessary scarcity, Cloudflare is now empowering publishers with the capability to require payment for content.
Like many AI companies searching for the best solution to compensate content publishers fairly, Perplexity took a different tack, launching Comet Plus, a new subscription service, in the summer of 2025, and pairing it with the company’s Comet browser. The concept is a compensation model that generates revenue for content publishers while providing users with access to premium content from publishers and journalists.
“We built Comet Plus into the Comet browser because we believe publishers need to stay at the center of the information ecosystem while earning better revenue than they would anywhere else. Traditional revenue sharing or licensing deals aren’t enough to support journalism, and they don’t reflect the reality of consumer demands in the AI age,” Jessica Chan, head of content and publishing partners at Perplexity.
How AI marketplaces work
Each of these companies solves the problem differently. Panigrahi said TollBit has a very flexible approach. It has some clients who choose to sue AI companies and receive payment for the content they used to train their algorithms. Others prefer to strike a one-on-one deal, while small publishers need a solution to make their content attractive to AI companies.
TollBit helps publisher clients establish rates and rules for access to their content. The company’s clients also include AI companies that pay a fee to access content. TollBit services are free to publishers. They also receive analytics, a bot paywall and monetization infrastructure at no charge. Initially, the company has focused on text-based content and is developing the capabilities to support audio and video. Currently, more than 3,000 publishers are Tollbit clients.
ProRata introduced Gist.ai in December 2024, and after less than a year, it has more than 100 partners and more than 750 titles, granting access to their content to generate AI answers without any upfront payments. The Atlantic, Time, Fortune, ADWEEK, BuzzFeed and Lee Enterprises are some of the high-profile media companies partnering with ProRata. The ProRata model, as implied by its name, provides its partners with a pro-rata payment (50/50 share).
Cloudflare positions its network in front of publishers’ content. All visitors, whether human or bot, must pass through the network to access content. Publishers can use a one-click tool to determine which AI crawlers to block — generally, blocking all AI crawlers or allowing all of them access. Cloudflare has various plans, primarily for its clients who require cybersecurity protection; however, all of its plans, including the free plan, include the capability to stop AI crawlers en masse.
“Having every AI company pay the same crawler rate is probably the wrong method. It’s also too complicated, as it’s difficult for publishers to set that rate. Publishers are experimenting with several options, one of which is X number of slots for AI companies to access content. We plan to run an experiment with an auction of those slots,” Prince said.
According to Chan, Perplexity’s Comet Plus compensates content publishers from three different search categories: humans searching manually, crawler bots and user agents. To address publishers’ concerns about losing ad revenue from their sites, Perplexity offers a summary of citations and links back to publishers’ sites for users. Users can access Comet Plus with a separate $5 subscription; it is included in Pro and Max memberships. Perplexity allocates almost all revenues to publishers; the remainder covers Perplexity’s costs.
“Premium content is like paywalled content. For example, if you are a publication with a paywall or subscription-based access, then you make that premium or subscription paywalled content available through Comet Plus, and then that’s where we would share in revenue,” Chan said.
An advantage for small- to medium-sized publishers
Logic dictates that the largest news and content publishers will gain the most revenue from participating in an AI marketplace. They may be by volume, but not necessarily revenue generated per piece of content — small- and medium-sized publishers may have an advantage, according to Panigrahi.
“The most surprising thing that we’ve learned building TollBit is that it will actually work in their favor. For a given query, several factors influence the decision to pay for content. One will be newness and another some marker of quality, but the biggest one is uniqueness. If you have a piece of content about a topic that no one else has written about, then you can command a much higher premium for it. This is why we’re excited about local news,” Panigrahi said.
Prince agrees. Major news sources — The New York Times, The Wall Street Journal, The Washington Post and The Boston Globe — often cover the same stories, and may be worth less to AI companies than more unique content or story topics from local news outlets, for example.
“My hunch is that going forward, those publishers that create more differentiated, unique and local content that you can’t find anywhere else will receive a higher price in the marketplace than those publishers doing high quality journalism, but of the same stories as other publications,” Prince said.
Jansen said ProRata is also focused on local news. Integrating its answer engine with a local news site/channel allows readers to access important local content when they demand it. They don’t have to search Google or use an AI engine; they can stay within the news site’s platform.
“Our AI companion widget can be integrated with a publisher’s site in a maximum of two hours. It’s four lines of code. We provide it free of charge to the industry, and we only ask for a feed of your content; then, you are AI-compliant. Your local publication will have an AI tool,” Jansen said.
The search for a fair compensation model for publishers in the AI ecosystem is a dynamic and fluid situation. Undoubtedly, all these companies will tweak their current models or scrap them entirely for something new as AI technology matures. Now is the time for all content publishers to seek their best solution.
Bob Sillick has held many senior positions and served a myriad of clients during his 47 years in marketing and advertising. He has been a freelance/contract content researcher, writer, editor and manager since 2010. He can be reached at bobsillick@gmail.com.
Comments
No comments on this item Please log in to comment by clicking here