The old internet bargain: crawlers got content, sites got visitors

For a long time, the web ran on a simple trade. If you owned a website, you let search engines crawl it because they sent you traffic in return. Google would read your site, index it, someone would search, your page would show up, and a human would click through. Once that human landed on your site, you monetized their attention — ads, email capture, subscriptions, affiliate clicks. The crawler got the content, the website got the visitor, the visitor got the answer, and that model funded a massive part of the internet.