About AI for content and products
If you want to block your page from search engine results, use another process such as password security or noindex. If your web page is blocked which has a robots.txt file, its URL can continue to appear in search engine results, though the research consequence will not likely have a description. Image files, video files, PDFs, and other non-HTML