wish to be discovered. Onion websites are “crawled” and added to the record delivered their “robots.txt” file permits it, and when It's not on their blacklist of web sites with abuse content. Website operators could also submit their unique .onion web pages for indexing.