Tag: Site Crawling

Spider for Rust: Build a Site and Resource Inventory

A site inventory needs more than addresses: response status, saved material, and the resources each page declares all help later migration and diagnosis. This t