If your site is under 500 URLs you don’t need to pay for a crawler. Screaming Frog’s free version crawls up to 500 URLs and covers most of what a small site needs:
– Broken links (4xx) and redirect chains
– Missing / duplicate titles and meta descriptions
– Pages with noindex you forgot about
– Image alt text gaps
I run it monthly on client sites before touching anything else. Paid version only matters when you need JS rendering or more URLs.
Good call. Also worth knowing: you can schedule crawls and compare two crawls in the paid version, which is great after a migration. For the free version, I export to CSV and diff manually.
@danny_ranks yes, user agent config works in the free version as far as I remember. It’s under Configuration > User-Agent. You can set it to Googlebot smartphone.
.
