Website crawler software: Find and remove unused files on your website, license key search programm, Web Extract with Web Spider / Web Crawler and more.
Create an accurate XML sitemap in seconds! Just enter your URL and go. Inspyder Sitemap Creator will crawl your site and generate an XML sitemap that's ready for submission to all the major search engines. After creating your sitemap, it can upload it to your website and notify (or ping) the search engines. Inspyder Sitemap Creator can be scheduled to run automatically so that your sitemap is always accurate, even if your site changes frequently.
Search Engine Studio automatically indexes your web site (using 4 methods e.g. HTTP crawler) and then creates a professional, ultra fast search engine for your website or an offline search for CD-ROM / DVD distributions. No installation on server required thanks to a built-in highly optimized database engine! You can fully customize the search results (no ads!) and has tons of Google-like features. Indexes HTML, PDF, Office,Flash docs & many more
With this software you can create professional navigation schemes (12 types, e.g: sitemap trees, dhtml menus, Google sitemaps, and more). SiteXpert can also generate a search engine (Internet, Intranet or Offline). SiteXpert comes with hundreds of predefined graphical schemes. Included are free DreamWeaver/FrontPage/GoLive/NOF plugins. You can use the crawler module to automatically read existing website structure (through http/ftp/local disk)
Zoom is a package for web developers to add a powerful custom search engine to their website in a matter of minutes. No complicated server-side setup procedures, just index and upload from your own Windows computer. Features include: "Google-like" context descriptions, wildcard and exact phrase searching, logging, statistic reports, sort by date options, spelling suggestions and more. Available for PHP, ASP, JavaScript and CGI platforms.
Build a custom web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, government databases, real estate listings, or build a custom email extractor.
Creating your own web grabber that can Screen Scrape the data to a database or Excel has never been easier. The website spider / crawlers can even sort and filter the webpages while spidering.
BlackWidow is a multi function internet tool. It is a site ripper, offline browser, web site scanner, a site mapping tool and a site mirroring tool. Use it to scan a site, and create a complete profile of the site structure, files, external links and even link errors. BlackWidow will scan HTTP sites, SSL sites (HTTPS) and FTP sites. uses threads, pull links from Java Scripts and Java Scripts files, Print the structure of the web site + more.
Website Ripper Copier is the most fully-featured high-speed webcopier for getting Web data. It empowers you to easily download websites for offline browsing, grab site files of certain types (picture, movie, MP3, PDF, Flash, webzip ..), teleport files as a download manager with HTTP, HTTPS and FTP resumption support, and mirror sites. It is also a site map explorer and tabbed anti pop-up Web browser! It is the best pro robot for today`s sites!
WebCloner is a site ripper, images and Flash downloader, offline browser, and e-book compiler (CHM, EXE just for Pro version) with advanced configuration options. This way WebCloner allows you to scan, download, browse and convert to e-books (CHM) entire websites or just parts of them. WebCloner saves the content directly into packed web files so it will come easier for you to archive, share and password protect saved content.
WebCloner is a site ripper, images and Flash downloader, offline browser, and e-book compiler with advanced configuration options. WebCloner allows you to scan, download, browse and convert to e-books (EXE or CHM) entire websites or just parts of them. It is the only offline browser that can compile EXE based e-books. WebCloner saves the content directly into packed web files and also it comes with a built in image browser.
Manage and track visits by robots, spiders and crawlers to your website. Identify which spiders visited, when they visited, and what pages they indexed. Direct spiders to your most important pages, and exclude them from non-public pages. Maximizes effectiveness with search engines by avoiding over/under submitting. Excellent to improve search engine placements for users of automatic submission software such as WebPosition, TopDog, AddWeb, etc.
