URL Extractor

Paste an email, a web page’s HTML, a chat export or a server log and get a clean list of every link in it. The extractor finds http, https and ftp addresses and links that start with www, trims the punctuation that sticks to links in sentences, keeps brackets that belong to the address (as in Wikipedia links), removes duplicates and can list just the domains, with a count of links per domain.

How to use this tool

1. Paste the text or HTML.
2. Choose full links or domains only, and tick “bare domains” to also catch addresses written without http or www.
3. Optionally remove tracking query strings, keep only certain domains or file types, or leave some out.
4. Copy the list; the table counts links per domain.

Formula

Link = (http:// | https:// | ftp:// | www.) followed by characters up to a space, quote or angle bracket, minus trailing punctuation

A full stop, comma, colon, quote or closing bracket at the end of a link is treated as sentence punctuation, unless the bracket has a matching opening bracket inside the link. Each link is checked with the browser’s URL parser. Duplicates are compared with the host name in lower case and a trailing slash ignored, so HTTPS://Docs.Example.com/start and https://docs.example.com/start count as one. Bare domains must end in a common top-level domain, which keeps file names such as main.js out.

Example

The sample text gives 4 links: https://docs.example.com/start?ref=mail, www.example.org, the Wikipedia link with its “(disambiguation)” part intact, and the PDF link. The upper-case copy of the docs link is removed as a duplicate. With “domains only”, the list is docs.example.com, example.org, en.wikipedia.org and files.example.com.