Useful when you need to display, store, deduplicate, sort, compare, etc, URLs.
[!NOTE]
This package does not do URL sanitization. Garbage in, garbage out. If you use this in a server context and accept URLs as user input, it's up to you to protect against invalid URLs, path traversal attacks, etc.
Install
npm install normalize-url
Usage
import normalizeUrl from 'normalize-url';normalizeUrl('sindresorhus.com');//=> 'http://sindresorhus.com'normalizeUrl('//www.sindresorhus.com:80/../baz?b=bar&a=foo');//=> 'http://sindresorhus.com/baz?a=foo&b=bar'
API
normalizeUrl(url, options?)
URLs with custom protocols are not normalized and just passed through by default. Supported protocols are: https, http, file, and data. Use the customProtocols option to add support for additional protocols.
Human-friendly URLs with basic auth (for example, user:password@sindresorhus.com) are not handled because basic auth conflicts with custom protocols. Basic auth URLs are also deprecated.
Note: a scheme with a dot in it followed by // is a custom protocol, so it is passed through as well and nothing in the URL is normalized. example.com/path is a host and gets http: prepended, but example.com://host/path is a scheme named example.com and is left alone. Use the customProtocols option to normalize one on purpose.
Whatever the options, a normalized URL also gets these changes:
Tab, carriage return, and line feed are removed from anywhere, as a URL parser does. Whitespace and C0 controls are removed from the ends, including U+00A0 and a byte order mark, which are often left over from copying a URL. Other whitespace inside the URL is kept, and escaped where a URL parser escapes it.
A URL parser lowercases the protocol. For a built-in protocol, it also lowercases the host, converts an internationalized host to Punycode, resolves . and .. path segments, and removes a default port.
Trailing dots are removed from a domain host, unless that changes the name: 159.189.213.. keeps them, since 159.189.213 is the IPv4 address 159.189.0.213. The host of a custom protocol is opaque, so its dots are kept.
Duplicate slashes in the path are removed, except the two of a protocol-like segment such as /https://.
Percent escapes are uppercased (%2f → %2F), except in the media type of a data URL, where percent-looking text is literal MIME content. Escapes of unreserved characters (letters, digits, -, ., _, ~) are decoded (%7E → ~), except in a fragment, the media type of a data URL, the host of a custom protocol, and a path that does not start with /. Fragment escapes stay encoded so they cannot become text directives. The path and the query also decode other escapes where the character means the same unescaped, as in %5B → [. Escapes that would change the structure, like %2F, %3F, %23, and %26, stay escaped, so %2F and / stay different.
The query is written the way URLSearchParams writes it: a space in a value is +, a space in a key is %20, a ? is %3F, and a % that starts no escape is %25. Empty parameters, as in ?a=1&&b=2, are removed. This happens with sortQueryParameters: false too. An escape that is not valid UTF-8 becomes U+FFFD, as URLSearchParams decodes it, so ?q=%E9 becomes ?q=%EF%BF%BD, which changes the value for a server that reads the query as Latin-1.
A data URL gets its MIME type and charset value lowercased. is removed when nothing follows it, and so is on . Quotes are removed from a parameter value that does not need them, and whitespace is removed from the ends of base64 data.
Usually 'https' or 'http'. A trailing : is accepted. A value that is not a scheme, such as 'https://', throws a TypeError.
Note: A protocol that is not built in, such as 'ftp', is best-effort. The URL it creates is normalized, while the same URL written with that protocol is passed through as a custom protocol, unless the protocol is in customProtocols.
customProtocols
Type: string[]
Default: undefined
Protocols to normalize in addition to the built-in ones (https, http, file, data).
Useful for HTTP-like custom protocols such as Electron schemes or app-specific protocols.
The protocols should be specified without :, though a trailing one is accepted. The name is case-insensitive.
Note: Options that rewrite the path (removePath, transformPath, removeDirectoryIndex, and the percent-decoding of the path) only apply when the path starts with a /, as in sindre://host/a/b and sindre:/a/b. A URL whose path is opaque, such as sindre:a/b, has no segments for those options to work on, and a URL parser does not let the library change it, so they are silently ignored for it.
Note:transformPath and removeDirectoryIndex rebuild the path from its components, which cannot spell the two slashes of a protocol lookalike such as sindre://host/a/s3://b/index.html. One of them is dropped for those two options, while the library keeps both everywhere else.
Note: changing the scheme drops the port that was the default one for the old scheme, so forceHttp turns https://example.com:443 into http://example.com and forceHttps turns http://example.com:80 into https://example.com. A port that is not a default for either scheme is kept.
Note: changing the scheme drops the port that was the default one for the old scheme, so forceHttp turns https://example.com:443 into http://example.com and forceHttps turns http://example.com:80 into https://example.com. A port that is not a default for either scheme is kept.
Remove the protocol from the URL: http://sindresorhus.com → sindresorhus.com.
It removes an https://, an http:// or a leading // from a protocol-relative URL, and nothing else.
Note: The result is not a full URL, so normalizing it again can change it: a port that the removed protocol kept is the default one of the protocol that gets prepended next.
Remove query parameters that matches any of the provided strings or regexes. The match is against the percent-decoded key, the way URLSearchParams reads one, so + is a space and %2F is a slash: write a/b rather than a%2Fb.
Keeps only query parameters that matches any of the provided strings or regexes. The match is against the percent-decoded key, the same as removeQueryParameters. An empty array matches nothing, so it removes every parameter, the same as the true of removeQueryParameters.
Note: It overrides the removeQueryParameters option.
Note: the sole / of a root path is removed too, unless the removeSingleSlash option is set to falseand the input wrote that slash itself.
Note:removeSingleSlash: false reads the trailing slash off the input as it was written, so a slash that belongs to a dot segment a URL parser resolves away still counts. https://example.com/a/../ and https://example.com/a/.. are the same URL, but only the first keeps the root slash.
Removes the default directory index file from path that matches any of the provided strings or regexes. The match is against the percent-decoded component, the same as transformPath is given, so index.html matches a path spelled index%2Ehtml. It runs after transformPath, so it also removes an index file the function leaves at the end.
When true, the regex /^index\.[a-z]+$/ is used.
Note: removing the index leaves a trailing slash on what is left, whether or not the input wrote one, so http://example.com/foo/index.html becomes http://example.com/foo/ and then http://example.com/foo. The removeTrailingSlash option decides whether that slash stays.
Sorts the query parameters by key, in code unit order, so an uppercase letter sorts before a lowercase one and a digit before a letter. Parameters that share a key keep the order they were written in.
Note: The option only changes the order. The query is written the same way either way (see normalizeUrl).
Controls how query parameters with empty values are formatted.
'preserve' - Keep the original format (?key stays ?key, ?key= stays ?key=). If the same key appears with both formats (?a&a=), all instances will use the format without =.
'always' - Always include = for empty values (?key becomes ?key=)
'never' - Never include = for empty values (?key= becomes ?key)
Custom function to transform the URL path components. The function receives an array of non-empty path components and should return a modified array. Returning nothing removes the path. The components are percent-decoded, so an unreserved escape has become the character it stands for by the time the function sees it, and one it returns is not decoded again. It runs before removeDirectoryIndex, so the function still sees the index file, and it is skipped when removePath is set.
// Keep only the first path componentnormalizeUrl('https://example.com/api/v1/users', { transformPath: (pathComponents) => pathComponents.slice(0, 1)});//=> 'https://example.com/api'// Remove specific componentsnormalizeUrl('https://example.com/admin/users', { transformPath: (pathComponents) => pathComponents.filter(c => c !== 'admin')});//=> 'https://example.com/users'
Related
compare-urls - Compare URLs by first normalizing them