normalize-url

  • Version 10.0.0
  • Published
  • 72.5 kB
  • No dependencies
  • MIT license

Install

npm i normalize-url
yarn add normalize-url
pnpm add normalize-url

Overview

Normalize a URL

Index

Functions

Type Aliases

Functions

function normalizeUrl

normalizeUrl: (url: string, options?: Options) => string;
  • [Normalize](https://en.wikipedia.org/wiki/URL_normalization) a URL.

    URLs with custom protocols are not normalized and just passed through by default. Supported protocols are: https, http, file, and data. Use the customProtocols option to add support for additional protocols.

    Human-friendly URLs with basic auth (for example, user:password@sindresorhus.com) are not handled because basic auth conflicts with custom protocols. [Basic auth URLs are also deprecated.](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#access_using_credentials_in_the_url)

    A scheme with a dot in it followed by // is a custom protocol, so it is passed through as well and nothing in the URL is normalized. example.com/path is a host and gets http: prepended, but example.com://host/path is a scheme named example.com and is left alone. Use the customProtocols option to normalize one on purpose.

    Whatever the options, a normalized URL also gets these changes:

    - Tab, carriage return, and line feed are removed from anywhere, as a URL parser does. Whitespace and C0 controls are removed from the ends, including U+00A0 and a byte order mark, which are often left over from copying a URL. Other whitespace inside the URL is kept, and escaped where a URL parser escapes it. - A URL parser lowercases the protocol. For a built-in protocol, it also lowercases the host, converts an internationalized host to Punycode, resolves . and .. path segments, and removes a default port. - Trailing dots are removed from a domain host, unless that changes the name: 159.189.213.. keeps them, since 159.189.213 is the IPv4 address 159.189.0.213. The host of a custom protocol is opaque, so its dots are kept. - Duplicate slashes in the path are removed, except the two of a protocol-like segment such as /https://. - Percent escapes are uppercased (%2f → %2F), except in the media type of a data URL, where percent-looking text is literal MIME content. Escapes of unreserved characters (letters, digits, -, ., _, ~) are decoded (%7E → ~), except in a fragment, the media type of a data URL, the host of a custom protocol, and a path that does not start with /. Fragment escapes stay encoded so they cannot become text directives. The path and the query also decode other escapes where the character means the same unescaped, as in %5B → [. Escapes that would change the structure, like %2F, %3F, %23, and %26, stay escaped, so %2F and / stay different. - The query is written the way URLSearchParams writes it: a space in a value is +, a space in a key is %20, a ? is %3F, and a % that starts no escape is %25. Empty parameters, as in ?a=1&&b=2, are removed. This happens with sortQueryParameters: false too. An escape that is not valid UTF-8 becomes U+FFFD, as URLSearchParams decodes it, so ?q=%E9 becomes ?q=%EF%BF%BD, which changes the value for a server that reads the query as Latin-1. - A data URL gets its MIME type and charset value lowercased. text/plain is removed when nothing follows it, and so is charset=us-ascii on text/plain. Quotes are removed from a parameter value that does not need them, and whitespace is removed from the ends of base64 data.

    Parameter url

    URL to normalize, including [data URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs).

    Example 1

    import normalizeUrl from 'normalize-url';
    normalizeUrl('sindresorhus.com');
    //=> 'http://sindresorhus.com'
    normalizeUrl('//www.sindresorhus.com:80/../baz?b=bar&a=foo');
    //=> 'http://sindresorhus.com/baz?a=foo&b=bar'

Type Aliases

type Options

type Options = {
/**
Usually `'https'` or `'http'`. A trailing `:` is accepted. A value that is not a scheme, such as `'https://'`, throws a `TypeError`.
__Note:__ A protocol that is not built in, such as `'ftp'`, is best-effort. The URL it creates is normalized, while the same URL written with that protocol is passed through as a custom protocol, unless the protocol is in `customProtocols`.
@default 'http'
*/
readonly defaultProtocol?: 'https' | 'http' | (string & Record<never, never>);
/**
Protocols to normalize in addition to the built-in ones (`https`, `http`, `file`, `data`).
Useful for HTTP-like custom protocols such as Electron schemes or app-specific protocols.
The protocols should be specified without `:`, though a trailing one is accepted. The name is case-insensitive.
__Note:__ Options that rewrite the path (`removePath`, `transformPath`, `removeDirectoryIndex`, and the percent-decoding of the path) only apply when the path starts with a `/`, as in `sindre://host/a/b` and `sindre:/a/b`. A URL whose path is opaque, such as `sindre:a/b`, has no segments for those options to work on, and a URL parser does not let the library change it, so they are silently ignored for it.
__Note:__ `transformPath` and `removeDirectoryIndex` rebuild the path from its components, which cannot spell the two slashes of a protocol lookalike such as `sindre://host/a/s3://b/index.html`. One of them is dropped for those two options, while the library keeps both everywhere else.
@default undefined
@example
```
normalizeUrl('sindre://www.sorhus.com', {customProtocols: ['sindre']});
//=> 'sindre://sorhus.com'
normalizeUrl('sindre://www.sorhus.com/foo/', {customProtocols: ['sindre']});
//=> 'sindre://sorhus.com/foo'
```
*/
readonly customProtocols?: readonly string[];
/**
Prepends `defaultProtocol` to the URL if it's protocol-relative.
With `false`, a protocol-relative URL stays protocol-relative, even with `defaultProtocol` or `forceHttps` set.
@default true
@example
```
normalizeUrl('//sindresorhus.com');
//=> 'http://sindresorhus.com'
normalizeUrl('//sindresorhus.com', {normalizeProtocol: false});
//=> '//sindresorhus.com'
```
*/
readonly normalizeProtocol?: boolean;
/**
Normalizes HTTPS URLs to HTTP.
@default false
__Note:__ changing the scheme drops the port that was the default one for the old scheme, so `forceHttp` turns `https://example.com:443` into `http://example.com` and `forceHttps` turns `http://example.com:80` into `https://example.com`. A port that is not a default for either scheme is kept.
@example
```
normalizeUrl('https://sindresorhus.com');
//=> 'https://sindresorhus.com'
normalizeUrl('https://sindresorhus.com', {forceHttp: true});
//=> 'http://sindresorhus.com'
```
*/
readonly forceHttp?: boolean;
/**
Normalizes HTTP URLs to HTTPS.
This option cannot be used with the `forceHttp` option at the same time.
@default false
__Note:__ changing the scheme drops the port that was the default one for the old scheme, so `forceHttp` turns `https://example.com:443` into `http://example.com` and `forceHttps` turns `http://example.com:80` into `https://example.com`. A port that is not a default for either scheme is kept.
@example
```
normalizeUrl('http://sindresorhus.com');
//=> 'http://sindresorhus.com'
normalizeUrl('http://sindresorhus.com', {forceHttps: true});
//=> 'https://sindresorhus.com'
```
*/
readonly forceHttps?: boolean;
/**
Strip the [authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) part of a URL.
@default true
@example
```
normalizeUrl('https://user:password@sindresorhus.com');
//=> 'https://sindresorhus.com'
normalizeUrl('https://user:password@sindresorhus.com', {stripAuthentication: false});
//=> 'https://user:password@sindresorhus.com'
```
*/
readonly stripAuthentication?: boolean;
/**
Removes hash from the URL.
@default false
@example
```
normalizeUrl('sindresorhus.com/about.html#contact');
//=> 'http://sindresorhus.com/about.html#contact'
normalizeUrl('sindresorhus.com/about.html#contact', {stripHash: true});
//=> 'http://sindresorhus.com/about.html'
```
*/
readonly stripHash?: boolean;
/**
Remove the protocol from the URL: `http://sindresorhus.com` → `sindresorhus.com`.
It removes an `https://`, an `http://` or a leading `//` from a protocol-relative URL, and nothing else.
__Note:__ The result is not a full URL, so normalizing it again can change it: a port that the removed protocol kept is the default one of the protocol that gets prepended next.
@default false
@example
```
normalizeUrl('https://sindresorhus.com');
//=> 'https://sindresorhus.com'
normalizeUrl('https://sindresorhus.com', {stripProtocol: true});
//=> 'sindresorhus.com'
```
*/
readonly stripProtocol?: boolean;
/**
Strip the [text fragment](https://web.dev/text-fragments/) part of the URL
__Note:__ The text fragment will always be removed if the `stripHash` option is set to `true`, as the hash contains the text fragment.
@default true
@example
```
normalizeUrl('http://sindresorhus.com/about.html#:~:text=hello');
//=> 'http://sindresorhus.com/about.html'
normalizeUrl('http://sindresorhus.com/about.html#section:~:text=hello');
//=> 'http://sindresorhus.com/about.html#section'
normalizeUrl('http://sindresorhus.com/about.html#:~:text=hello', {stripTextFragment: false});
//=> 'http://sindresorhus.com/about.html#:~:text=hello'
normalizeUrl('http://sindresorhus.com/about.html#section:~:text=hello', {stripTextFragment: false});
//=> 'http://sindresorhus.com/about.html#section:~:text=hello'
```
*/
readonly stripTextFragment?: boolean;
/**
Removes `www.` from the URL.
__Note:__ A `www.` that is not followed by a proper domain is left alone, so `www.com` and `www.a.b` are not changed.
@default true
@example
```
normalizeUrl('http://www.sindresorhus.com');
//=> 'http://sindresorhus.com'
normalizeUrl('http://www.sindresorhus.com', {stripWWW: false});
//=> 'http://www.sindresorhus.com'
```
*/
readonly stripWWW?: boolean;
/**
Removes query parameters that matches any of the provided strings or regexes. The match is against the percent-decoded key, the way `URLSearchParams` reads one, so `+` is a space and `%2F` is a slash: write `a/b` rather than `a%2Fb`.
Global and sticky regex flags are stripped.
@default [/^utm_\w+/i]
@example
```
normalizeUrl('www.sindresorhus.com?foo=bar&ref=test_ref', {
removeQueryParameters: ['ref']
});
//=> 'http://sindresorhus.com/?foo=bar'
```
If a boolean is provided, `true` will remove all the query parameters.
```
normalizeUrl('www.sindresorhus.com?foo=bar', {
removeQueryParameters: true
});
//=> 'http://sindresorhus.com'
```
`false` will not remove any query parameter.
```
normalizeUrl('www.sindresorhus.com?foo=bar&utm_medium=test&ref=test_ref', {
removeQueryParameters: false
});
//=> 'http://sindresorhus.com/?foo=bar&ref=test_ref&utm_medium=test'
```
*/
readonly removeQueryParameters?: ReadonlyArray<RegExp | string> | boolean;
/**
Keeps only query parameters that matches any of the provided strings or regexes. The match is against the percent-decoded key, the same as `removeQueryParameters`. An empty array matches nothing, so it removes every parameter, the same as the `true` of `removeQueryParameters`.
__Note:__ It overrides the `removeQueryParameters` option.
Global and sticky regex flags are stripped.
@default undefined
@example
```
normalizeUrl('https://sindresorhus.com?foo=bar&ref=unicorn', {
keepQueryParameters: ['ref']
});
//=> 'https://sindresorhus.com/?ref=unicorn'
```
*/
readonly keepQueryParameters?: ReadonlyArray<RegExp | string>;
/**
Removes trailing slash.
__Note:__ the sole `/` of a root path is removed too, unless the `removeSingleSlash` option is set to `false` **and** the input wrote that slash itself.
__Note:__ `removeSingleSlash: false` reads the trailing slash off the input as it was written, so a slash that belongs to a dot segment a URL parser resolves away still counts. `https://example.com/a/../` and `https://example.com/a/..` are the same URL, but only the first keeps the root slash.
@default true
@example
```
normalizeUrl('http://sindresorhus.com/redirect/');
//=> 'http://sindresorhus.com/redirect'
normalizeUrl('http://sindresorhus.com/redirect/', {removeTrailingSlash: false});
//=> 'http://sindresorhus.com/redirect/'
normalizeUrl('http://sindresorhus.com/', {removeTrailingSlash: false});
//=> 'http://sindresorhus.com'
```
*/
readonly removeTrailingSlash?: boolean;
/**
Remove a sole `/` pathname in the output. This option is independent of `removeTrailingSlash`.
The `/` is kept when a query or a hash follows it, as in `https://sindresorhus.com/?a=1`.
@default true
@example
```
normalizeUrl('https://sindresorhus.com/');
//=> 'https://sindresorhus.com'
normalizeUrl('https://sindresorhus.com/', {removeSingleSlash: false});
//=> 'https://sindresorhus.com/'
```
*/
readonly removeSingleSlash?: boolean;
/**
Removes the default directory index file from path that matches any of the provided strings or regexes. The match is against the percent-decoded component, the same as `transformPath` is given, so `index.html` matches a path spelled `index%2Ehtml`. It runs after `transformPath`, so it also removes an index file the function leaves at the end.
When `true`, the regex `/^index\.[a-z]+$/` is used.
__Note:__ removing the index leaves a trailing slash on what is left, whether or not the input wrote one, so `http://example.com/foo/index.html` becomes `http://example.com/foo/` and then `http://example.com/foo`. The `removeTrailingSlash` option decides whether that slash stays.
Global and sticky regex flags are stripped.
@default false
@example
```
normalizeUrl('www.sindresorhus.com/foo/default.php', {
removeDirectoryIndex: [/^default\.[a-z]+$/]
});
//=> 'http://sindresorhus.com/foo'
```
*/
readonly removeDirectoryIndex?: boolean | ReadonlyArray<RegExp | string>;
/**
Removes an explicit port number from the URL.
Port 443 is always removed from HTTPS URLs and 80 is always removed from HTTP URLs regardless of this option.
@default false
@example
```
normalizeUrl('sindresorhus.com:123', {
removeExplicitPort: true
});
//=> 'http://sindresorhus.com'
```
*/
readonly removeExplicitPort?: boolean;
/**
Sorts the query parameters by key, in code unit order, so an uppercase letter sorts before a lowercase one and a digit before a letter. Parameters that share a key keep the order they were written in.
__Note:__ The option only changes the order. The query is written the same way either way (see `normalizeUrl`).
@default true
@example
```
normalizeUrl('www.sindresorhus.com?b=two&a=one&c=three', {
sortQueryParameters: false
});
//=> 'http://sindresorhus.com/?b=two&a=one&c=three'
```
*/
readonly sortQueryParameters?: boolean;
/**
Controls how query parameters with empty values are formatted.
- `'preserve'` - Keep the original format (`?key` stays `?key`, `?key=` stays `?key=`). If the same key appears with both formats (`?a&a=`), all instances will use the format without `=`.
- `'always'` - Always include `=` for empty values (`?key` becomes `?key=`)
- `'never'` - Never include `=` for empty values (`?key=` becomes `?key`)
@default 'preserve'
@example
```
normalizeUrl('www.sindresorhus.com?a&b=', {emptyQueryValue: 'always'});
//=> 'http://sindresorhus.com/?a=&b='
normalizeUrl('www.sindresorhus.com?a&b=', {emptyQueryValue: 'never'});
//=> 'http://sindresorhus.com/?a&b'
```
*/
readonly emptyQueryValue?: 'preserve' | 'always' | 'never';
/**
Removes the entire URL path, leaving the host with no path. The query and the hash stay. It takes precedence over the `transformPath` option.
@default false
@example
```
normalizeUrl('https://example.com/path/to/page', {
removePath: true
});
//=> 'https://example.com'
```
*/
readonly removePath?: boolean;
/**
Custom function to transform the URL path components.
The function receives an array of non-empty path components and should return a modified array. Returning nothing removes the path. The components are percent-decoded, so an unreserved escape has become the character it stands for by the time the function sees it, and one it returns is not decoded again. It runs before `removeDirectoryIndex`, so the function still sees the index file, and it is skipped when `removePath` is set.
@default false
@example
```
// Keep only the first path component
normalizeUrl('https://example.com/api/v1/users', {
transformPath: (pathComponents) => pathComponents.slice(0, 1)
});
//=> 'https://example.com/api'
// Remove specific components
normalizeUrl('https://example.com/admin/users', {
transformPath: (pathComponents) => pathComponents.filter(c => c !== 'admin')
});
//=> 'https://example.com/users'
```
*/
readonly transformPath?:
| ((pathComponents: string[]) => readonly string[] | undefined)
| false;
};

    Package Files (1)

    Dependencies (0)

    No dependencies.

    Dev Dependencies (1)

    Peer Dependencies (0)

    No peer dependencies.

    Badge

    To add a badge like this onejsDocs.io badgeto your package's README, use the codes available below.

    You may also use Shields.io to create a custom badge linking to https://www.jsdocs.io/package/normalize-url.

    • Markdown
      [![jsDocs.io](https://img.shields.io/badge/jsDocs.io-reference-blue)](https://www.jsdocs.io/package/normalize-url)
    • HTML
      <a href="https://www.jsdocs.io/package/normalize-url"><img src="https://img.shields.io/badge/jsDocs.io-reference-blue" alt="jsDocs.io"></a>