Skip to content

package

robots-parser

v3.0.1

A specification compliant robots.txt parser with wildcard (*) matching support.

node >=10.0.00 depscjstypesopen source
license
MIT
deps
0
unpacked
45.2 kB
files
10
node
>=10.0.0
npm install robots-parser@3.0.1

README

GitHub-flavored package documentation.

markdown
downloads

Across all versions.

Download trend unavailable.

compatibility

Runtime metadata.

node>=10.0.0
cjstypes
keywords

Tagged topics.

robots.txtparseruser-agentscraperspiderbotrobots-exclusion-standard