Skip to content

package

robots-parser

v3.0.0

A specification compliant robots.txt parser with wildcard (*) matching support.

node >=0.100 depscjstypesopen source
license
MIT
deps
0
unpacked
42.5 kB
files
10
node
>=0.10
npm install robots-parser@3.0.0

README

GitHub-flavored package documentation.

markdown
downloads

Across all versions.

Download trend unavailable.

compatibility

Runtime metadata.

node>=0.10
cjstypes
keywords

Tagged topics.

robots.txtparseruser-agentscraperspiderbotrobots-exclusion-standard