Our agent
The good agent we're asking you to let in
If our agent behaved badly, nothing it found would mean anything. So it follows the same rules we'd want every agent to follow.
What it does
- Reads your robots.txt first. If it's told to stay out, it stops there.
- Reads your home page, one product page, your cart and checkout pages, and a few standard files (sitemap, agent manifests, security.txt).
- If your store advertises a UCP shopping catalog in
/.well-known/ucp, sends one catalog search to read a few products and prices, the way a shopping agent would. - Visits your home page twice, once signed and once unsigned, to see whether your bot protection treats a verified agent differently.
- Goes slowly: one request at a time, about a second apart, at most 30 requests per check.
What it never does
- Never adds to cart, creates an account, logs in, fills in a form or pays.
- Only reads. Every request is an HTTP GET, except that one read-only catalog search. It never touches carts, checkout or orders.
- Never publishes a store's failing results by name. Only stores that pass are listed.
How it identifies itself
Every request carries this user agent:
Mozilla/5.0 (compatible; PegasusKYA/0.1; +https://pegasuskya.ai/our-agent)
And a Web Bot Auth signature (HTTP Message Signatures, Ed25519) covering the site it's visiting:
Signature-Agent: "https://pegasuskya.ai"
Signature-Input: sig1=("@authority" "signature-agent");created=…;expires=…;keyid="…";alg="ed25519";nonce="…";tag="web-bot-auth"
Signature: sig1=:…:
Our public key is published at /.well-known/http-message-signatures-directory. Current key id: loading…
Anyone can verify a request really came from us by checking the signature against that key. Many bot-protection services can do this for you.
How to opt out
Add this to your robots.txt and our agent will stop at that file and record nothing else:
User-agent: PegasusKYA Disallow: /
To have a store removed entirely, email [email protected] from an address at that store's domain.