AskSami's site reader

The user agent

Mozilla/5.0 (compatible; AskSami/1.0; +https://asksami.app)

What starts a visit

• An AskSami customer gives your address as their own product's live site. The read runs after they pay, and again when they change the address or ask for a fresh read.
• Someone pastes a link to one of your pages into their chat, adds it as one of their marketing channels, or gives it while setting up. That one page is read, once, for them.
• While it writes a customer's honest read, AskSami reads the App Store or Google Play listings of the competitors its searches found there: one listing page each, with robots.txt read first.

How much it reads

• A site read: at most 40 pages, and no more than 8 of them blog posts, on the one site it was given. When that site's front page redirects to another host on the same domain (app.example.com from example.com), it reads that host instead. It goes no more than three links deep from the front page. A page listed in the sitemap, or one that an earlier read kept facts from, counts as one link from the front page. It allows at most 8 seconds and 512 KB for a page, and five minutes for the whole read.
• The order: the front page first, then pages whose address says what they are (pricing, sign up, features, about, docs, the FAQ), then the rest, then blog posts, and legal pages (privacy, terms, cookies) last.
• Besides the pages: robots.txt, /sitemap.xml and any sitemap your robots.txt names, up to five sitemap files.
• Links someone pastes: up to 20 in one message, all read within 25 seconds together, and at most 8 seconds each.

robots.txt

A site read fetches robots.txt before any page, and obeys the group for asksami, or the group for every crawler when there is no group for asksami. Every page except the front page is checked against it before it is requested, so a page that robots.txt closes is never asked for. The front page, and any page reached through a redirect, is checked when the answer comes back: if it is closed, it is dropped unread. When the front page redirects to another host on the same domain, that host's robots.txt is read before anything from it is kept. When robots.txt closes the front page, nothing on the site is read.

A robots.txt that answers with a server error is taken to close the whole site. One that answers not found, cannot be reached, or asks the reader to slow down closes nothing.

To keep the site read out of your whole site, add this group:

User-agent: asksami
Disallow: /

A link that a person pastes into their chat, adds on their Channels tab or gives while setting up is fetched for them the way a browser would fetch it when they asked it to, and robots.txt is not read for it.

When it asks as a browser

Reddit feeds, thread pages and share links are read with an ordinary browser user agent, because that is what Reddit's pages need. If a pasted or handed over link refuses AskSami/1.0 with a 403 or a 429, it is asked for once more as a browser. For a 429, it is first asked for again as AskSami/1.0 after the wait the response names, up to six seconds. The site read never changes its user agent.

What it keeps

• From a site read: facts about the product (what it does, who it is for, its prices, its sign up), each with the address of the page it came from and a short quote. Not a copy of the site.
• From a pasted link: the page's title and its description, up to 4,000 characters. For a Reddit or Hacker News post or comment, its text up to 4,000 characters, who posted it, when, and its votes and comment count where they can be read.

What it is not

Pages that AskSami opens during a chat with its web tools, and its web searches, are fetched by the AI provider's own fetcher, not by AskSami/1.0, so they do not show this user agent.

Reaching the people who run it

Write to support@asksami.app. AskSami is run by Ancestorii Ltd, a company registered in the UK.