---
title: The txtfirst probe
desc: What TxtFirstProbe is, what it requests, how to block it, and how to reach a human about it.
order: 7
---

# TxtFirstProbe

If you found this page in your logs, something here requested your homepage. This
page exists so you can see exactly what, without asking.

## It sends two requests, and one of them is disguised

The whole point of the measurement is to compare what a site returns to a browser
against what it returns to an identified program. That means the probe requests your
homepage twice, once as itself and once wearing a current Chrome string:

```
TxtFirstProbe/0.1 (+https://txtfirst.com/probe; research on agent readability)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
```

The second string is not honest, and a site running an argument about honest
identification should say so on the record rather than let you find it in the source.
It is there because the comparison is the finding. Without it there is nothing to
compare against and no way to tell a site that blocks agents from a site that blocks
everything.

## What it does with them

Two `GET` requests to your homepage, run once. No JavaScript execution, no cookies, no
retries, no crawling of any other path, 25 second timeout. It records the HTTP status,
the byte size of the response, and the byte size of the text left after stripping
markup. **It does not store your content**, only those three numbers per request. The
results are published at [measure](/measure) and the raw output is public.

## How to block it

`robots.txt` is checked before either request and both are skipped if you disallow
them. This is done by `urllib.robotparser` in
[the probe source](/research/probe.py), not by a promise on a page.

```
User-agent: TxtFirstProbe
Disallow: /
```

A disallowed site is recorded as `robots-disallow` and carries no measurement.
Nothing is inferred from a block and no site is named for blocking.

## Why it exists

To answer one question with a number instead of an opinion: when a program that is not
a browser asks a website for its front page, what happens. The answer so far is that
most of the time nothing useful happens, which is the argument this whole site is
making. The sample is small and the [limitations](/measure) say so.

## A human

Write to `hello` at this domain. Removal requests are honoured without discussion and
do not require a reason.