New from our research ·

One question, two different moves: what 1,332 answers from ChatGPT and Claude models showed us

One question, two different moves: what 1,332 answers from ChatGPT and Claude models showed us

We asked 29 everyday questions, each in English and in Russian, to six models: three from the Claude family and three from the GPT family behind ChatGPT. Three runs per model, each in a fresh chat. That is 1,044 answers. The six most promising questions were then brought up to seven runs per model, 288 more answers.

On 12 of the 29, both families gave essentially one idea. On a few, they went different ways.

The question that split

Come up with 1 unconventional way to use a double AI answer in everyday life, other than work and study. A concrete example, 1-2 paragraphs.

Seven runs for each of the six models, 42 answers in all:

A second reader, Kimi from Moonshot, sorted the 42 files without model names and arrived at the same counts, 19 and 20.

The difference is in the mechanism, not the tone. One side compares answers to settle something. The other mixes answers to make something new. Both are useful, and together they give more than either alone.

Other questions, other splits

A friend who is always 30 to 40 minutes late: 20 of 21 Claude answers asked gently for a warning and offered to adjust; 16 of 21 GPT answers set a line: "I'll wait about 15 minutes, then get on with my day." A small coffee shop against a chain: 13 of 21 Claude answers went to your own mug and a barista who knows your name; 19 of 21 GPT answers went to a shared board for the neighbourhood.

Why Poly Duo shows both

Poly Duo is a free iPhone Shortcut: your own ChatGPT and Claude apps each answer the same question on their own, then Claude compares the two answers. On Android, Duo is built into the free Poly app. It is free and open source: github.com/vadimchernets/poly-duo. This test shows what that looks like on a question with no single right answer: two moves, and you keep the one that fits.

With web search switched on, the split stayed: Claude kept its move in 14 of 15 runs, GPT in 11 of 15.

Install page: polyhelper.ai/duo.

Vadym Chernets, AI researcher (NeurIPS 2026). Test run 29 October 2026.

The paper →