How It Works

Start a chat, type your question (LaTeX works) and choose the models that should answer. They answer in parallel, each in its own tab, so you can compare the answers directly.

Hard questions can take several minutes. You may leave the page and come back later.

For every question you choose:

• Models: which models answer. Click a model to select it and open its menu; the switch "Use this model" in the menu deselects it.
• Thinking effort: how long a model may reason. More effort means a slower and more costly answer.
• Web search: whether a model may search the internet.
• Code execution: whether a model may run code.

We recommend not to start every question with the strongest model at the highest thinking effort. For many questions that costs more credit and time than needed; start lower and step up when the answer is not good enough.

After the models answer, click "Include" on every answer you want to build on. Only included answers are visible to the models in the next turn, and you can include several.

Start a new chat for each topic: unrelated questions in one chat make the answers worse.

You can attach PDFs, images, TeX, text and code files to a chat. The models read them, and they stay with the conversation until you remove them. Not every model reads every file type.

The share button in a chat creates a view-only link. Anyone with the link can read the chat, including later messages, until you withdraw the link.

In your settings you can describe your research area and background. The models then answer at your level. Once you have described your research, the incognito switch in a chat leaves the description out.

Every account has a free credit, renewed on a 30-day rolling basis and shown under My Chats. How much it is depends on the funding of the project. Before you send a question, each model's menu shows what a median answer uses. Answers that fail or time out are not counted.

The free credit is for initial usage only. If you need more, you may purchase credit individually or with your research group (see Prepaid Credit).

Heavy-thinking premium models are marked "premium model for prepaid credit" on the front page. They are available for prepaid credit, for your own account or for a pool.

Your questions and attached files are sent to the companies whose models you select (currently OpenAI, Anthropic and Google), through their business interfaces. According to their terms of service, content sent this way is not used to train their models. We have set up the service so that the companies store your data for as short a time as possible. Your chats are stored on our server and are private.

ScienceBench is run by Christian Stump, professor of mathematics at Ruhr University Bochum. The benchmark is described in the paper Benchmarks in Leipzig. Legal details are in the Impressum.

Under My Chats you can buy credit for your own account via PayPal. You can also write to contact@sciencebench.ai to request credit by bank transfer or to discuss other possibilities.

A research group, institute or project can hold one prepaid credit pool that all its members use. You send us the members' email addresses; each member signs up with that address and is added to the pool once the account is verified.

A pool can carry a monthly limit per member, so that no single member uses up the shared credit. Each member sees the pool and the own remaining credit under My Chats.

To set up a pool, write to contact@sciencebench.ai to discuss the possibilities.

Heavy-thinking premium models are marked "premium model for prepaid credit" on the front page. They are available for prepaid credit, for your own account or for a pool.