Leaderboard

TydiQA

tydiqa

Record

Category: knowledge
Scored models: 33

Scores

Model	Score
microsoft/Phi-3.5-MoE-instruct	67.1
meta-llama/Meta-Llama-3-8B	66.8
meta-llama/Meta-Llama-3-70B	66.8
microsoft/Phi-3.5-mini-instruct	62.2
microsoft/Phi-3.5-mini-instruct	62.2
microsoft/Phi-3.5-mini-instruct	62.2
allenai/open-instruct-human-mix-65b	58.5
allenai/open-instruct-human-mix-65b	58.5
allenai/tulu-30b	58.2
—	58.2
allenai/tulu-30b	58.2
allenai/tulu-30b	58.2
allenai/open-instruct-cot-13b	52.2
allenai/tulu-65b	51.8
allenai/open-instruct-sni-7b	47.9
allenai/tulu-13b	46.1
—	46.1
allenai/tulu-7b	45.7
allenai/tulu-7b	45.7
—	45.7
allenai/open-instruct-flan-v2-7b	45
allenai/open-instruct-cot-7b	44.4
allenai/open-instruct-dolly-7b	43.6
allenai/open-instruct-human-mix-7b	42.2

Remaining fields

Identity, launch, related records, and measured speed are shown above. This is the rest of the normalized record.

category: knowledge
id: tydiqa
name: TydiQA
schema version: local-ai-registry/v1

rows

variant	conf	context	model id	org	rank	root	score
Phi-3.5-MoE-instruct	medium	MEGA TyDi QA	phi-3-5-moe-instruct	microsoft	1	microsoft/Phi-3.5-MoE-instruct	67.1
llama-3-tulu-2-8b	low	GSM8k 8-shot cot \| BBH 3-shot cot	meta-llama-3-8b	allenai	2	meta-llama/Meta-Llama-3-8B	66.8
llama-3-tulu-2-70b	low	GSM8k 8-shot cot \| BBH 3-shot cot	meta-llama-3-70b	allenai	3	meta-llama/Meta-Llama-3-70B	66.8
Phi-3.5-mini-instruct	medium	MEGA TyDi QA	phi-3-5-mini-instruct	microsoft	4	microsoft/Phi-3.5-mini-instruct	62.2
Phi-3.5-mini-instruct-bnb-4bit	medium	MEGA TyDi QA	phi-3-5-mini-instruct	unsloth	5	microsoft/Phi-3.5-mini-instruct	62.2
Phi-3.5-mini-instruct	medium	MEGA TyDi QA	phi-3-5-mini-instruct	unsloth	6	microsoft/Phi-3.5-mini-instruct	62.2
open-instruct-human-mix-65B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	7	allenai/open-instruct-human-mix-65b	58.5
open-instruct-human-mix-65b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	8	allenai/open-instruct-human-mix-65b	58.5
tulu-30B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	9	allenai/tulu-30b	58.2
tulu-30B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	10	—	58.2
tulu-30b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	11	allenai/tulu-30b	58.2
tulu-30B-GPTQ	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	12	allenai/tulu-30b	58.2
open-instruct-cot-13b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	13	allenai/open-instruct-cot-13b	52.2
tulu-65b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	14	allenai/tulu-65b	51.8
open-instruct-sni-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	15	allenai/open-instruct-sni-7b	47.9
tulu-13B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	16	allenai/tulu-13b	46.1
tulu-13B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	17	—	46.1
tulu-7B-GGUF	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	18	allenai/tulu-7b	45.7
tulu-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	19	allenai/tulu-7b	45.7
tulu-7B-fp16	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	TheBloke	20	—	45.7
open-instruct-flan-v2-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	21	allenai/open-instruct-flan-v2-7b	45
open-instruct-cot-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	22	allenai/open-instruct-cot-7b	44.4
open-instruct-dolly-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	23	allenai/open-instruct-dolly-7b	43.6
open-instruct-human-mix-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	24	allenai/open-instruct-human-mix-7b	42.2
open-instruct-code-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	25	allenai/open-instruct-code-alpaca-7b	36.7
open-instruct-unnatural-instructions-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	26	allenai/open-instruct-unnatural-instructions-7b	36.3
open-instruct-self-instruct-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	27	allenai/open-instruct-self-instruct-7b	35.4
open-instruct-pythia-6.9b-tulu	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	28	allenai/open-instruct-pythia-6.9b-tulu	33.4
open-instruct-stanford-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	29	allenai/open-instruct-stanford-alpaca-7b	31.2
open-instruct-baize-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	30	allenai/open-instruct-baize-7b	29.8
open-instruct-oasst1-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	31	allenai/open-instruct-oasst1-7b	26.8
open-instruct-sharegpt-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	32	allenai/open-instruct-sharegpt-7b	25.5
open-instruct-gpt4-alpaca-7b	low	GSM CoT \| BBH CoT \| TydiQA Closed-book	Unknown	allenai	33	allenai/open-instruct-gpt4-alpaca-7b	23.6

Provenance & metadata (1)

source

kind: leaderboard-scrape
paths: benchmarks/tydiqa.html
url: huggingface.co/spaces/tydiqa ↗