0.42
self_reported

Input data provenance

unknown 100.0%

Evaluation source

@measurement.evaluate(params=0, batch=1)
def evaluate(params, batch):
    tokens = np.asarray(batch, dtype=np.uint32)
    logits = model.apply({"params": params}, jnp.asarray(tokens))
    return (np.asarray(logits.argmax(axis=-1)) == (tokens.sum(axis=1) % 2)).astype(np.float32)

Aggregation source

@measurement.aggregate
def aggregate(outputs):
    return float(np.concatenate([np.asarray(o) for o in outputs]).mean())

Inputs and outputs

5 batches · page 1 of 1

Batch 1
Sample 1

Input

Loading vector…

Output

Loading vector…
Sample 2

Input

Loading vector…

Output

Loading vector…
Sample 3

Input

Loading vector…

Output

Loading vector…
Sample 4

Input

Loading vector…

Output

Loading vector…
Sample 5

Input

Loading vector…

Output

Loading vector…
Sample 6

Input

Loading vector…

Output

Loading vector…
Sample 7

Input

Loading vector…

Output

Loading vector…
Sample 8

Input

Loading vector…

Output

Loading vector…
Batch 2
Sample 1

Input

Loading vector…

Output

Loading vector…
Sample 2

Input

Loading vector…

Output

Loading vector…
Sample 3

Input

Loading vector…

Output

Loading vector…
Sample 4

Input

Loading vector…

Output

Loading vector…
Sample 5

Input

Loading vector…

Output

Loading vector…
Sample 6

Input

Loading vector…

Output

Loading vector…
Sample 7

Input

Loading vector…

Output

Loading vector…
Sample 8

Input

Loading vector…

Output

Loading vector…
Batch 3
Sample 1

Input

Loading vector…

Output

Loading vector…
Sample 2

Input

Loading vector…

Output

Loading vector…
Sample 3

Input

Loading vector…

Output

Loading vector…
Sample 4

Input

Loading vector…

Output

Loading vector…
Sample 5

Input

Loading vector…

Output

Loading vector…
Sample 6

Input

Loading vector…

Output

Loading vector…
Sample 7

Input

Loading vector…

Output

Loading vector…
Sample 8

Input

Loading vector…

Output

Loading vector…
Batch 4
Sample 1

Input

Loading vector…

Output

Loading vector…
Sample 2

Input

Loading vector…

Output

Loading vector…
Sample 3

Input

Loading vector…

Output

Loading vector…
Sample 4

Input

Loading vector…

Output

Loading vector…
Sample 5

Input

Loading vector…

Output

Loading vector…
Sample 6

Input

Loading vector…

Output

Loading vector…
Sample 7

Input

Loading vector…

Output

Loading vector…
Sample 8

Input

Loading vector…

Output

Loading vector…
Batch 5
Sample 1

Input

Loading vector…

Output

Loading vector…
Sample 2

Input

Loading vector…

Output

Loading vector…
Sample 3

Input

Loading vector…

Output

Loading vector…
Sample 4

Input

Loading vector…

Output

Loading vector…
Sample 5

Input

Loading vector…

Output

Loading vector…
Sample 6

Input

Loading vector…

Output

Loading vector…
Sample 7

Input

Loading vector…

Output

Loading vector…
Sample 8

Input

Loading vector…

Output

Loading vector…