vrr/heretic
2
0
Fork 0
mirror of https://github.com/p-e-w/heretic.git synced 2026-10-15 14:01:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: set batch size on HFLM object
Some checks failed
CI / Check and build (Python 3.10) (push) Has been cancelled
Details
CI / Check and build (Python 3.11) (push) Has been cancelled
Details
CI / Check and build (Python 3.12) (push) Has been cancelled
Details
CI / Check and build (Python 3.13) (push) Has been cancelled
Details

Browse source
This commit is contained in:
Philipp Emanuel Weidmann 2026-04-01 14:27:43 +05:30
parent 96c7a7d98a
commit 887d43a8d9
No known key found for this signature in database
GPG key ID: CE6A7756AD520E0F
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/heretic/main.py
View file

@ -966,6 +966,7 @@ def run():
hflm = HFLM(
pretrained=model.model, # ty:ignore[invalid-argument-type]
tokenizer=model.tokenizer, # ty:ignore[invalid-argument-type]
batch_size="auto",
)
table = Table()
@ -989,7 +990,6 @@ def run():
results = lm_eval.simple_evaluate(
model=hflm,
tasks=[benchmark.task],
batch_size="auto",
)
return results["results"][benchmark.task]