[CODE] safety-research/safety-tooling: v1.0.0

Initial release What's Changed add all code by @jplhughes in https://github.com/safety-research/safety-tooling/pull/1 add base url by @jplhughes in https://github.com/safety-research/safety-tooling/pull/2 rename core folder by @jplhughes in https://github.com/safety-research/safety-tooling/pull/3 update readme by @jplhughes in https://github.com/safety-research/safety-tooling/pull/6 add openai batch api by @jplhughes in https://github.com/safety-research/safety-tooling/pull/8 Add InferenceAPI Tests and CI Workflow by @devin-ai-integration in https://github.com/safety-research/safety-tooling/pull/5 feat: add new claude-3-5 models from 20241022 by @dpaleka in https://github.com/safety-research/safety-tooling/pull/9 docs: update test running instructions and add submodule docs by @devin-ai-integration in https://github.com/safety-research/safety-tooling/pull/12 Together api by @Wanff in https://github.com/safety-research/safety-tooling/pull/13 Together api finetuning support + refactoring finetuning api by @Wanff in https://github.com/safety-research/safety-tooling/pull/16 Feat/finetune api save id and model by @Wanff in https://github.com/safety-research/safety-tooling/pull/22 Feat/finetune api save id and model by @Wanff in https://github.com/safety-research/safety-tooling/pull/23 Support Comments in SECRETS by @devin-ai-integration in https://github.com/safety-research/safety-tooling/pull/15 Add SAFETYTOOLING_PRINT_PROMPTS by @dpaleka in https://github.com/safety-research/safety-tooling/pull/25 Abhay/misc by @abhay-sheshadri in https://github.com/safety-research/safety-tooling/pull/26 updates to together ai ft script by @jplhughes in https://github.com/safety-research/safety-tooling/pull/17 raise error for hf format by @jplhughes in https://github.com/safety-research/safety-tooling/pull/27 Add redis caching by @dpaleka in https://github.com/safety-research/safety-tooling/pull/24 Added support for vllm hosted models by @abhay-sheshadri in https://github.com/safety-research/safety-tooling/pull/28 wandb & vllm updates by @jplhughes in https://github.com/safety-research/safety-tooling/pull/29 fix: add logprobs of the generated tokens (not top_logprobs) in vllm api by @dpaleka in https://github.com/safety-research/safety-tooling/pull/30 BatchInferenceAPI by @dpaleka in https://github.com/safety-research/safety-tooling/pull/32 Chunking in Batch API by @dpaleka in https://github.com/safety-research/safety-tooling/pull/33 Added support for o1-2024-12-17 and structured outputs by @shahofblah in https://github.com/safety-research/safety-tooling/pull/35 Add new models and equivalences by @devin-ai-integration in https://github.com/safety-research/safety-tooling/pull/37 feat: change anthropic batch polling to hours by @dpaleka in https://github.com/safety-research/safety-tooling/pull/36 Dpo support for oai by @Wanff in https://github.com/safety-research/safety-tooling/pull/42 vllm small fixes by @jplhughes in https://github.com/safety-research/safety-tooling/pull/34 Add microsecond precision to log file name by @dpaleka in https://github.com/safety-research/safety-tooling/pull/46 logprob updates to together ai inference by @jplhughes in https://github.com/safety-research/safety-tooling/pull/44 support multiple completions for VLLM by @andyrdt in https://github.com/safety-research/safety-tooling/pull/47 timeout, new 4o, text normalization by @jplhughes in https://github.com/safety-research/safety-tooling/pull/48 Johnh/bunch of small changes by @jplhughes in https://github.com/safety-research/safety-tooling/pull/49 fix: add redis support for caching by @dpaleka in https://github.com/safety-research/safety-tooling/pull/38 Abhay/support for new models by @abhay-sheshadri in https://github.com/safety-research/safety-tooling/pull/50 fix(readme): update finetuning command path by @JCocola in https://github.com/safety-research/safety-tooling/pull/52 llama 3b and deepseek-r1 on together by @dpaleka in https://github.com/safety-research/safety-tooling/pull/51 improve logging by @Revys17 in https://github.com/safety-research/safety-tooling/pull/56 Feature/non strict kwargs by @jplhughes in https://github.com/safety-research/safety-tooling/pull/57 non strict models by @jplhughes in https://github.com/safety-research/safety-tooling/pull/58 Feature/simplify reqs by @jplhughes in https://github.com/safety-research/safety-tooling/pull/59 move to .env file by @jplhughes in https://github.com/safety-research/safety-tooling/pull/60 feat: add special handling for seed parameter when calling Anthropic models by @dpaleka in https://github.com/safety-research/safety-tooling/pull/63 fix: vars from .env should override globally set vars, like SECRETS did by @dpaleka in https://github.com/safety-research/safety-tooling/pull/64 fix bitrot in README.md example by @g-w1 in https://github.com/safety-research/safety-tooling/pull/66 env fix to look in submodule by @jplhughes in https://github.com/safety-research/safety-tooling/pull/68 turn openrouter error messages into openai ones so you can use openrouter with openai api by @g-w1 in https://github.com/safety-research/safety-tooling/pull/67 bump deps by @jplhughes in https://github.com/safety-research/safety-tooling/pull/71 vllm automatic deployment using hf models by @jplhughes in https://github.com/safety-research/safety-tooling/pull/72 Create LICENSE by @jplhughes in https://github.com/safety-research/safety-tooling/pull/74 Allowing prefilling on DeepSeek API by @aryopg in https://github.com/safety-research/safety-tooling/pull/75 Abhay/add together prefill format by @abhay-sheshadri in https://github.com/safety-research/safety-tooling/pull/73 Fix venv python version and pre-commit config by @EbTech in https://github.com/safety-research/safety-tooling/pull/76 Adds a slightly more robust version of from_almj_prompt_format by @JacksonKaunismaa in https://github.com/safety-research/safety-tooling/pull/70 Ran pre-commit autoupdate by @EbTech in https://github.com/safety-research/safety-tooling/pull/81 OpenRouter integration and tests by @dpaleka in https://github.com/safety-research/safety-tooling/pull/78 oai fix by @jplhughes in https://github.com/safety-research/safety-tooling/pull/82 open router retries by @jplhughes in https://github.com/safety-research/safety-tooling/pull/83 fix stop reason bug by @jplhughes in https://github.com/safety-research/safety-tooling/pull/84 Add logprob support for DeepSeek by switching to the OpenAI API by @kxcloud in https://github.com/safety-research/safety-tooling/pull/80 Bugfix: allow OpenAI completions by @kxcloud in https://github.com/safety-research/safety-tooling/pull/86 Fix backoff logic in anthropic API retry code by @Revys17 in https://github.com/safety-research/safety-tooling/pull/88 robustify anthropic and openai inference to various errors by @g-w1 in https://github.com/safety-research/safety-tooling/pull/77 add gopher prompt so that you can prompt base models in a chat format by @g-w1 in https://github.com/safety-research/safety-tooling/pull/69 Add GPT 4.1 support by @EbTech in https://github.com/safety-research/safety-tooling/pull/90 make Together models not error out by @dpaleka in https://github.com/safety-research/safety-tooling/pull/87 Log expenditure using model-specific token counts by @EbTech in https://github.com/safety-research/safety-tooling/pull/92 Fix bug in Anthropic Reasoning Mode by @aryopg in https://github.com/safety-research/safety-tooling/pull/91 Fix broken check.py by importing the right utils by @EbTech in https://github.com/safety-research/safety-tooling/pull/93 Another utils.count_tokens miscall by @EbTech in https://github.com/safety-research/safety-tooling/pull/94 fix open router choices None bug by @jplhughes in https://github.com/safety-research/safety-tooling/pull/97 New Contributors @jplhughes made their first contribution in https://github.com/safety-research/safety-tooling/pull/1 @devin-ai-integration made their first contribution in https://github.com/safety-research/safety-tooling/pull/5 @dpaleka made their first contribution in https://github.com/safety-research/safety-tooling/pull/9 @Wanff made their first contribution in https://github.com/safety-research/safety-tooling/pull/13 @abhay-sheshadri made their first contribution in https://github.com/safety-research/safety-tooling/pull/26 @shahofblah made their first contribution in https://github.com/safety-research/safety-tooling/pull/35 @andyrdt made their first contribution in https://github.com/safety-research/safety-tooling/pull/47 @JCocola made their first contribution in https://github.com/safety-research/safety-tooling/pull/52 @Revys17 made their first contribution in https://github.com/safety-research/safety-tooling/pull/56 @g-w1 made their first contribution in https://github.com/safety-research/safety-tooling/pull/66 @aryopg made their first contribution in https://github.com/safety-research/safety-tooling/pull/75 @EbTech made their first contribution in https://github.com/safety-research/safety-tooling/pull/76 @JacksonKaunismaa made their first contribution in https://github.com/safety-research/safety-tooling/pull/70 @kxcloud made their first contribution in https://github.com/safety-research/safety-tooling/pull/80 Full Changelog: https://github.com/safety-research/safety-tooling/commits/v1.0.0

John Hughes, Daniel Paleka, Aram Ebtekar, Rowan Wang, Jacob G-W, Abhay Sheshadri, Aryo Pradipta Gema, Caleb Larson, kxcloud, Andy Arditi, JacksonKaunismaa, Jorio Cocola, & Shubhorup Biswas. (2025). safety-research/safety-tooling: v1.0.0 (v1.0.0). Zenodo. https://doi.org/10.5281/zenodo.15363603