---
title: README
emoji: 📈
colorFrom: yellow
colorTo: green
sdk: static
pinned: false
---

Edit this `README.md` markdown file to author your organization card.
<div class="grid lg:grid-cols-2 gap-x-4 gap-y-7">
	<p class="lg:col-span-3">
	  Welcome to CARROT-LLM-Routing! For a given desired trade off between performance and cost, 
      CARROT makes it easy to pick the best model among a set of 13 LLMs for any query. Below you may read the CARROT paper, replicate the training process of CARROT, or see how to utilize CARROT out of the box for routing.
	</p>
	<a href="https://arxiv.org/" class="block overflow-hidden group">
		<div
			class="w-40 h-39 object-cover mb-2 rounded-lg flex items-center justify-center bg-[#ECFAFF]"
		>
			<img alt="" src="fmselect_gpt4o_comparison.png" class="w-40" />
		</div>
		<div class="underline">Read the paper</div>
	</a>
	<a
		href="https://github.com/somerstep"
		class="block overflow-hidden"
	>
		<div
			class="w-40 h-39 object-cover mb-2 rounded-lg flex items-center justify-center bg-[#ECFAFF]"
		>
			<img alt="" src="logo.png" class="w-40" />
		</div>
		<div class="underline">Train CARROT</div>
>

As is, CARROT supports routing to the following collection of large language models.

| Model                      | Input Token Cost ($ per 1M tokens) | Output Token Cost ($ per 1M tokens) |
|----------------------------|----------------------------------|-----------------------------------|
| claude-3-5-sonnet-v1       | 3                                | 15                                |
| titan-text-premier-v1      | 0.5                              | 1.5                               |
| openai-gpt-4o             | 2.5                              | 10                                |
| openai-gpt-4o-mini        | 0.15                             | 0.6                               |
| granite-3-2b-instruct     | 0.1                              | 0.1                               |
| granite-3-8b-instruct     | 0.2                              | 0.2                               |
| llama-3-1-70b-instruct    | 0.9                              | 0.9                               |
| llama-3-1-8b-instruct     | 0.2                              | 0.2                               |
| llama-3-2-1b-instruct     | 0.06                             | 0.06                              |
| llama-3-2-3b-instruct     | 0.06                             | 0.06                              |
| llama-3-3-70b-instruct    | 0.9                              | 0.9                               |
| mixtral-8x7b-instruct     | 0.6                              | 0.6                               |
| llama-3-405b-instruct     | 3.5                              | 3.5                               |