Task Caching Demo

Demonstrates the effect of using a dynamically generated cheatsheet (Task Caching) on model inference. Uses SambaNova API via litellm.

SambaNova Model Name

Enter the SambaNova model name (e.g., sambanova/DeepSeek-R1-Distill-Llama-70B). Ensure the 'sambanova/' prefix if required by litellm configuration.

SambaNova API Key

Please Enter your SambaNova API Key, otherwise by default will use Changran's key, but RPM is low

Paste your training data below, one example per line. This data will be used to build a cumulative cheatsheet. The process may take some time depending on the number of examples.

Training Data

Generated Cheatsheet

Important: Ensure SAMBANOVA_API_KEY is set as a secret in your Hugging Face Space or as an environment variable if running locally. SAMBANOVA_BASE_URL is set to https://api.sambanova.ai/v1 by default if not found in environment.