Knowledge Tree

🤖 APIs

5 min read

OpenRouter

  • Integration Info:
    • Base URL: https://openrouter.ai/api/v1
    • SDK: @openrouter/sdk (js/ts), openrouter (python)
    • OpenAI SDK compatibility: Pass baseURL and API key. Optional headers: HTTP-Referer, X-Title.
  • Configuration:
    • Max Fallback Models: 3 (Exceeding causes 400 errors).
    • Deduplication: Client must deduplicate variants (e.g., model vs model:online).
    • Model variants (suffixes): :free (free tier), :extended (context window), :exacto (specific provider), :thinking (CoT), :online (web search), :nitro (inference speed).
    • Provider routing properties: order (preference array), allow_fallbacks (boolean), require_parameters (boolean), data_collection (allow or deny), quantizations (precision requirements).
    • Plugins:
      • Web search: plugins: [{ id: ‘web’, max_results: number }]
      • Response healing (auto-fixes malformed JSON): plugins: [{ id: ‘response-healing’ }]
  • Model Fallbacks:
    • Fallbacks triggered by rate limiting, provider downtime, context length errors, moderation flags.
  • Request Parameters:
    • Supported parameters: model, messages, stream, max_tokens, temperature, top_p, top_k, frequency_penalty, presence_penalty, stop, seed, tools, tool_choice, response_format.
  • Streaming Details:
    • SSE cancellation: Supported using AbortController.
    • Cancellation supported: OpenAI, Anthropic, Fireworks, DeepInfra, Together, Cohere, DeepSeek.
    • Cancellation not supported: Groq, Google, AWS Bedrock, Mistral, Perplexity.
  • Tool Calling:
    • Tool choice modes: auto, none, or explicit function object.
    • Sequential execution: Disable parallel execution with parallel_tool_calls: false.
  • Structured Outputs:
    • Schema configuration: Enforce schema via response_format: { type: ‘json_schema’, json_schema: { name, strict: true, schema } }. strict must be true.
    • Supported: OpenAI GPT-4o+, Gemini, Claude Sonnet 4.5+, Fireworks.
  • Cost Optimization:
    • Prompt caching: Long system prompts cached automatically (check usage.prompt_tokens_details.cached_tokens).
    • Fallbacks: Fallback to cheaper models (e.g. models: [‘expensive/model’, ‘cheaper/fallback’]).
    • Credit limits: Set credit limits per API key in dashboard or via API key management.
    • Free variants: Use :free variants for development (e.g., meta-llama/llama-3.3-70b-instruct:free).
    • Generation monitoring: Query endpoint GET https://openrouter.ai/api/v1/generation?id=${genId} for metadata and usage.
  • Security:
    • Env variables, credit limits, guardrails, Zero Data Retention (ZDR) for sensitive data.
  • Basic Request Examples:
    • Curl: 
      curl -X POST https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "meta-llama/llama-3.3-70b-instruct:free", "messages": [{"role": "user", "content": "hello"}]}'
    • JavaScript SDK: 
      import { OpenRouter } from '@openrouter/sdk';
const openRouter = new OpenRouter({ apiKey: process.env.OPENROUTER_API_KEY });
const completion = await openRouter.chat.send({
  model: 'anthropic/claude-sonnet-4',
  messages: [{ role: 'user', content: 'Hello!' }]
});
    • Streaming Cancellation Example: 
      const controller = new AbortController();
const stream = await openRouter.chat.send({
  model: 'openai/gpt-4o',
  messages: messages,
  stream: true
}, { signal: controller.signal });
controller.abort();

Groq

  • Integration Info:
  • Production Models:
    • llama-3.3-70b-versatile: 128k context, general purpose.
    • llama-3.3-70b-specdec: 8k context, ultra-fast speculative decoding.
    • llama-3.1-8b-instant: 128k context, rapid response.
    • llama3-70b-8192: 8k context, dialogue/generation.
    • llama3-8b-8192: 8k context, fast/cost-effective.
    • qwen-2.5-32b: 128k context, high creative writing performance.
    • qwen-2.5-coder-32b: 128k context, code generation.
    • qwen-qwq-32b: 128k context, complex reasoning.
    • gemma2-9b-it: 8k context, instruction following.
    • mistral-saba-24b: 32k context, optimized for Arabic/Farsi/Hebrew/Urdu.
  • Preview Models:
    • llama-4-scout-17b-16e-instruct: 128k context, multimodal text and image, MoE.
    • llama-4-maverick-17b-128e-instruct: 128k context, multimodal visual reasoning.
    • deepseek-r1-distill-llama-70b: 128k context, math/logic reasoning.
    • deepseek-r1-distill-qwen-32b: 128k context, distilled reasoning.
    • qwen3-32b: 128k context, thinking and non-thinking modes.
    • llama-3.2-1b-preview / llama-3.2-3b-preview: 128k context, lightweight.
  • Safety & Moderation:
    • Content moderation: llama-guard-3-8b, llama-guard-4-12b (multimodal).
    • Prompt injection: llama-prompt-guard-2-86m, llama-prompt-guard-2-22m (lightweight).
  • Audio Models:
    • Transcriptions (STT): whisper-large-v3, whisper-large-v3-turbo, distil-whisper-large-v3-en (English-optimized). Formats supported: mp3, mp4, mpeg, mpga, m4a, wav, webm. Max file size: 25MB.
    • Speech (TTS): playai-tts, playai-tts-arabic.
  • Vision (Multimodal):
    • Capability: Supports image understanding inputs (e.g. Llama 4 Scout, Maverick).
    • JSON payload structure: Use image_url properties inside messages content array.
  • Compound AI Systems:
    • Systems: compound (Llama 3.3 70B + GPT-OSS 120B), compound-mini.
    • Built-in tools: Web search, Wolfram Alpha, code execution, browser automation.
  • Rate Limits & Pricing:
    • Free/Developer Tiers: 30 RPM, 6,000 TPM.
    • Pricing: Token-based (pay-per-token with developer tier).
  • Request Parameters:
    • Supported parameters: model, messages, max_tokens, temperature, top_p, stream, stop, seed, tools, tool_choice, response_format.
  • Basic Request Examples:
    • Curl: 
      curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-3.3-70b-versatile", "messages": [{"role": "user", "content": "hello"}]}'
    • Python SDK: 
      from groq import Groq
client = Groq(api_key=GROQ_API_KEY)
response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)
    • Speech transcription (STT) bash example: 
      curl https://api.groq.com/openai/v1/audio/transcriptions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -F file=@audio.mp3 \
  -F model=whisper-large-v3
  • Tool Calling Example: 
    const response = await fetch('https://api.groq.com/openai/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${GROQ_API_KEY}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    model: 'llama-3.3-70b-versatile',
    messages: [{ role: 'user', content: 'What is 25 * 4?' }],
    tools: [{
      type: 'function',
      function: {
        name: 'calculate',
        description: 'Perform mathematical calculations',
        parameters: {
          type: 'object',
          properties: {
            expression: { type: 'string', description: 'Math expression' }
          },
          required: ['expression']
        }
      }
    }],
    tool_choice: 'auto'
  })
});
  • Streaming Example: 
    const response = await fetch('https://api.groq.com/openai/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${GROQ_API_KEY}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    model: 'llama-3.3-70b-versatile',
    messages: [{ role: 'user', content: 'Tell me a story' }],
    stream: true
  })
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  const chunk = decoder.decode(value);
  const lines = chunk.split('\n').filter(line => line.startsWith('data: '));
  for (const line of lines) {
    if (line === 'data: [DONE]') continue;
    const json = JSON.parse(line.slice(6));
    process.stdout.write(json.choices[0]?.delta?.content || '');
  }
}

Graph View

Backlinks