title: Vision Understanding (Multimodal: Image / Video)

Vision Understanding (Multimodal: Image / Video)

Image and video chat use the same endpoint as text chat, /v1/chat/completions: change content from a string to an array mixing text with images / video. For available models and prices see the Model Plaza (vision models usually have vl / vision / omni in the name); below we use qwen3-vl-flash.

How to Pass Media

Both images and videos support two ways of passing the media; the material type is set via type: image_url for images, video_url for videos (only some models support it):

Materialtype valuePublic URLBase64
Imageimage_url{ "url": "https://example.com/photo.jpg" }{ "url": "data:image/jpeg;base64,/9j/4AA..." }
Videovideo_url{ "url": "https://example.com/clip.mp4" }{ "url": "data:video/mp4;base64,AAAAIG..." }

So a multimodal message's content array looks like this:

{
  "role": "user",
  "content": [
    { "type": "text", "text": "Your question" },
    {
      "type": "image_url",
      "image_url": { "url": "https://example.com/photo.jpg" }
    },
    {
      "type": "video_url",
      "video_url": { "url": "https://example.com/clip.mp4" }
    }
  ]
}
  • Public URL: simplest when the file is already online; prefer this
  • Base64 data URL: for local files, read and encode as data:<mime>;base64,<data>; the request body gets large, especially for video

Parameter Summary

Vision models differ in which parameters they accept. Summary below (✅ = universal, ⚠️ = only some models):

ParameterNotesAvailability
model / messages / streamSame as Text Chat, see its request fields table✅ All models
temperature / max_tokensSampling temperature / reply length cap✅ Almost all models
content array + type: "text"Text fragment✅ All vision models
type: "image_url" (URL or Base64)Image input✅ All vision models
type: "video_url" (URL or Base64)Video input⚠️ Only some models
thinking (e.g. "thinking": {"type": "enabled"})Enables reasoning; the reply includes the chain of thought⚠️ Only some models; syntax varies by model
reasoning_effort (e.g. "low" / "medium" / "high")Reasoning depth level; trades latency for quality⚠️ Only some models; allowed values vary
IMPORTANT

Vision model parameters are not standardized: for example doubao-seed-2-0-pro-260215, deepseek-v4-flash-vision-exp, and Kimi K3 differ in whether they accept video input and whether they have thinking / reasoning_effort parameters and which values they allow. Unsupported parameters may be ignored or rejected with an error. Which model accepts which parameters — check the Model Plaza model details and your own testing; just run the examples below against different model names to confirm.

Image Understanding Example

cURL
Python (openai)
Node.js (openai)
curl https://api-model.shuntianchuang.cn/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-token" \
  -d '{
    "model": "qwen3-vl-flash",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe this image in one sentence" },
          { "type": "image_url", "image_url": { "url": "https://example.com/photo.jpg" } }
        ]
      }
    ]
  }'

For Base64, just swap the url for a data URL:

# Encode a local image and inline it (macOS: base64 -i, Linux: base64 -w 0)
IMG=$(base64 -i photo.jpg)
curl https://api-model.shuntianchuang.cn/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-token" \
  -d '{
    "model": "qwen3-vl-flash",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe this image in one sentence" },
          { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,'" "$IMG" '" } }
        ]
      }
    ]
  }'

Video Understanding Example

For video, switch type to video_url — everything else stays the same. Note that only some models accept video input: an error or silent ignore means the current model doesn't support it; switch to a video-capable model and retry:

cURL
Python (openai)
Node.js (openai)
curl https://api-model.shuntianchuang.cn/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-your-token" \
  -d '{
    "model": "doubao-seed-2-0-pro-260215",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Summarize this video" },
          { "type": "video_url", "video_url": { "url": "https://example.com/clip.mp4" } }
        ]
      }
    ]
  }'
TIP

Images and videos both consume a lot of input tokens (proportional to resolution / duration). In high-volume scenarios, keep media sizes down to save cost.