Multi-Modal AI: Vision and Image Understanding

Multi-modal AI models can see, understand, and reason about images. From analyzing screenshots and charts to reading documents and identifying objects, vision capabilities unlock entirely new categories of AI applications. This guide covers vision-enabled Chinese AI models and how to use them through TokenEase.

Chinese Multi-Modal Model Landscape

Model	Vision Capabilities	Image Size	Best For
deepseek DeepSeek-V4-VL	Image understanding, OCR, charts	Up to 4K	Complex visual reasoning
zhipu GLM-4V	Image description, document reading	Up to 8K	Chinese document analysis
qwen Qwen-VL-Plus	Object detection, visual QA	Up to 4K	General vision tasks
kimi Kimi-VL	Long-image understanding	Up to 16K	Multi-page documents

Image Input Format

All vision models use the same OpenAI-compatible format for images:

import base64

def encode_image(image_path):
    with open(image_path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

# Build message with image
messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image in detail."},
            {
                "type": "image_url",
                "image_url": {
                    "url": f"data:image/jpeg;base64,{encode_image('photo.jpg')}"
                }
            }
        ]
    }
]

Use Case 1: Image Description and Captioning

import requests

API_KEY = "your-tokenease-api-key"
BASE_URL = "https://tokenease.io/v1"

def describe_image(image_path, model="qwen"):
    base64_image = encode_image(image_path)
    
    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model,
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": "请详细描述这张图片，包括场景、人物、物体和氛围。用中文回答。"},
                        {
                            "type": "image_url",
                            "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
                        }
                    ]
                }
            ],
            "max_tokens": 500
        }
    )
    
    return response.json()["choices"][0]["message"]["content"]

# Usage
description = describe_image("beijing_street.jpg", model="zhipu")
print(description)

Use Case 2: OCR and Document Reading

Extract text from images, receipts, forms, and scanned documents:

def extract_text_from_image(image_path, model="qwen"):
    base64_image = encode_image(image_path)
    
    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model,
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": "请提取图片中的所有文字，保持原有格式和布局。只输出文字内容，不要添加解释。"},
                        {
                            "type": "image_url",
                            "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
                        }
                    ]
                }
            ],
            "max_tokens": 1000
        }
    )
    
    return response.json()["choices"][0]["message"]["content"]

# Extract text from invoice, receipt, or form
invoice_text = extract_text_from_image("invoice.jpg", model="deepseek")
print(invoice_text)

Use Case 3: Chart and Graph Analysis

Let AI read and interpret data visualizations:

def analyze_chart(image_path, model="deepseek"):
    base64_image = encode_image(image_path)
    
    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model,
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": """分析这张图表，请提供：
1. 图表类型和主题
2. 关键数据点
3. 趋势分析
4. 数据洞察和建议

用中文回答。"""},
                        {
                            "type": "image_url",
                            "image_url": {"url": f"data:image/png;base64,{base64_image}"}
                        }
                    ]
                }
            ],
            "max_tokens": 800
        }
    )
    
    return response.json()["choices"][0]["message"]["content"]

# Analyze sales chart, stock chart, or survey results
analysis = analyze_chart("sales_chart.png", model="deepseek")

Use Case 4: Visual Question Answering

def visual_qa(image_path, question, model="qwen"):
    base64_image = encode_image(image_path)
    
    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model,
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": question},
                        {
                            "type": "image_url",
                            "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
                        }
                    ]
                }
            ],
            "max_tokens": 300
        }
    )
    
    return response.json()["choices"][0]["message"]["content"]

# Examples
print(visual_qa("product.jpg", "这个产品的颜色是什么？"))
print(visual_qa("menu.jpg", "这份菜单里有哪些川菜？"))
print(visual_qa("screenshot.jpg", "这个页面有哪些按钮？"))

Use Case 5: Multi-Image Comparison

Send multiple images in a single request for comparison:

def compare_images(image_paths, model="deepseek"):
    content = [{"type": "text", "text": "比较以下图片，说明它们的相似之处和不同之处。用中文回答。"}]
    
    for path in image_paths:
        base64_image = encode_image(path)
        content.append({
            "type": "image_url",
            "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
        })
    
    response = requests.post(
        f"{BASE_URL}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": model,
            "messages": [{"role": "user", "content": content}],
            "max_tokens": 600
        }
    )
    
    return response.json()["choices"][0]["message"]["content"]

# Compare product variations or design iterations
comparison = compare_images(["design_v1.jpg", "design_v2.jpg"], model="deepseek")

Image Preprocessing for Better Results

• Resolution: Most models work best with 1024x1024 or smaller. Very large images get downscaled.
• Format: JPEG for photos, PNG for screenshots and diagrams with text.
• Cropping: Crop to the relevant region. Extra background noise confuses the model.
• Contrast: Ensure text is clearly visible. Low-contrast images reduce OCR accuracy.
• File size: Keep base64 strings under 5MB to avoid request size limits.

Handling Image URLs vs. Base64

You can also pass image URLs instead of base64:

messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {
                "type": "image_url",
                "image_url": {"url": "https://example.com/image.jpg"}
            }
        ]
    }
]

# Note: The URL must be publicly accessible.
# For private images, use base64 encoding.

Cost Considerations

Model	Image Input	Text Output	Typical Cost/Image
Qwen-VL-Plus	~$0.001-0.003	~$0.001/1K tokens	~$0.002-0.005
GLM-4V	~$0.002-0.005	~$0.002/1K tokens	~$0.004-0.010
DeepSeek-V4-VL	~$0.002-0.004	~$0.002/1K tokens	~$0.004-0.008

Production Patterns

Batch Image Processing

from concurrent.futures import ThreadPoolExecutor

def process_image_batch(image_paths, prompt, model="qwen", max_workers=5):
    def process_one(path):
        return {
            "path": path,
            "result": visual_qa(path, prompt, model)
        }
    
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        results = list(executor.map(process_one, image_paths))
    
    return results

# Process 100 product images
results = process_image_batch(
    product_images,
    "提取产品名称、价格和主要特性",
    model="qwen",
    max_workers=5
)

Conclusion

Vision-enabled AI opens doors to applications that were impossible with text-only models. Invoice processing, quality inspection, visual search, accessibility tools — the use cases are endless.

Start with simple image description and OCR. Progress to chart analysis and visual QA. For production, preprocess images (resize, crop, ensure contrast) and batch process for efficiency. With TokenEase, switching between Qwen-VL, GLM-4V, and DeepSeek-V4-VL is a one-line change.