Updated · Sources & corrections
In this guide
Quick answer: Choose a vision model by the document or image task you need to verify. Qwen2.5-VL, LLaVA and InternVL are candidates to evaluate, not interchangeable winners. Compare exact checkpoints on your own OCR, chart or screenshot examples, then check their licences and runtime requirements.
This is a documentation-based shortlist of established open-weight releases. It is not an exhaustive current leaderboard or an independent benchmark. “Open-weight” does not establish that every checkpoint has the same licence or deployment permissions.
A shortlist with explicit checkpoints
| Candidate | Reference | What to verify |
|---|---|---|
| Qwen2.5-VL-7B-Instruct | Publisher model card | Text extraction and structured document tasks; image processing settings and the exact checkpoint licence. |
| Qwen2.5-VL-72B-Instruct | Publisher model card | The custom Qwen licence and deployment resources. The larger checkpoint is not an automatic accuracy improvement for every input. |
| LLaVA through Ollama | Ollama registry entry | The selected tag, image support and whether its answers meet your captioning or visual-question task. |
| InternVL2_5-8B | Publisher model card | The documented runner, image tiling and preprocessing requirements before comparing results. |
The Qwen 7B and 72B cards identify different licences. Check the exact file you download, including any conversion or fine-tune; a family name is not a licence review. Vendor-reported evaluations can help choose candidates, but they do not measure your documents.
Match the evaluation to the task
| Task | Check | Common misleading success |
|---|---|---|
| Invoice OCR | Compare transcribed values with labeled originals, including dates, totals and missing fields. | Plausible text that changes a digit. |
| Chart questions | Check axis units, legend, series and the exact value needed. | A fluent answer using the wrong series. |
| Screenshot understanding | Verify the described control against the actual screenshot. | A confident description of an absent button. |
| Captioning | Score required objects and unsupported details using the same rubric across candidates. | More elaborate wording mistaken for greater accuracy. |
Include clean and difficult examples: rotated pages, small text, low contrast and multiple columns where relevant. Keep a held-out set so tuning your prompt does not become the whole evaluation. Record errors as well as successes. A few good outputs do not establish a general accuracy percentage.
Send an image using the documented interface
Ollama’s vision documentation distinguishes SDK inputs from REST inputs: the native REST API expects base64 image data in an images array. A filesystem path is not itself base64 data.
For the CLI, the official reference shows the image path in the prompt. This illustrative example uses a registry-listed vision tag and a local file you supply:
ollama pull llava:7b
ollama run llava:7b "Describe this image: ./photo.jpg"
Check the file exists and that your installed version supports the selected model. These commands were reviewed against documentation, not executed against a live model here. Do not add the unsupported --image flag from the earlier version of this guide.
Plan memory and data handling
Download size, model weights and peak runtime memory are different measurements. Image resolution, preprocessing, precision, context and concurrency can change memory use. Start with a candidate your runner supports, observe actual allocation and retain headroom; this page does not promise a fixed VRAM requirement or speed.
Local processing can reduce external data transfers, but check the entire application: a frontend, telemetry feature or cloud fallback can still send data elsewhere. Verify its configuration before using private documents. For setup choices, see LM Studio versus Ollama. Our context-allocation checks help distinguish requested settings from the running allocation.
Choose after checking results
Use the same inputs and scoring rules for each candidate. Record checkpoint, runner version, precision, image settings, prompt and failure cases. Choose based on the error types your workflow can tolerate. Keep verification for fields that drive payments, records or other consequential actions rather than trusting fluent output.
Sources and verification scope
Reviewed against the linked publisher cards, registry and Ollama documentation on 24 October 2026. No local model benchmark, memory measurement or accuracy score is claimed. This update removes unsupported rankings, broad privacy guarantees and fixed hardware claims, and corrects the vision CLI examples. The URL and original publication date are preserved. Read our sourcing and correction policy.
To build a native image request after selecting a checkpoint, use our Ollama REST image-payload guide to check file bytes and base64 locally before an optional request to your local model server.