New benchmark UNIKIE-BENCH evaluates large multimodal models for document information extraction

Researchers have introduced UNIKIE-BENCH, a new benchmark designed to systematically evaluate the performance of Large Multimodal Models (LMMs) in extracting key information from visual documents. The benchmark features two tracks: one for constrained-category KIE with predefined schemas and another for open-category KIE. Experiments using 15 state-of-the-art LMMs highlighted significant performance drops when dealing with varied schemas, long-tail information, and complex layouts, indicating ongoing challenges in accuracy and reasoning for LMMs in this domain. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Provides a standardized evaluation framework for LMMs in document information extraction, highlighting current limitations.

RANK_REASON This is a research paper introducing a new benchmark for evaluating LMMs.

Read on arXiv cs.CV →

• UNIKIE-BENCH
• Large Multimodal Models
• LMMs
• Zhenghao Liu

• paper
• other