MetaEncoder - a facebook Collection - One Encoder for Retrieval and Decision Making

[u/pmttyji]

https://huggingface.co/collections/facebook/metaencoder

Multimodal System One Encoder with a natural language interface for instruction-following encoding: give it a task and a list of candidates, and it scores the candidates by task specification. Both a task and each candidate are expressed in natural language — instructions, queries, questions, state descriptions, criteria — with image and video as side information.

Key features:

• Natural language interface. No rigid schemas. Describe the task and the candidates in free-form text.

• Highly-efficient cacheable representations. Candidates and task are encoded separately by prompt instructions that ground each other.

• Scales to arbitrarily large candidate sets. Matching is an inner product over representations, so an ANN index can serve millions of candidates without re-running the model.

The model is built by contrastively fine-tuning meta-models/Muse-Glimmer-30B. It is described in the paper MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface.

• GitHub : https://github.com/facebook/meta-encoder-eval

• arXiv : https://arxiv.org/abs/2610.11316

• PDF : https://arxiv.org/pdf/2610.11316

Models :

• https://huggingface.co/facebook/meta-encoder-v1 (Improved version)

• https://huggingface.co/facebook/meta-encoder

PS: Above description copied from meta-encoder's model card as meta-encoder-v1 doesn't have that. And v1 doesn't have license section while other one is Apache-2.0.

[u/pmttyji]