Vlms, or visual language models, rely on visual encoders to process images for language understanding. The interaction between visual and language brains is facilitated through a projection layer, enhancing reasoning capabilities with larger models. Tiling images for processing can significantly impact performance due to increased memory requirements.
