Hugging Face's logo
Activity Feed

AI & ML interests

VLM, Geo Reasoning, Visual Retriving

Recent Activity
etzion  updated a Space 1 day ago
TLVLM/README
etzion  updated a Space 1 day ago
TLVLM/README
etzion  updated a collection 2 months ago
CLIPs
View all activity

Organization Card
Community About org cards

TLV R&D VLMs for Image Retrieving and Visual Reasoning

Vision-Language Retrieving Models

RGB Retrieving models

Model Name Model Type Base Model Training Set Owner Link Freezed Parameters Channels
ImiClip CLIP openai/clip-vit-base-patch32 DM Etzion TLVLM/ImiClip Vision Encoder RGB
ImiClip_v2 CLIP openai/clip-vit-base-patch32 DM + RSICD Etzion TLVLM/ImiClip_v2 Vision Encoder RGB
ImiClip_v3 CLIP openai/clip-vit-base-patch32 DM + RSICD Etzion TLVLM/ImiClip_v3 ❌ RGB
ImiGlip SigLIP google/siglip-so400m-patch14-384 DM Etzion TLVLM/ImiGlip Vision Encoder RGB
ImiGlip_V2 SigLIP google/siglip-so400m-patch14-384 DM + RSICD Etzion TLVLM/ImiGlip_V2 Vision Encoder RGB
ImiGlip_V3 SigLIP google/siglip-so400m-patch14-384 DM + RSICD Etzion TLVLM/ImiGlip_V3 ❌ RGB
ImiGlip2 SigLIP2 google/siglip2-so400m-patch14-384 DM + RSICD Etzion TLVLM/ImiGlip2 Both Encoders + Logits RGB
ImiGlip2n SigLIP2 google/siglip2-so400m-patch16-naflex DM + RSICD Etzion TLVLM/ImiGlip2n Both Encoders + Logits RGB
ImiGem_V1 EmbeddingGemma2 google/embeddinggemma-2 DM + RSICD Etzion TLVLM/ImiGem_V1 All (LoRA: R=15, α=32) RGB + Gray
ImiGem_V2 EmbeddingGemma2 google/embeddinggemma-2 DM + RSICD Etzion TLVLM/ImiGem_V2 ❌ RGB + Gray

Model Recall@K Performance for DM + RSICD (validation set):

Here we examine the performance of the finetuned models, as well there base models.

Model Recall@1 Recall@5 Recall@10 Recall@50 Recall@100 Recall@500 Recall@1000 Recall@9781
CLIP 1.0 3.8 5.8 17.2 29.2 67.6 83.4 100
ImiCLIP 5.5 16.7 26.4 53.5 68.7 94.5 98.3 100
ImiGlip2n_V1 9.8 24.6 35.6 62.7 75.4 96.0 99.0 100
ImiClip_V2 10.1 25.2 35.8 64.5 79.4 97.3 99.1 100
SIGLIP 10.1 24.4 33.2 57.6 68.0 91.6 96.9 100
ImiGlip2_V1 14.3 34.8 45.8 71.5 83.0 97.4 99.6 100
ImiClip_V3 15.4 37.0 47.4 75.4 86.0 99.1 99.9 100
ImiGLIP 18.5 42.8 53.6 76.4 86.5 98.9 99.9 100
ImiGLIP_V2 22.0 47.6 59.4 83.9 90.8 99.5 100 100
ImiGLIP_V3 27.7 54.0 64.7 86.1 93.6 99.5 100 100
EmbeddingGemma2 32.0 56.9 64.4 86.4 91.0 99.0 99.6 100
ImiGem_V1 65.0 87.9 93.4 98.7 99.4 100 100 100
ImiGem_V2 65.9 89.7 94.2 98.6 99.5 99.9 100 100

As we can see, ImiGem is the clear winner, where V2 perform better for low K values

Grayscale Retrieving models

Model Name Model Type Base Model Training Set Owner Link Freezed Parameters Channels
HiGlip SigLIP google/siglip-so400m-patch14-384 HI Etzion TLVLM/HiGlip Both Encoders + Logits GRAY
HiGlip2 SigLIP google/siglip-so400m-patch14-384 HI Etzion TLVLM/HiGlip2 Text Encoder GRAY

Image Captioning & VQA Models
Model Name Model Type Base Model Training Set Owner Link Freezed Parameters
ImiBlip_V1 BLIP Salesforce/blip-image-captioning-base DM + RSICD (Captions Only) Etzion TLVLM/ImiBlip_V1 ❌
ImiBlipVQA_V1 BLIP Salesforce/blip-vqa-base DM (VQA Only) Etzion TLVLM/ImiBlipVQA_V1 ❌
ImiBlipVQA_V2 BLIP Salesforce/blip-vqa-base DM + RSICD (VQA Only) Etzion TLVLM/ImiBlipVQA_V2 ❌
ImiBlipVQA_V3 BLIP Salesforce/blip-vqa-base DM + RSICD (VQA + Captions) Etzion TLVLM/ImiBlipVQA_V3 ❌

Runtime
Model Type Base Model Time per Single Text Time per Single Image Time per 10,000 Texts Time per 10,000 Images
CLIP openai/clip-vit-base-patch32 0.0129 0.0101 129.4 100.8
SigLIP (1+2) google/siglip-so400m-patch14-384 0.0578 0.0189 577.5 188.9
SigLIP2n google/siglip2-so400m-patch16-naflex 0.0257 0.0189 257.0 188.6

Important notes:

  • Time reported in seconds.
  • All the calculations conduct on NVIDIA A40 GPU
  • Avr. Text length: 633±93 Characters
  • Avr. Image size: 536×536 Pixels

Datasets

Collections

Here you can find the model Collections

models 0

None public yet

datasets 0

None public yet