sellGuided Research sellDFKI sellAugmented Vision sellVision-Language Models

Enhancing Zero-Shot Classification with CLIP-Driven Description Selection (Guided Research)

Nosheen Nazir , Muhammad Saif Ullah Khan

Abstract

This guided research focuses on prompt and description selection for CLIP-style zero-shot classification. The project analyzes how class descriptions affect recognition quality and investigates automatic selection strategies to improve robustness across classes.

Topic

Investigate CLIP-driven methods for selecting class descriptions/prompts that maximize zero-shot classification performance while maintaining stability across datasets and class granularity.

Tasks

1. Build a baseline zero-shot CLIP evaluation pipeline.
2. Implement and compare description selection strategies.
3. Analyze performance sensitivity to prompt wording and class semantics.

Related Literature

• [1] Learning Transferable Visual Models From Natural Language Supervision (CLIP)
• [2] Prompt Engineering Guide for Vision-Language Models
• [3] Tip-Adapter: Training-Free CLIP-Adapter for Better Vision-Language Modeling

Maintained by saifkhichi96 on GitHub.

The website is distributed under different open-source licenses. For more details, see the notice at the bottom of the page.