Make a model describe and answer questions about real photos. Caption images with BLIP and compare greedy, beam-search and nucleus-sampling decoding, ask BLIP-VQA questions about each photo, then generate a pool of candidate captions and rerank them with CLIP image-text similarity. Model weights download the first time each is loaded: BLIP captioning (~1 GB), BLIP-VQA (~1.5 GB) and CLIP (~600 MB).
Data Download
Captioning & Decoding
Visual Question Answering
CLIP Reranking & Results
Premium Project
This guided, GPU-powered project is available to Premium members. Upgrade to train models end-to-end in an interactive notebook.