Search Results for author:
Found 10 papers, 4 papers with code
Visual Perception by Large Language Model's Weights
Following this paradigm, we propose VLoRA with the perceptual weights generator.
Multi-Modal Generative Embedding Model
Existing models usually tackle these two types of problems by decoupling language modules into a text decoder for generation, and a text encoder for embedding.
Stare at What You See: Masked Image Modeling without Reconstruction
However, unlike the low-level features such as pixel values, we argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image. This raises one question: is reconstruction necessary in Masked Image Modeling (MIM) with a teacher model?
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks.
Ranked #2 on
Video Retrieval
on QuerYD
(using extra training data)
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment
and 2) how to mitigate the impact of these factors?
Ranked #2 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
To enable VL pre-training, we jointly optimize the HD-VILA model by a hybrid Transformer that learns rich spatiotemporal features, and a multimodal Transformer that enforces interactions of the learned video features with diversified texts.
Ranked #16 on
Video Retrieval
on MSR-VTT
Unifying Multimodal Transformer for Bi-directional Image and Text Generation
We adopt Transformer as our unified architecture for its strong performance and task-agnostic design.
Learning Fine-Grained Motion Embedding for Landscape Animation
To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation.
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
To tackle this, we propose a fully Transformer visual embedding for VLP to better learn visual relation and further promote inter-modal alignment.
