Does the current version support multi-image input in a single conversation?

[AQingRoad]

Hi, thank you for your great work on PaDT!

I’m trying to run inference with multiple images in a single conversation turn — that is, passing several "type": "image" entries within one message like this:
messages = [ { "role": "user", "content": [ {"type": "image", "image": "/path/to/img1.jpg"}, {"type": "image", "image": "/path/to/img2.jpg"}, {"type": "text", "text": "Please describe the two images."} ] } ]
However, when I run inference, I encounter the following error:
RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 0.
Target sizes: [1, 719, 152626]. Tensor sizes: [2, 719, 152626]

It seems that the model’s forward path assumes a single image per conversation,
and the logits.masked_fill_ operation fails when more than one image is provided.

Could you please confirm whether the current version supports multi-image input in a single chat message?
If not, do you plan to add support for multi-image reasoning (e.g., image comparison or cross-image description) in future releases?

Thanks again for your excellent work! 🙏

[Dyb3438]

@AQingRoad Thank you for your interest in our work. Our current work focuses solely on visual perception and understanding tasks involving a single image. Multi-image tasks within a single conversation are not considered in the code that has been released. The main novel concept proposed by this work is that image patches appear as decodable tokens within the generated sequence, which boosts image perception and understanding tasks.

Fortunately, our subsequent work will address problems involving the understanding of multiple images, such as image comparison and object matching, as you mentioned.

Thank you.