Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification

Yan He Chen; Mei Chen Yeh

doi:10.1145/3460426.3463592

Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification

Yan He Chen, Mei Chen Yeh

Department of Computer Science and Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

We address the generalized zero-shot fine-grained image classification problem, in which classes are visually similar and training images for some classes are not available. We leverage auxiliary information in the form of textual descriptions to facilitate the task. Specifically, we propose a text-enhanced attribute-based attention mechanism to compute features from the most relevant image regions guided from the most relevant attributes. Experiments on two popular datasets of CUB and AWA2 show the effectiveness of the proposed method.

Original language	English
Title of host publication	ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval
Publisher	Association for Computing Machinery, Inc
Pages	447-450
Number of pages	4
ISBN (Electronic)	9781450384636
DOIs	https://doi.org/10.1145/3460426.3463592
Publication status	Published - 2021 Aug 24
Event	11th ACM International Conference on Multimedia Retrieval, ICMR 2021 - Taipei, Taiwan Duration: 2021 Nov 16 → 2021 Nov 19

Publication series

Name	ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval

Conference

Conference	11th ACM International Conference on Multimedia Retrieval, ICMR 2021
Country/Territory	Taiwan
City	Taipei
Period	2021/11/16 → 2021/11/19

Keywords

Attention
Fine-grained recognition
Generalized zero-shot learning
Neural networks

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Computer Science Applications
Computer Vision and Pattern Recognition
Human-Computer Interaction
Software

Access to Document

10.1145/3460426.3463592

Cite this

Chen, Y. H., & Yeh, M. C. (2021). Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification. In ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval (pp. 447-450). (ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval). Association for Computing Machinery, Inc. https://doi.org/10.1145/3460426.3463592

Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification. / Chen, Yan He; Yeh, Mei Chen.
ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2021. p. 447-450 (ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Chen, YH & Yeh, MC 2021, Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification. in ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval. ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval, Association for Computing Machinery, Inc, pp. 447-450, 11th ACM International Conference on Multimedia Retrieval, ICMR 2021, Taipei, Taiwan, 2021/11/16. https://doi.org/10.1145/3460426.3463592

Chen YH, Yeh MC. Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification. In ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc. 2021. p. 447-450. (ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval). doi: 10.1145/3460426.3463592

@inproceedings{e0d6dbe85ea34c8e9eb587b11266b2e4,

title = "Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification",

abstract = "We address the generalized zero-shot fine-grained image classification problem, in which classes are visually similar and training images for some classes are not available. We leverage auxiliary information in the form of textual descriptions to facilitate the task. Specifically, we propose a text-enhanced attribute-based attention mechanism to compute features from the most relevant image regions guided from the most relevant attributes. Experiments on two popular datasets of CUB and AWA2 show the effectiveness of the proposed method.",

keywords = "Attention, Fine-grained recognition, Generalized zero-shot learning, Neural networks",

author = "Chen, {Yan He} and Yeh, {Mei Chen}",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 11th ACM International Conference on Multimedia Retrieval, ICMR 2021 ; Conference date: 16-11-2021 Through 19-11-2021",

year = "2021",

month = aug,

day = "24",

doi = "10.1145/3460426.3463592",

language = "English",

series = "ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval",

publisher = "Association for Computing Machinery, Inc",

pages = "447--450",

booktitle = "ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval",

}

TY - GEN

T1 - Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification

AU - Chen, Yan He

AU - Yeh, Mei Chen

PY - 2021/8/24

Y1 - 2021/8/24

N2 - We address the generalized zero-shot fine-grained image classification problem, in which classes are visually similar and training images for some classes are not available. We leverage auxiliary information in the form of textual descriptions to facilitate the task. Specifically, we propose a text-enhanced attribute-based attention mechanism to compute features from the most relevant image regions guided from the most relevant attributes. Experiments on two popular datasets of CUB and AWA2 show the effectiveness of the proposed method.

AB - We address the generalized zero-shot fine-grained image classification problem, in which classes are visually similar and training images for some classes are not available. We leverage auxiliary information in the form of textual descriptions to facilitate the task. Specifically, we propose a text-enhanced attribute-based attention mechanism to compute features from the most relevant image regions guided from the most relevant attributes. Experiments on two popular datasets of CUB and AWA2 show the effectiveness of the proposed method.

KW - Attention

KW - Fine-grained recognition

KW - Generalized zero-shot learning

KW - Neural networks

UR - http://www.scopus.com/inward/record.url?scp=85114899877&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85114899877&partnerID=8YFLogxK

U2 - 10.1145/3460426.3463592

DO - 10.1145/3460426.3463592

M3 - Conference contribution

AN - SCOPUS:85114899877

T3 - ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval

SP - 447

EP - 450

BT - ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval

PB - Association for Computing Machinery, Inc

T2 - 11th ACM International Conference on Multimedia Retrieval, ICMR 2021

Y2 - 16 November 2021 through 19 November 2021

ER -

Text-enhanced attribute-based attention for generalized zero-shot fine-grained image classification

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this