Multilabel Deep Visual-Semantic Embedding

Mei Chen Yeh; Yi Nan Li

doi:10.1109/TPAMI.2019.2911065

Multilabel Deep Visual-Semantic Embedding

Mei Chen Yeh, Yi Nan Li

Research output: Contribution to journal › Article › peer-review

2 Citations (Scopus)

Abstract

Inspired by the great success from deep convolutional neural networks (CNNs) for single-label visual-semantic embedding, we exploit extending these models for multilabel images. We propose a new learning paradigm for multilabel image classification, in which labels are ranked according to its relevance to the input image. In contrast to conventional CNN models that learn a latent vector representation (i.e., the image embedding vector), the developed visual model learns a mapping (i.e., a transformation matrix) from an image in an attempt to differentiate between its relevant and irrelevant labels. Despite the conceptual simplicity of our approach, the proposed model achieves state-of-the-art results on three public benchmark datasets.

Original language	English
Article number	8691414
Pages (from-to)	1530-1536
Number of pages	7
Journal	IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume	42
Issue number	6
DOIs	https://doi.org/10.1109/TPAMI.2019.2911065
Publication status	Published - 2020 Jun 1
Externally published	Yes

Keywords

convolutional neural networks
Multilabel classification
visual semantic embedding

ASJC Scopus subject areas

Software
Computer Vision and Pattern Recognition
Computational Theory and Mathematics
Artificial Intelligence
Applied Mathematics

Access to Document

10.1109/TPAMI.2019.2911065

Fingerprint Dive into the research topics of 'Multilabel Deep Visual-Semantic Embedding'. Together they form a unique fingerprint.

Cite this

@article{b195f9682ff448d1bf6c0fef3bfc465f,

title = "Multilabel Deep Visual-Semantic Embedding",

abstract = "Inspired by the great success from deep convolutional neural networks (CNNs) for single-label visual-semantic embedding, we exploit extending these models for multilabel images. We propose a new learning paradigm for multilabel image classification, in which labels are ranked according to its relevance to the input image. In contrast to conventional CNN models that learn a latent vector representation (i.e., the image embedding vector), the developed visual model learns a mapping (i.e., a transformation matrix) from an image in an attempt to differentiate between its relevant and irrelevant labels. Despite the conceptual simplicity of our approach, the proposed model achieves state-of-the-art results on three public benchmark datasets.",

keywords = "convolutional neural networks, Multilabel classification, visual semantic embedding",

author = "Yeh, {Mei Chen} and Li, {Yi Nan}",

note = "Funding Information: The authors would like to thank the reviewers for their helpful comments. The authors would also like to thank Fang Li for her help in conducting the experiments. This work was supported by the Ministry of Science and Technology of Taiwan (MOST 106-2221-E-003-031-MY2).",

year = "2020",

month = jun,

day = "1",

doi = "10.1109/TPAMI.2019.2911065",

language = "English",

volume = "42",

pages = "1530--1536",

journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",

issn = "0162-8828",

publisher = "IEEE Computer Society",

number = "6",

}

TY - JOUR

T1 - Multilabel Deep Visual-Semantic Embedding

AU - Yeh, Mei Chen

AU - Li, Yi Nan

N1 - Funding Information: The authors would like to thank the reviewers for their helpful comments. The authors would also like to thank Fang Li for her help in conducting the experiments. This work was supported by the Ministry of Science and Technology of Taiwan (MOST 106-2221-E-003-031-MY2).

PY - 2020/6/1

Y1 - 2020/6/1

N2 - Inspired by the great success from deep convolutional neural networks (CNNs) for single-label visual-semantic embedding, we exploit extending these models for multilabel images. We propose a new learning paradigm for multilabel image classification, in which labels are ranked according to its relevance to the input image. In contrast to conventional CNN models that learn a latent vector representation (i.e., the image embedding vector), the developed visual model learns a mapping (i.e., a transformation matrix) from an image in an attempt to differentiate between its relevant and irrelevant labels. Despite the conceptual simplicity of our approach, the proposed model achieves state-of-the-art results on three public benchmark datasets.

AB - Inspired by the great success from deep convolutional neural networks (CNNs) for single-label visual-semantic embedding, we exploit extending these models for multilabel images. We propose a new learning paradigm for multilabel image classification, in which labels are ranked according to its relevance to the input image. In contrast to conventional CNN models that learn a latent vector representation (i.e., the image embedding vector), the developed visual model learns a mapping (i.e., a transformation matrix) from an image in an attempt to differentiate between its relevant and irrelevant labels. Despite the conceptual simplicity of our approach, the proposed model achieves state-of-the-art results on three public benchmark datasets.

KW - convolutional neural networks

KW - Multilabel classification

KW - visual semantic embedding

UR - http://www.scopus.com/inward/record.url?scp=85084720672&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85084720672&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2019.2911065

DO - 10.1109/TPAMI.2019.2911065

M3 - Article

C2 - 30990418

AN - SCOPUS:85084720672

VL - 42

SP - 1530

EP - 1536

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 6

M1 - 8691414

ER -