short-paper

Deep Adaptive Feature Aggregation in Multi-task Convolutional Neural Networks

Authors:

Yilong YinAuthors Info & Claims

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Pages 2213 - 2216

https://doi.org/10.1145/3340531.3412132

Published: 19 October 2020 Publication History

Get Access

Abstract

Convolutional Neural Network (CNN) based multi-task learning methods have been widely used in a variety of applications of computer vision. Towards effective multi-task CNN architectures, recent studies automatically learn the optimal combinations of task-specific features at single network layers. However, they generally construct an unchanged operation of feature aggregation after training, regardless of the characteristics of input features. In this paper, we propose a novel Adaptive Feature Aggregation (AFA) layer for multi-task CNNs, in which a dynamic aggregation mechanism is designed to allow each task to adaptively determine the degree to which the feature aggregation of different tasks is needed according to the feature dependencies. On both pixel-level and image-level tasks, we demonstrate that our approach significantly outperforms the previous state-of-the-art methods of multi-task CNNs.

Supplementary Material

MP4 File (3340531.3412132.mp4)

In presentation, we introduce the Adaptive Feature Aggregation layer for multi-task CNNs, in which a dynamic aggregation mechanism is designed to allow each task to determine the degree to which the feature aggregation of different tasks is needed.

Download
109.76 MB

References

[1]

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV. 801--818.

Google Scholar

[2]

Eran Eidinger, Roee Enbar, and Tal Hassner. 2014. Age and gender estimation of unfiltered faces. TIFS, Vol. 9, 12 (2014), 2170--2179.

Google Scholar

[3]

Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L Yuille. 2019. NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reduction. In CVPR. 3205--3214.

Google Scholar

[4]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.

Google Scholar

[5]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In CVPR. 7132--7141.

Google Scholar

[6]

Brendan Jou and Shih-Fu Chang. 2016. Deep cross residual learning for multitask visual recognition. In MM. 998--1007.

Google Scholar

[7]

Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. 2016. Deeper depth prediction with fully convolutional residual networks. In 3DV. 239--248.

Google Scholar

[8]

Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang. 2019. Selective Kernel Networks. In CVPR. 510--519.

Google Scholar

[9]

Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. 2016. Cross-stitch networks for multi-task learning. In CVPR. 3994--4003.

Google Scholar

[10]

Sebastian Ruder. 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017).

Google Scholar

[11]

Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. 2012. Indoor segmentation and support inference from rgbd images. In ECCV. 746--760.

Google Scholar

Cited By

View all

Cui CZong JMa YWang XGuo LChen MYin Y(2024)Tri-Branch Convolutional Neural Networks for Top-k Focused Academic Performance PredictionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.317506835:1(439-450)Online publication date: Jan-2024
https://doi.org/10.1109/TNNLS.2022.3175068
Yuan HHe YDu PSong LXu Y(2024)Real-Time Facial Attribute Recognition Using Multi-Task Learning2024 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)10.1109/I2MTC60896.2024.10561176(1-6)Online publication date: 20-May-2024
https://doi.org/10.1109/I2MTC60896.2024.10561176
Hou QKe YWang KQin FWang Y(2024)Synchronous composition and semantic line detection based on cross-attentionMultimedia Systems10.1007/s00530-024-01307-x30:3Online publication date: 9-Apr-2024
https://doi.org/10.1007/s00530-024-01307-x
Show More Cited By

Index Terms

Deep Adaptive Feature Aggregation in Multi-task Convolutional Neural Networks
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Multi-task learning
    2. Machine learning approaches
      1. Neural networks

Recommendations

Towards dropout training for convolutional neural networks

Recently, dropout has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in convolutional and pooling layers is still not clear. This paper ...
Feature Analysis of Unsupervised Learning for Multi-task Classification Using Convolutional Neural Network

This study analyzes the characteristics of unsupervised feature learning using a convolutional neural network (CNN) to investigate its efficiency for multi-task classification and compare it to supervised learning features. We keep the conventional CNN ...
Convolutional adaptive denoising autoencoders for hierarchical feature extraction

Convolutional neural networks (CNNs) are typical structures for deep learning and are widely used in image recognition and classification. However, the random initialization strategy tends to become stuck at local plateaus or even diverge, which results ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

October 2020

3619 pages

ISBN:9781450368599

DOI:10.1145/3340531

General Chairs:
Mathieu d'Aquin
DSI, Insight, NUI Galway, Ireland
,
Stefan Dietze
GESIS, Cologne, Germany, Heinrich-Heine-University Düsseldorf, Germany, L3S Research Center, Germany
,
Program Chairs:
Claudia Hauff
TU Delft, The Netherlands
,
Edward Curry
DSI, Insight, NUI Galway, Ireland
,
Philippe Cudre Mauroux
eXascale, University of Fribourg, Switzerland

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Key R&D Program of China
National Natural Science Foundation of China

Conference

CIKM '20

Sponsor:

CIKM '20: The 29th ACM International Conference on Information and Knowledge Management

October 19 - 23, 2020

Virtual Event, Ireland

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
185
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cui CZong JMa YWang XGuo LChen MYin Y(2024)Tri-Branch Convolutional Neural Networks for Top-k Focused Academic Performance PredictionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.317506835:1(439-450)Online publication date: Jan-2024
https://doi.org/10.1109/TNNLS.2022.3175068
Yuan HHe YDu PSong LXu Y(2024)Real-Time Facial Attribute Recognition Using Multi-Task Learning2024 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)10.1109/I2MTC60896.2024.10561176(1-6)Online publication date: 20-May-2024
https://doi.org/10.1109/I2MTC60896.2024.10561176
Hou QKe YWang KQin FWang Y(2024)Synchronous composition and semantic line detection based on cross-attentionMultimedia Systems10.1007/s00530-024-01307-x30:3Online publication date: 9-Apr-2024
https://doi.org/10.1007/s00530-024-01307-x
Feng QChen S(2023)Learning multi-tasks with inconsistent labels by using auxiliary big taskFrontiers of Computer Science10.1007/s11704-022-2251-x17:5Online publication date: 22-Feb-2023
https://doi.org/10.1007/s11704-022-2251-x
Cui CShen ZHuang JChen MXu MWang MYin Y(2022)Adaptive Feature Aggregation in Deep Multi-Task Convolutional Neural NetworksIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2021.308782332:4(2133-2144)Online publication date: Apr-2022
https://doi.org/10.1109/TCSVT.2021.3087823

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Towards dropout training for convolutional neural networks

Feature Analysis of Unsupervised Learning for Multi-task Classification Using Convolutional Neural Network

Convolutional adaptive denoising autoencoders for hierarchical feature extraction