Article

Classifying matrices with a spectral regularization

Authors:

Ryota Tomioka,

Kazuyuki AiharaAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 895 - 902

https://doi.org/10.1145/1273496.1273609

Published: 20 June 2007 Publication History

Get Access

Abstract

We propose a method for the classification of matrices. We use a linear classifier with a novel regularization scheme based on the spectral l₁-norm of its coefficient matrix. The spectral regularization not only provides a principled way of complexity control but also enables automatic determination of the rank of the coefficient matrix. Using the Linear Matrix Inequality technique, we formulate the inference task as a single convex optimization problem. We apply our method to the motor-imagery EEG classification problem. The method not only improves upon conventional methods in the classification performance but also determines a subspace in the signal that concentrates discriminative information without any additional feature extraction step. The method can be easily generalized to regression problems by changing the loss function. Connections to other methods are also discussed.

References

[1]

Altun, Y., & Smola, A. (2006). Unifying Divergence Minimization and Statistical Inference via Convex Duality. In Proc. COLT2006.

Digital Library

Google Scholar

[2]

Blankertz, B., Dornhege, G., Krauledat, M., Müller, K., R., Kunzmann, V., Losch, F., & Curio, G. (2006). The Berlin Brain-Computer Interface: EEG-based communication without subject training. IEEE Trans. Neural Sys. Rehab. Eng., 14, 147--152.

Crossref

Google Scholar

[3]

Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Cambridge University Press.

Digital Library

Google Scholar

[4]

Dudík, M., Phillips, S. J., & Schapire, R. E. (2004). Performance Guarantees for Regularized Maximum Entropy Density Estimation. In Lect. Notes Comput. Sci., vol. 3120, 472--486. Springer.

Crossref

Google Scholar

[5]

Grant, M., Boyd, S., & Ye, Y. (2006). CVX: Matlab Software for Disciplined Convex Programming. http://www.stanford.edu/~boyd/cvx/, Version 1.0RC.

Google Scholar

[6]

Jaakkola, T. S., & Haussler, D. (1999). Probabilistic kernel regression models. Proceedings of the Seventh International Workshop on Artificial Intelligence and Statistics. Morgan Kaufmann.

Google Scholar

[7]

Koles, Z. J. (1991). The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalogr. Clin. Neurophysiol., 79, 440--447.

Crossref

Google Scholar

[8]

Pfurtscheller, G., & da Silva, F. H. L. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol., 110, 1842--1857.

Crossref

Google Scholar

[9]

Ramoser, H., Müller-Gerking, J., & Pfurtscheller, G. (2000). Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehab. Eng., 8, 441--446.

Crossref

Google Scholar

[10]

Sturm, J. F. (1999). Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimization Methods and Software, 11, 625--653. Software available at http://sedumi.mcmaster.ca/

Crossref

Google Scholar

[11]

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc. B., 58, 267--288.

Crossref

Google Scholar

[12]

Tomioka, R., Aihara, K., & Müüller, K.-R. (2007). Logistic Regression for Single Trial EEG Classification. Advances in Neural Inf. Proc. Systems (NIPS 06). MIT press.

Google Scholar

[13]

Tomioka, R., Hill, J., Blankertz, B., & Aihara, K. (2006). Adapting Spatial Filtering Methods for Nonstationary BCIs. Proceedings of 2006 Workshop on Information-Based Induction Sciences (IBIS2006) (pp. 65--70)

Google Scholar

Cited By

View all

Wang WQi FWipf DCai CYu TLi YZhang YYu ZWu W(2023)Sparse Bayesian Learning for End-to-End EEG DecodingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.329956845:12(15632-15649)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3299568
Mehtiyev AAl-Najjar ASadreazami HAmini M(2023)DeepEnsemble: A Novel Brain Wave Classification in MI-BCI using Ensemble of Deep Learners2023 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE56470.2023.10043385(1-5)Online publication date: 6-Jan-2023
https://doi.org/10.1109/ICCE56470.2023.10043385
Khoyani AKaur HAmini MSadreazami H(2022)Motor Imagery Brain Activity Recognition through Data Augmentation using DC-GANs and Mu-Sigma2022 IEEE Sensors10.1109/SENSORS52175.2022.9967231(1-4)Online publication date: 30-Oct-2022
https://doi.org/10.1109/SENSORS52175.2022.9967231
Show More Cited By

Classifying matrices with a spectral regularization
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Spectral Regularization Algorithms for Learning Large Incomplete Matrices

We use convex relaxation techniques to provide a sequence of regularized low-rank solutions for large-scale matrix completion problems. Using the nuclear norm as a regularizer, we provide a simple and very efficient convex algorithm for minimizing the ...
Spectral k-support norm regularization
NIPS'14: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2

The k-support norm has successfully been applied to sparse vector prediction problems. We observe that it belongs to a wider class of norms, which we call the box-norms. Within this framework we derive an efficient algorithm to compute the proximity ...
Spectral Characterizations for Hermitian Centrosymmetric K-Matrices and Hermitian Skew-Centrosymmetric K-Matrices

Let A and K be real symmetric matrices with K²=I. In the article "A spectral characterization of generalized real symmetric centrosymmetric and generalized real symmetric skew-centrosymmetric matrices" [D. Tao and M. Yasuda, SIAM J. Matrix Anal. Appl., ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
348
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang WQi FWipf DCai CYu TLi YZhang YYu ZWu W(2023)Sparse Bayesian Learning for End-to-End EEG DecodingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.329956845:12(15632-15649)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3299568
Mehtiyev AAl-Najjar ASadreazami HAmini M(2023)DeepEnsemble: A Novel Brain Wave Classification in MI-BCI using Ensemble of Deep Learners2023 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE56470.2023.10043385(1-5)Online publication date: 6-Jan-2023
https://doi.org/10.1109/ICCE56470.2023.10043385
Khoyani AKaur HAmini MSadreazami H(2022)Motor Imagery Brain Activity Recognition through Data Augmentation using DC-GANs and Mu-Sigma2022 IEEE Sensors10.1109/SENSORS52175.2022.9967231(1-4)Online publication date: 30-Oct-2022
https://doi.org/10.1109/SENSORS52175.2022.9967231
Sadreazami HMohammad-Taheri YAmini M(2022)Hierarchical Spectral-Temporal Feature Learning for Motor Task Recognition in Brain Computer Interfaces2022 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)10.1109/I2MTC48687.2022.9806696(1-5)Online publication date: 16-May-2022
https://doi.org/10.1109/I2MTC48687.2022.9806696
Zheng QWang YHeng P(2021)Multitask Feature Learning Meets Robust Tensor Decomposition for EEG ClassificationIEEE Transactions on Cybernetics10.1109/TCYB.2019.294691451:4(2242-2252)Online publication date: Apr-2021
https://doi.org/10.1109/TCYB.2019.2946914
Sadreazami SAmini MAhmad MSwamy M(2021)EEGCAPS: Brain Activity Recognition using Modified Common Spatial Patterns and Capsule Network2021 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS51556.2021.9401332(1-5)Online publication date: May-2021
https://doi.org/10.1109/ISCAS51556.2021.9401332
Qi FWu WLiu KYu TCao Y(2021)A Logistic Regression Based Framework for Spatio-Temporal Feature Representation and Classification of Single-Trial EEGCognitive Systems and Signal Processing10.1007/978-981-16-2336-3_36(387-394)Online publication date: 5-May-2021
https://doi.org/10.1007/978-981-16-2336-3_36
Arabshahi RRouhani M(2020)A convolutional neural network and stacked autoencoders approach for motor imagery based brain-computer interface2020 10th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE50421.2020.9303717(295-300)Online publication date: 29-Oct-2020
https://doi.org/10.1109/ICCKE50421.2020.9303717
Aydarkhanov RUšćumlić MChavarriaga RGheorghe Ldel R Millán J(2020)Spatial covariance improves BCI performance for late ERPs components with high temporal variabilityJournal of Neural Engineering10.1088/1741-2552/ab95eb17:3(036030)Online publication date: 25-Jun-2020
https://doi.org/10.1088/1741-2552/ab95eb
Sasaki MIversen JCallan D(2019)Music Improvisation Is Characterized by Increase EEG Spectral Power in Prefrontal and Perceptual Motor Cortical Sources and Can be Reliably Classified From Non-improvisatory PerformanceFrontiers in Human Neuroscience10.3389/fnhum.2019.0043513Online publication date: 10-Dec-2019
https://doi.org/10.3389/fnhum.2019.00435
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Spectral Regularization Algorithms for Learning Large Incomplete Matrices

Spectral k-support norm regularization

Spectral Characterizations for Hermitian Centrosymmetric K-Matrices and Hermitian Skew-Centrosymmetric K-Matrices