Rehashing Kernel Evaluation in High Dimensions
Paris Siminelakis, Kexin Rong, Peter Bailis, Moses Charikar, and Philip Levis
Published in Proceedings of the 36th International Conference on Machine Learning (ICML '19), June 2019.
Kernel methods are effective but do not scale wellto large scale data, especially in high dimensions where the geometric data structures used to accelerate kernel evaluation suffer from the curse of dimensionality. Recent theoretical advances have proposed fast kernel evaluation algorithms lever-aging hashing techniques with worst-case asymptotic improvements. However, these advances are largely confined to the theoretical realm due to concerns such as super-linear preprocessing time and diminishing gains in non-worst case datasets.In this paper, we close the gap between theory and practice by addressing these challenges via provable and practical procedures for adaptive sample size selection, preprocessing time reduction, and refined variance bounds that quantify the data-dependent performance of random sampling and hashing-based kernel evaluation methods. Our experiments show that these new tools offer up to 10x improvement in evaluation time on a range of synthetic and real-world datasets.
Paper (3MB)
BibTeX entry
@inproceedings{siminelakis19icml, author = "Paris Siminelakis and Kexin Rong and Peter Bailis and Moses Charikar and Philip Levis", title = "{Rehashing Kernel Evaluation in High Dimensions}", booktitle = "{Proceedings of the 36th International Conference on Machine Learning (ICML '19)}", year = {2019}, month = {June} }