Parallel Support Vector Machines Is A Supervised Machine...

Parallel Support Vector Machine Junfeng Wu Junming Chen May 6, 2016 1 INTRODUCTION Support vector machines is a supervised machine learning alogrithom used for classification. The problem could be written : minimize 1 |w |2 2 yi((w,xi)+b)−1≠¥0 where w is a linear combination of the training data: n w = ÃŽ ±i k(xi ) i=1 this could be further written in a dual form[5]: min 1ÃŽ ±TQÃŽ ±Ã¢Ë†â€™eTÃŽ ± ÃŽ ±2 0≠¤ÃŽ ±i ≠¤C, yTÃŽ ±=0, ∀i ≠¤n where Q is the kernel matrix. This dual form is a quadratic programming problem with linear constraints. A solver to this problem like SMO or IPM reuquires a time complexity of O(n3) and space complexity of O(n2), which makes SVM hard to scale. In our final project we implemented two parallel quadratic programming solvers for SVM and implemented a matrix factorization algorithm to improve the performance. Later we evaluated our implementation. 1 i =1,...,l, 2 SMO SOLVER IN PARALLEL SMO is one of the most common ways to solve quadratic programming problem. SMO is a iterative alogrithm. In each iteration, the alogorithm optimize one pair of Lagrange multipliers (ÃŽ ±1,ÃŽ ±2) that could best accelerate the convergence util the lagrange multipliers meet the convergence condition. 2.1 SELECTING AND UPDATING ÃŽ ± PAIR We select alpha1 and alpha2 that make the largest progress towards the global maximum value on each side of the hyper plane according to the heuristic function. The heuristic function is as follow: n f (i) = aj yj k(xj ,xi )− yi j=1 The selection of ÃŽ ± pair is as