As an emerging research direction of machine learning, the multi-view learning (MVL) pays attention to the tasks that learn from datasets with several distinct views to achieve better generalization performance. Recently, various Support Vector Machine (SVM)-based algorithms with solid theoretical foundation have been proposed for MVL. However, there is a constraining assumption for these algorithms, i.e., in the learning process, different views are important equally for an instance in a data set, and the same view is important equally for all instances in a data set. In fact, an instance generally has different adaptability to different views, namely, the degree to which the information from different views accurately describes the instance varies. And naturally, different instances in a data set also have different adaptability to the same view. In this paper, the concept of view vector of each instance is proposed first, which quantitatively describes the adaptability of a specific instance to different views. It also reflects the characteristics of different instances that some instances are more suitable to be represented by a view, while others tend to be better represented by another view. Then, a new instance-based multi-view SVM algorithm, named IBMvSVM, is proposed by building the view vector of each instance into the multi-view SVM learning. IBMvSVM focuses on characteristics of each instance itself in different views rather than treating them equally. Experiments performed on 48 multi-view datasets reveal the superiority of IBMvSVM algorithm on generalization against several recently state-of-the-art MVL algorithms.