Cosine similarity is a measure of similarity between two vectors of an inner product space
that measures the cosine of the angle between them. The cosine of 0° is 1, and it is less
than 1 for any other angle. It is thus a judgement of orientation and not magnitude: two vectors
with the same orientation have a Cosine similarity of 1, two vectors at 90° have a similarity
of 0, and two vectors diametrically opposed have a similarity of -1, independent of their
magnitude. Cosine similarity is particularly used in positive space, where the outcome
is neatly bounded in [0,1]. Note that these bounds apply for any number
of dimensions, and Cosine similarity is most commonly used in high-dimensional positive
spaces. For example, in Information Retrieval and text mining, each term is notionally assigned
a different dimension and a document is characterised by a vector where the value of each dimension
corresponds to the number of times that term appears in the document. Cosine similarity
then gives a useful measure of how similar two documents are likely to be in terms of
their subject matter. The technique is also used to measure cohesion
within clusters in the field of data mining. Cosine distance is a term often used for the
complement in positive space, that is: . It is important to note, however, that this is
not a proper distance metric as it does not have the triangle inequality property and
it violates the coincidence axiom; to repair the triangle inequality property whilst maintaining
the same ordering, it is necessary to convert to Angular distance
One of the reasons for the popularity of Cosine similarity is that it is very efficient to
evaluate, especially for sparse vectors, as only the non-zero dimensions need to be considered.
Definition The cosine of two vectors can be derived by
using the Euclidean dot product formula:
Given two vectors of attributes, A and B, the cosine similarity, cos(θ), is represented
using a dot product and magnitude as
The resulting similarity ranges from −1 meaning exactly opposite, to 1 meaning exactly
the same, with 0 usually indicating independence, and in-between values indicating intermediate
similarity or dissimilarity. For text matching, the attribute vectors A
and B are usually the term frequency vectors of the documents. The cosine similarity can
be seen as a method of normalizing document length during comparison.
In the case of information retrieval, the cosine similarity of two documents will range
from 0 to 1, since the term frequencies cannot be negative. The angle between two term frequency
vectors cannot be greater than 90°. If the attribute vectors are normalized by
substracting the vector means, the measure is called centered cosine similarity and is
equivalent to the Pearson Correlation Coefficient. Angular similarity
The term "cosine similarity" has also been used on occasion to express a different coefficient,
although the most common use is as defined above. Using the same calculation of similarity,
the normalised angle between the vectors can be used as a bounded similarity function within
[0,1], calculated from the above definition of similarity by:
in a domain where vector coefficients may be positive or negative, or
in a domain where the vector coefficients are always positive.
Although the term "cosine similarity" has been used for this angular distance, the term
is oddly used as the cosine of the angle is used only as a convenient mechanism for calculating
the angle itself and is no part of the meaning. The advantage of the angular similarity coefficient
is that, when used as a difference coefficient the resulting function is a proper distance
metric, which is not the case for the first meaning. However for most uses this is not
an important property. For any use where only the relative ordering of similarity or distance
within a set of vectors is important, then which function is used is immaterial as the
resulting order will be unaffected by the choice.
Confusion with "Tanimoto" coefficient The cosine similarity may be easily confused
with the Tanimoto metric - a specialised form of a similarity coefficient with a similar
algebraic form:
In fact, this algebraic form was first defined by Tanimoto as a mechanism for calculating
the Jaccard coefficient in the case where the sets being compared are represented as
bit vectors. While the formula extends to vectors in general, it has quite different
properties from cosine similarity and bears little relation other than its superficial
appearance. Ochiai coefficient
This coefficient is also known in biology as Ochiai coefficient, or Ochiai-Barkman coefficient,
or Otsuka-Ochiai coefficient:
Here, and are sets, and is the number of elements in . If sets are represented as bit vectors,
the Ochiai coefficient can be seen to be the same as the cosine similarity.
Properties Cosine similarity is related to Euclidean
distance as follows. Denote Euclidean distance by the usual , and observe that
by expansion. When A and B are normalized to unit length, so the previous is equal to
Null distribution: For data which can be negative as well as positive, the null distribution
for cosine similarity is the distribution of the dot product of two independent random
unit vectors. This distribution has a mean of zero and a variance of , and although the
distribution is bounded between -1 and +1, as grows large the distribution is increasingly
well-approximated by the normal distribution. For other types of data, such as bitstreams,
the null distribution will take a different form, and may have a nonzero mean.
See also Sørensen's quotient of similarity
Hamming distance Correlation
Dice's coefficient Jaccard index
SimRank Information retrieval
References
External links Online Cosine Similarity Calculator
Weighted cosine measure A tutorial on cosine similarity using Python