LSA: Interesting, but is it Useful?
I ran the last two months' worth of text messages between my friend Kiara and I into the latent semantic analysis program I discussed in my last blog post.
The program uses a sparse matrix to analyze the relationships between words, and delivers a set of concepts related to the documents and terms.
When I fed the text messages (in a txt doc form) into the program, it returned "concepts" such as these:
Concept 1: lol love you ha ha us maybe your Concept 2: Would like maybe going go us would be maybe we could
Concept 1 seems to indicate that we are friends, as we speak a lot about one another and share funny stories. Concept 2 is more interesting - Kiara and I often share places we would like to visit and plan trips over text. The second concept seems to be a reflection of wanting to visit places together.
While this is fun, it is clear that I'll need to do a lot more work to make the results of LSA useful. At the moment it feels a bit like reading a horoscope - something that I project meaning onto, but devoid of intrinsic usefulness to my particular dataset. It reminds me of a great quote from Introduction to Latent Semantic Analysis: Discourse, Processes: