Digital Services


The benefits of project and process framework can be applied to every part of existence.


LSA: Interesting, but is it Useful?

I ran the last two months' worth of text messages between my friend Kiara and I into the latent semantic analysis program I discussed in my last blog post.

The program uses a sparse matrix to analyze the relationships between words, and delivers a set of concepts related to the documents and terms.

When I fed the text messages (in a txt doc form) into the program, it returned "concepts" such as these:

Concept 1:
ha ha

Concept 2:
Would like
would be
maybe we could

Concept 1 seems to indicate that we are friends, as we speak a lot about one another and share funny stories. Concept 2 is more interesting - Kiara and I often share places we would like to visit and plan trips over text. The second concept seems to be a reflection of wanting to visit places together.

While this is fun, it is clear that I'll need to do a lot more work to make the results of LSA useful. At the moment it feels a bit like reading a horoscope - something that I project meaning onto, but devoid of intrinsic usefulness to my particular dataset. It reminds me of a great quote from Introduction to Latent Semantic Analysis: Discourse, Processes:

One might consider [LSA’s] maximal knowledge of the world to be analogous to a well-read Nun’s knowledge of sex, a level of knowledge often deemed a sufficient basis for advising the young.
— Landauer, T. K., Foltz, P. W., & Laham, D.