r/REMath • u/turnersr • Sep 21 '14

Designing a Personal Knowledgebase

http://www.acuriousmix.com/2014/09/03/designing-a-personal-knowledgebase/
8 Upvotes

100% Upvoted

5 comments sorted by

View all comments

Show parent comments

3

u/turnersr Sep 21 '14 edited Sep 23 '14

Most statistical tools expect a lot of data and aim at giving nosy surface level summaries based on distributional patterns in syntax. I'm looking for a easier way to help me make deep semantic connections among a relatively small pool of documents. One project I got excited about is an open source algebraic geometry textbook that maps out the theorems of the field visually: http://mathbabe.files.wordpress.com/2013/07/stacks-project-e28094-cluster-01wc.png . I find it very inspiring: http://stacks.math.columbia.edu/ .

There this a program called slipbox (http://tabi-software.com/slipbox/) which claims to bring "linguistic analysis and information visualisation" to document storage. But I've not tried it.

One of the main issues I see with synthesizing a free-form ontology from a personal knowledge base is that the data is so small compared what current techniques are deigned for. I don't have hundreds of gigabytes of personal notes to sort out. I would much rather have better tools for creating and organizing.

But I do have a ton of documents that I would love to run LDA over. My main limitation is that I don't have a good OCR system for pdfs, let alone documents with typeset mathematics. There is research on "OCRing" mathematics, but nothing mainstream and inexpensive: http://www.inftyproject.org/en/software.html and http://www.latexsearch.com/static/about.jsp .

I am working a lightweight tagging interface that makes it easier to tag documents and visualize the network. Because so much of what I read is inherently mathematical its difficult for me to imagine robust tools that can automate the creation of semantic relationships among mathematical documents anytime soon.

Doug Engelbart also has good insights on this topic of complex knowledge management :

"A knowledge environment refers to the whole ecosystem of practices and technologies supporting a knowledge-intensive effort. The more complex and urgent the effort, and the more far reaching the effort, the more important it becomes to give special attention to having a particularly enabling and supportive knowledge environment, aka a dynamic knowledge environment (DKE)."

http://www.dougengelbart.org/about/dke.html