Text Processing and Analysis

Text

File

URL

Statistics

Scores

Misspelled words

Stop words

Stemming

Filtering

character, word, paragraph, sentence, syllable, etc. counting

line feed, return, tab, special characters, etc.

repetition words in a short range

Frequent words

Word length

bi-gram, tri-gram, n-gram

Unique words

Lexical density

Flesch-Kincaid

Gunning-Fog

Coleman-Liau

SMOG

Lau-King Chinese Readability

POS tagger

Parser

NER

location, person, thing, etc.

event, year, telephone, address, etc.

Summarization/annotation

sentiment/opinion analysis

concept clustering

Histogram

Similar words, concept graph

Word cloud

Title

Name

Abstract

Conclusion

References

http://www.addedbytes.com/lab/readability-score/

http://www.ghacks.net/2008/03/16/text-statistics/

https://github.com/DaveChild/Text-Statistics

http://www.usingenglish.com/resources/text-statistics.php

http://www.usingenglish.com/resources/wordcheck/

https://wiki.projectbamboo.org/display/BPUB/Text+Analysis,+Data-Mining+and+Machine+Learning

http://www.clarabridge.com/

http://contour.dac.us/Explore?ReturnUrl=%2findex.html

http://www.provalisresearch.com/wordstat/wordstat.html

http://www.maxqda.com/products/maxqda10

http://textanalysis.com/