Mastering natural language processing with Python : maximize your NLP capabilities while creating amazing NLP projects in Python
Cover -- Copyright -- Credits -- About the Authors -- About the Reviewer -- www.PacktPub.com -- Table of Contents -- Preface -- Chapter 1: Working with Strings -- Tokenization -- Tokenization of text into sentences -- Tokenization of text in other languages -- Tokenization of sentences into words -- Tokenization using TreebankWordTokenizer -- Tokenization using regular expressions -- Normalization -- Eliminating punctuation -- Dealing with stop words -- Calculate stopwords in English -- Substituting and correcting tokens -- Replacing words using regular expressions -- Example of the replacement of a text with another text -- Performing substitution before tokenization -- Dealing with repeating characters -- Example of deleting repeating characters -- Replacing a word with its synonym -- Example of substituting word a with its synonym -- Applying Zipf's law to text -- Similarity measures -- Applying similarity measures using Ethe edit distance algorithm -- Applying similarity measures using Jaccard's Coefficient -- Applying similarity measures using the Smith Waterman distance -- Other string similarity metrics -- Summary -- Chapter 2: Statistical Language Modeling -- Understanding word frequency -- Develop MLE for a given text -- Hidden Markov Model estimation -- Applying smoothing on the MLE model -- Add-one smoothing -- Good Turing -- Kneser Ney estimation -- Witten Bell estimation -- Develop a back-off mechanism for MLE -- Applying interpolation on data to get mix and match -- Evaluate a language model through perplexity -- Applying metropolis hastings in modeling languages -- Applying Gibbs sampling in language processing -- Summary -- Chapter 3: Morphology - Getting Our Feet Wet -- Introducing morphology -- Understanding stemmer -- Understanding lemmatization -- Developing a stemmer for non-english language -- Morphological analyzer
eBook, English, 2016
Online-ausg View all formats and editions
Packt Publishing, Birmingham, UK, 2016
Online-Ressource (1 online resource (1 v.)) : Illustrationen.
9781783989041, 9781783989058, 1783989041, 178398905X
1047865400