Work has continued on analyzing the mood and sentiment of a text by splitting the text into sentences using a regular expression that accounted for .!?’” sentence endings but wouldn’t split on common abbreviations like Mr., Mrs., and other titles. After I had a list of sentences I pulled in my proper nouns list, which turned out to be the most accurate list of potential character name matches, and compared the two lists with a regular expression that appended the sentence to a dictionary under the key of the character name if it was found in the sentence.
From there I used pattern to parse and return the mood of the sentence, categorized into four types of grammatical moods; indicative, imperative, conditional, or subjunctive. From the pattern API documentation, an example of an indicative (fact or belief) sentence would be “It rains.” An example of an imperative (command or warning) sentence would be “Don’t rain!” An example of a conditional (conjecture) sentence would be “It might rain.” An example of a subjunctive (wish or opinion) sentence would be “I hope it rains.” I created a separate dictionary to hold the mood data which is also grouped by character name as the key token.
For sentiment analysis I used the pattern.vector module. The module contains machine learning tools that use the previous classification of a body of text to predict the future classification of a body of text. From the documentation, classification is a supervised machine learning method that uses labeled documents (i.e., Document objects with a type) as training examples to statistically predict the label (class, type) of new documents, based on their similarity to the training examples using a distance metric (e.g., cosine similarity). A Document is a bag-of-words representation of a text, i.e., unordered words + word count.
It uses a Naive Bayes classifier to extract the information from a text (in this case a text that labels sentences as a good or bad movie review) and inform the prediction of unknown text (in this case, whether a sentence has a good or bad sentiment tone). A Naive Bayes classifier assumes that the presence or absence of a particular feature is unrelated to the presence or absence of any other feature, given the class variable. For example, a fruit may be considered to be an apple if it is red, round, and about 3″ in diameter. The classifier considers each of these features to contribute independently to the probability that this fruit is an apple, regardless of the presence or absence of the other features.
To use the classifier I placed the trainer file reviews.csv in the same folder as my script and imported the corresponding modules into my script, then used the train function on a NB object with the list of classified text pulled in from the csv file. I then iterated over the keys and values in my list of characters and sentences and called the classify function (from the NB object) on the value, appending this to a list with the character name as the key. This gives a range of 0-5 for each sentence; 0 being bad or negative tone, 5 being good or positive tone.
While I was debugging the scripts I found a different way to preserve the dictionary format of my keys and values so that I could write and read quickly while preserving the nested structure. In Python it is called pickling; the module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. I had previously been using text files to write to and then read from the dictionaries I was creating, but when reading in the data I would have to recreate the structure by splitting and stripping on different punctuation points in the text.
Here is a character analyses of Noah Claypole from Oliver Twist, and the literary critique of the character from SparkNotes for comparison:
Character Synopsis of Noah Claypole: A charity boy and Mr. Sowerberry’s apprentice. Noah is an overgrown, cowardly bully who mistreats Oliver and eventually joins Fagin’s gang.
Whether Noah:[['Whether Noah Claypole, whose rapacity was none of the least comprehensive, would have acceded even to these glowing terms, had he been a perfectly free agent, is very doubtful; but as he recollected that, in the event of his refusal, it was in the power of his new acquaintance to give him up to justice immediately (and more unlikely things had come to pass), he gradually relented, and said he thought that would suit him.'], , ['conditional']]
Poor Noah:[['Poor Noah!'], , ['indicative']]
Noah Claypole:[["'I'm Mister Noah Claypole,' said the charity-boy, 'and you're under me.", 'said Noah Claypole.', 'That Oliver Twist was moved to resignation by the example of these good people, I cannot, although I am his biographer, undertake to affirm with any degree of confidence; but I can most distinctly say, that for many months he continued meekly to submit to the domination and ill-treatment of Noah Claypole: who used him far worse than before, now that his jealousy was roused by seeing the new boy promoted to the black stick and hatband, while he, the old one, remained stationary in the muffin-cap and leathers.', 'One day, Oliver and Noah had descended into the kitchen at the usual dinner-hour, to banquet upon a small joint of mutton--a pound and a half of the worst end of the neck--when Charlotte being called out of the way, there ensued a brief interval of time, which Noah Claypole, being hungry and vicious, considered he could not possibly devote to a worthier purpose than aggravating and tantalising young Oliver Twist.', 'CHAPTER VII\nOLIVER CONTINUES REFRACTORY\nNoah Claypole ran along the streets at his swiftest pace, and paused not once for breath, until he reached the workhouse-gate.', "And the cocked hat and cane having been, by this time, adjusted to their owner's satisfaction, Mr. Bumble and Noah Claypole betook themselves with all speed to the undertaker's shop.", 'Now, Mr. and Mrs. Sowerberry having gone out to tea and supper: and Noah Claypole not being at any time disposed to take upon himself a greater amount of physical exertion than is necessary to a convenient performance of the two functions of eating and drinking, the shop was not closed, although it was past the usual hour of shutting-up.', 'At the upper end of the table, Mr. Noah Claypole lolled negligently in an easy-chair, with his legs thrown over one of the arms: an open clasp-knife in one hand, and a mass of buttered bread in the other.', "'Never mind whether they're two mile off, or twenty,' said Noah Claypole; for he it was; 'but get up and come on, or I'll kick yer, and so I give yer notice.'", 'Through these streets, Noah Claypole walked, dragging Charlotte after him; now stepping into the kennel to embrace at a glance the whole external character of some small public-house; now jogging on again, as some fancied appearance induced him to believe it too public for his purpose.', 'asked Noah Claypole.', "Noah Claypole's mind might have been at ease after this assurance, but his body certainly was not; for he shuffled and writhed about, into various uncouth positions: eyeing his new friend meanwhile with mingled fear and suspicion.", 'Whether Noah Claypole, whose rapacity was none of the least comprehensive, would have acceded even to these glowing terms, had he been a perfectly free agent, is very doubtful; but as he recollected that, in the event of his refusal, it was in the power of his new acquaintance to give him up to justice immediately (and more unlikely things had come to pass), he gradually relented, and said he thought that would suit him.', "Noah Claypole, bespeaking his good lady's attention, proceeded to enlighten her relative to the arrangement he had made, with all that haughtiness and air of superiority, becoming, not only a member of the sterner sex, but a gentleman who appreciated the dignity of a special appointment on the kinchin lay, in London and its vicinity.", 'Noah Claypole, or Morris Bolter as the reader pleases, punctually followed the directions he had received, which--Master Bates being pretty well acquainted with the locality--were so exact that he was enabled to gain the magisterial presence without asking any question, or meeting with any interruption by the way.', 'CHAPTER XLV\nNOAH CLAYPOLE IS EMPLOYED BY FAGIN ON A SECRET MISSION\nThe old man was up, betimes, next morning, and waited impatiently for the appearance of his new associate, who after a delay that seemed interminable, at length presented himself, and commenced a voracious assault on the breakfast.', "Peeping out, more than once, when he reached the top, to make sure that he was unobserved, Noah Claypole darted away at his utmost speed, and made for the Jew's house as fast as his legs would carry him.", 'Stretched upon a mattress on the floor, lay Noah Claypole, fast asleep.', 'Mr. Noah Claypole: receiving a free pardon from the Crown in consequence of being admitted approver against Fagin: and considering his profession not altogether as safe a one as he could wish: was, for some little time, at a loss for the means of a livelihood, not burdened with too much work.'], [0, 0, 0, 4, 0, 2, 0, 2, 0, 4, 0, 4, 0, 1, 4, 1, 2, 2, 4], ['indicative', 'indicative', 'indicative', 'conditional', 'indicative', 'indicative', 'indicative', 'indicative', 'conditional', 'indicative', 'indicative', 'conditional', 'conditional', 'indicative', 'indicative', 'indicative', 'conditional', 'indicative', 'conditional']]
As Noah:[["As Noah's red nose grew redder with anger, and as he crossed the road while speaking, as if fully prepared to put his threat into execution, the woman rose without any further remark, and trudged onward by his side."], , ['indicative']]
If Noah:[["If Noah had been attired in his charity-boy's dress, there might have been some reason for the Jew opening his eyes so wide; but as he had discarded the coat and badge, and wore a short smock-frock over his leathers, there seemed no particular reason for his appearance exciting so much attention in a public-house."], , ['conditional']]
Mister Noah:[["'I'm Mister Noah Claypole,' said the charity-boy, 'and you're under me.", "Oliver, shut that door at Mister Noah's back, and take them bits that I've put out on the cover of the bread-pan."], [0, 0], ['indicative', 'indicative']]
Tone in sentences containing character (a negative-positive scale of 0-5):
0, 0, 0, 4, 0, 2, 0, 2, 0, 4, 0, 4, 0, 1, 4, 1, 2, 2, 4, 3, 0, 0, 0 Average: 1.4 (negative)
Mood combined with tone (indicative: fact or belief, imperative: command or warning, conditional: conjecture, subjunctive: wish or opinion):
0 (negative) indicative, 0 (negative) indicative, 0 (negative) indicative, 4 (positive) conditional, 0 (negative) indicative, 2 (negative) indicative, 0 (negative) indicative, 2 (negative) indicative, 0 (negative) conditional, 4 (positive) indicative, 0 (negative) indicative, 4 (positive) conditional, 0 (negative) conditional, 1 (negative) indicative, 4 (positive) indicative, 1 (negative) indicative, 2 (negative) conditional, 2 (negative) indicative, 4 (positive) conditional, 3 (neutral) indicative, 0 (negative) conditional, 0 (negative) indicative, 0 (negative) indicative
Occurrences in text: indicative: 16, imperative: 0, conditional: 7, subjunctive: 0
Postulation: The author uses overall negative factual and belief based sentences combined with overall negative conjecture based sentences to construct this antagonistic character in the story.
From here I’ll continue to hone the tool, expand the data on each character, and narrow the characters extracted from the text so that I can ensure the entities that are returned are more accurate.