Corpus Analysis ~ Monday, January 18, 2021

Although our soon to be ex-president has been expunged from Twitter, the corpus of his tweets can be downloaded elsewhere (thanks, Nate!).

I did some quick textual analysis (this is what I do for a living). I did this with a perl script that find words in a target corpus that are disproportionately frequenc compared to a corpus of general English text.

Originally, I did this to analyze the lexical proclivities of H. P. Lovecraft. The top 100 words from the HPL corpus are below.

get words from hpl.txt
get words from reference texts
word tgtfreq reffreq
gods 216 0
arkham 184 0
willett 169 0
nameless 167 0
curwen 161 0
amidst 137 1
madness 124 1
abyss 121 0
innsmouth 113 0
ghouls 110 0
ruins 101 1
akeley 100 0
grotesque 99 1
nightmare 96 0
legends 93 0
earth's 92 0
cosmic 90 0
oddly 87 0
peaks 86 0
curiously 85 0
accursed 85 1
blackness 84 0
asenath 82 0
carven 81 0
image 80 1
derby 80 0
laboratory 78 0
specimens 77 0
shapes 76 1
fragments 76 1
odour 75 1
prints 75 1
shocking 74 0
primal 74 0
vaguely 73 1
crumbling 72 0
abysses 72 0
towers 72 1
masonry 70 1
abnormal 69 1
ammi 68 0
morbid 67 0
tomb 67 1
miskatonic 66 0
gaunts 65 0
pickman 65 0
glimpsed 65 1
cult 64 0
visions 64 1
lore 64 0
ages 60 1
elwood 60 0
kadath 60 0
onyx 60 0
planet 59 1
capt 57 0
archaic 57 0
cyclopean 56 0
legend 56 1
necronomicon 56 0
stench 55 0
blasphemous 55 0
spectral 55 0
danforth 55 0
antique 54 0
carvings 54 0
shunned 54 0
pawtuxet 53 0
gulfs 53 0
hints 52 1
prodigious 52 0
whateley 52 0
jermyn 51 0
antarctic 51 0
blake 51 0
fabulous 50 1
sculptures 50 0
sane 50 0
allen 50 0
shewed 49 1
titan 49 0
piping 49 1
nyarlathotep 49 0
aout 48 0
slopes 48 1
galley 48 1
ward's 48 0
nahum 48 0
ulthar 47 0
debris 46 0
harbour 46 0
hideously 46 0
brooding 44 0
dimensions 44 0
edward's 44 0
slanting 44 1
furtive 43 1

get words from tweets_01-08-2021.justtxt.txt
get words from reference texts
word tgtfreq reffreq
trump 1742 0
fake 970 0
obama 714 0
media 689 0
biden 565 0
hillary 470 0
dems 454 0
endorsement 349 1
totally 332 1
fbi 325 0
maga 318 0
wow 310 0
usa 296 0
collusion 293 0
crooked 291 1
vets 268 0
mueller 250 0
congratulations 241 1
pelosi 240 0
obamacare 231 0
donald 222 0
iran 219 0
nancy 217 0
pm 216 0
hoax 207 1
rally 199 1
ratings 194 0
korea 190 0
guy 189 1
congressman 187 0
schiff 180 0
phony 177 0
golf 171 0
mike 169 0
healthcare 152 0
isis 151 0
comey 149 0
via 146 1
bernie 144 0
deals 137 0
rigged 134 0
coronavirus 131 0
makeamericagreatagain 130 0
cnn 129 0
scam 124 0
focus 122 1
cont 118 0
adam 115 1
dishonest 114 0
ukraine 114 1
obstruction 111 0
syria 111 0
schumer 109 0
covid 105 0
reporting 104 1
chuck 104 0
nbc 103 0
hurricane 101 1
agenda 101 0
wins 101 0
plus 97 0
iraq 96 0
testing 94 1
ted 94 0
drugs 94 1
israel 93 0
entrepreneurs 92 0
obama's 90 0
melania 90 0
apprentice 87 1
trump's 86 0
whistleblower 85 0
dossier 84 0
hopefully 83 0
rating 82 1
businesses 82 0
twitter 81 1
losers 81 0
dnc 78 0
nyc 77 0
haters 76 0
daca 76 0
usmca 76 0
jeff 75 0
continues 75 1
sanders 75 0
potential 74 0
mini 74 0
chris 73 0
ads 73 0
kim 73 0
lamestream 73 0
impeach 72 0
gop 71 0
phone 71 0
nato 70 0
nuclear 69 0
incompetent 69 0
relationship 69 0
mccabe 67 0
steve 66 0
supports 66 1
loser 65 0
television 65 0
today's 64 0
importantly 64 0
al 64 0
jeb 64 0
boring 63 0
patrol 63 1
infrastructure 63 0
emails 63 0
proven 61 0
aren't 61 0
winner 60 1
credibility 58 0
beach 58 0
biden's 58 0
talks 58 1
shutdown 56 0
putin 55 0
kevin 55 0
deliver 54 1
flynn 54 0
fisa 53 0
gotten 53 0
ny 53 0
congrats 52 0
doj 52 0
summit 52 0
dan 52 1
ukrainian 52 0
portland 52 0
disgusting 51 1
puppet 51 0
desperately 51 1
cuomo 50 0
anymore 50 0
interviewed 50 0
country's 49 0
protesters 49 0
terrorist 49 0
racist 49 0
mitt 49 0
pres 48 0
tv 48 0
responders 48 0
romney 48 0
biased 48 0
vaccine 48 0
weekend 47 0
puerto 47 0
brian 47 0
global 47 0
lightweight 47 0
eric 47 0
transcript 47 0
celebrity 46 0
mitch 46 0
nafta 46 0
ad 46 0
ms 46 1
jong 46 0
video 46 0
icymi 45 0
doral 45 0
strzok 45 0
mainstream 45 0
coverage 45 0
kelly 45 0
gov 44 0
polling 44 1
testify 44 1
terrorists 44 0
sanctuary 44 0
emergency 44 0
abc 44 0
lisa 43 0
shouldn't 43 1
luxury 43 0
talented 43 0
con 43 0
rebuilding 43 0
int 42 0
ebola 42 0
vegas 42 0
players 42 0
miami 42 0
las 41 0
rated 41 0
kag 41 1
reporter 41 1
medicare 41 0
vietnam 41 0
graham 41 0
terrorism 40 0
bipartisan 40 0
harder 40 0
kasich 40 0
americafirst 40 0
super 40 0
impact 40 0
refuses 39 1
focused 39 0
int'l 39 0
amazon 39 0
fracking 39 0
spied 39 1
shifty 39 0
mcconnell 38 0
football 38 0
ventilators 38 1
gdp 38 0
politically 38 1
bloomberg 38 0
russians 38 1
steele 38 0
rubio 37 0
el 37 0
kavanaugh 37 0
china's 37 0
brand 37 0
matt 37 0
sanctions 36 0
dc 36 0
games 36 0
turbines 36 0
fema 36 0
ok 36 0
pocahontas 36 1
info 36 0
nfl 36 0
sen 35 0
fan 35 1
saudi 35 0
reporters 35 0
deleted 35 0
rebuild 35 0
msdnc 35 0
oval 35 0
tech 35 0
thrilled 35 0
dopey 35 0
facebook 35 0
reagan 34 0
hearings 34 1
votetrump 34 0
bolton 34 0
weapons 34 1
premiums 34 0
classified 34 0
predicted 34 1
lou 34 0
tougher 34 0
transcripts 34 0
hosting 34 0
condolences 33 0
antifa 33 0
minneapolis 33 0
substantially 33 1
mccain 33 0
iranian 33 0
pme 33 0
transparency 33 0
cryin' 33 1
cia 32 0
jerry 32 0
fights 32 1
player 32 1
overrated 32 0
hardworking 32 0
wh 32 1
anthem 32 0
lyin' 32 0
arabia 31 1
deficit 31 0
illegals 31 0
hosted 31 0
pageant 31 0
memorial 31 0
turnberry 31 0
jon 31 0
wallace 31 0
bias 31 1
afghanistan 31 1
loopholes 30 0
rick 30 1
thugs 30 0
ensure 30 0
dow 30 0
campaigning 30 0
conflicted 30 0
businessman 30 0
draintheswamp 30 0
website 30 0
sooo 30 0
hispanic 30 0
negotiating 29 1
dallas 29 1
continuing 29 1
ryan 29 0
hits 29 0
elite 29 0
andy 29 1
looters 29 0
ronald 29 0
thanksgiving 29 1
homeland 29 0
trafficking 28 0
wi 28 0
airport 28 0
unbelievable 28 0
weren't 28 0
barack 28 0
pandemic 28 0
islamic 28 0
universe 28 1
brennan 28 0
commitment 28 0
tim 28 1
pols 27 0
spying 27 1
tape 27 1
ineffective 27 1
percent 27 0
google 27 0
leaked 27 0
timetogettough 27 0
starts 27 1
levels 27 0
celebapprentice 27 0
nc 27 0
comments 27 0
defund 27 0
kurds 27 0
cbs 27 0
bedminster 27 0
productive 26 0
medal 26 0
hates 26 0
brett 26 0
incorrect 26 0
dumbest 26 0
sean 26 0
ca 26 1
unfairly 26 0
lago 26 0
potus 26 0
clapper 26 0
mar 26 0
rallies 26 0
fl 26 0
purposely 26 0
levin 26 0
rebuilt 25 0
pete 25 0
mistakes 25 0
dorian 25 0
restaurant 25 1
auto 25 0
experts 25 1
msnbc 25 0
ivanka 25 0
terminated 25 1
incompetence 25 0
ig 25 0
championship 25 0
nadler 25 0
winners 25 0
garbage 25 0
vaccines 25 0
prescription 25 1
pastor 25 1
tuned 24 1
nh 24 0
anonymous 24 1
briefing 24 0
server 24 0
coup 24 0
fighters 24 0
hospitals 24 1
phenomenal 24 0
sham 24 1
ohr 24 0
tweets 24 0
editorial 24 1
guys 24 1
flights 23 0
opec 23 0
midas 23 0
tonight's 23 0
landslide 23 1
hillary's 23 0
humanitarian 23 1
knowingly 23 0
flu 23 0
doug 23 0
teams 23 1
investigate 23 1
ag 23 0
marco 23 0
investigating 23 0
embassy 22 1
omar 22 0
impeached 22 1
overtime 22 0
lowering 22 1
kerry 22 0
quo 22 0
cancel 22 0
amenities 22 0
penalty 22 1
tool 22 1
gee 22 0
congresswoman 22 0
ceo 22 0
carson 22 1
sc 22 0
obstruct 22 0
comeback 22 0
schiff's 22 0
cops 22 0
greg 22 0
tiffany 21 0
tightening 21 0
absentee 21 1
edwards 21 0
tweet 21 0
macy's 21 0
larry 21 0
update 21 0
upcoming 21 0
spa 21 0
paso 21 0
democrats' 21 0
aoc 21 0
probe 21 1
cohen 20 0
jr 20 1
danny 20 0
autism 20 0
nj 20 0
complaining 20 0
apologized 20 1
tampa 20 0
investigated 20 1
verification 20 0
fiction 20 1
stewart 20 0
asap 20 0
rino 20 0
options 20 0
quid 20 0
regarding 20 1
egypt 20 1
luther 20 0
fireworks 20 1
target 20 0
keystone 20 0
celebrating 20 1
crazed 20 0
hispanics 20 0
pundits 20 0

• Not surprising that trump's most distinctive word is "trump"!
• Kind of funny how obsessed he is with his enemies! He talks about them way more than his "accomplishments".
• Also funny that he talks about golf more than healthcare, ISIS, or coronavirus!