Hacking The Top 100 New STEEM Posts - First Text Mining Analysis for Fun & Profit!

Ever since I saw STEEM become #35,637 most visited website online today, I just had to take a look at what exactly was going on. Alexa is an Amazon-owned company that provides commercial web traffic data and analytics, and let’s just say they must be really impressed with STEEM.

But what I decided to do, is to go the extra mile. What exactly is going on INSIDE Steem?

So I wrote a little Python application which allowed me to extract a whole lot of data. For example, here are the top 10 tags sorted by the highest payouts.

Not surprisingly steem and introduceyourself are at the top, followed by steemit. But here is where we discover the first patterns - travel, life and photography are making the most money.

But you may be wondering, who is getting the most replies?

Interesting! Seems like the three forerunners are again at the top, followed by bitcoin, photography, and money. And it seems that photography and money have reached a healthy medium up until now.

But being the nervous ADHD brain that I am, I wasn’t satisfied by this cursory glance at the inner workings - so I decided to grab the Top 100 Trending Posts! Here is what I found out.

The average earnings for a Top 100 post are: $1881.4283168316824

Woah. That’s a lot more than I thought! What about average votes and responses?

Votes: 141.97029702970298

Responses: 33.59405940594059

What did the most successful “trending” post earn?: $15951.16

But this kind of data isn’t THAT interesting. What I didn’t tell you is that I also grabbed all the text from those posts. So let’s take a look at the most commonly used words in a trending title.

That looks about right, but most of these are common words that don’t tell us anything. What we have to do is get rid of those pesky stopwords (I, me, is, this, that). And a close look at the graph will show you that it’s only 20 examples and not 25 as the title says. Oops! Let’s try again.

Ahh, much better. “Looks like today, new Bitcoin posts on Steemit come first and have the most community importance, especially if they have some nice photography... This could get big. Better check your Ethereum accounts, northerners.”

But what about the trending posts themselves? What are the most common words there? Hold on tight, this is a big one.

“I would like one steem, people, and I’d also like to know about those good passwords…”

What I want to know now, is there anyone I should be jealous of? Which users have more than one post in the trending top 100?

Looks like simba and thedashguy are kicking ass. Unfortunately, I’ve read somewhere that katecloud has been hacked and can’t get the account back. Sorry to hear that, and I hope it gets resolved.

Which posts are getting the most votes, the most dollars, and the most responses?

What about the lengths of posts?

The average length of a trending top 100 post is: 496.35643564356434 words. The shortest is only 7 words long, and the longest is 4472.

Thank you for taking a look. Please upvote and comment if you can. I will be uploading a lot more content in the future.

P.S. If you would like to learn how to do this yourself, I recommend starting with the book Learning Python, or Python the Hard Way which both teach you the basics of Python programming. Then you can look at doing things like html requests, and driving a browser with Selenium. It takes a while, but it's worth it.