A new voter bot - newer, smarter, freer

in bots •  8 years ago  (edited)

TLDR;

I've made a bot you can use for free which is pretty dope. The idea is:

to automatically vote for posts that you would vote for anyway while also maximising curation rewards, bot by voting more regularly than a human can, and by picking great content which will probably be popular too.

You can check it out on GitHub, it runs as a Node.js server, designed for Heroku. There's a purple button you press to set it all up. Note, some assembly require but batteries included 🤓

There's a lot of features, but you'll have to read on find out... 😉

Voter - a new, smarter, freer bot

I haven't written a proper article on Steemit in 28 days. During that time, instead of writing articles, I've been using my free time to attempt to make a smart, free voter bot. I've named it Steem FOSSbot Voter, or Voter for short. 😁 FOSS stands for Free and Open Source Software. This means the source is freely available and open to (i.e. the license permits) modifying the code for your own purposes.

My original proposal for this idea is here.

There's already tons of voter bots and services, why do this?

All other well made voter bots I've seen are closed source, which means you cannot inspect the source or use it for yourself, and will be protected under a much more restrictive license. Most of these bots are commercial ventures, albeit often small scales ones, so there's often some cost, usually just casting a vote for the creators account. There's other bots that you can get added to, like a trail, or you can even pay someone to make a bot for you.

I want to make clear that I don't disagree with this approach or oppose these ventures (at least not simply because they are commercial). I do however believe in freely available software and I think voter bots are no exception. Steemit and the Steem blockchain are written as FOSS themselves, as well as countless other widely used software, so it's clear we all benefit from this approach, whereas only a few will benefit from the closed source model.

The second most different structural thing, is that you set up the bot to run as your own server, not on someone else servers, as is the norm. In stark contrast with any voting service, this means you do not give your posting key to another Steem user, your keys remain under your control.

Currently the bot is designed to work as a Heroku deployment. Heroku is a popular cloud computing service particularly popular with startups because it is free to use if you don't require much power. Thankfully our bot doesn't require that much and I believe it can be run entirely for free (though it's up to you to verify this, like any of my other claims 😅).

It should be possible to run the server locally, but I haven't tested it. It's next on my list, I'll be making installation instructions for this and making any changes to the code to make local install more simple.

I don't care about that, what does it do differently?

It's a pretty advanced bot, highly customisable, and allows you to tweak it in many different ways for your own needs and benefit.

Most of it is accessed through a basic but usable web app UI, accessible through your server URL. You can change the bot config, view logs and (my favourite!) stats about each time the bot was run, which votes were cast and how well it's doing for you.

It also sends you emails if you set that up, either every time the bot runs, or once per day.

Here's a few screenshots:

No, I mean how does it work?

Oh. 😅 Basically, Voter uses a scoring system for determining which posts to vote on. You use a web interface on your server to choose which aspects of a post you think contribute to a good post, and tell the bot how important you think each are.

There are a lot of aspects to choose from, which I call metrics, to borrow a term from analytics. They are each given short names, here's a few examples (there's 65 in total):

  • post_up_voted_num_whale: Number of whales that already voted for the post
  • author_is_followed: The post author is someone you follow
  • author_repuation: Author reputation
  • post_num_words: Number of words in the post
  • post_any_tag_blacklisted: Are any of the post tags are on the blacklist?
  • post_has_spanish_language_use: Post is (probably) in Spanish
  • post_images_only: Post is mainly just pictures (less than 100 words and several images)

Already you can see we are looking deep into the post and information about the post, i.e. metadata. We're performing a little bit of Natural Language Processing (NLP) and trying to expose data which may be an indictor of quality.

And really the whole point is that the bot allows you to decide how to define what is high quality using this data.

Strategic and cultural curation

In my bot proposal a month ago, I suggested there were two styles of curation, strategic and cultural curation.

Cultural curation votes for posts based on the content of the post and other related cultural, and thus social aspects. It has an indirect affect on payout, and thus curation reward, because culturally "good" posts often receive good rewards. At least that's the idea, if everyone does it, especially the whales. You think it contributes to the culture of Steem in a positive way, however you wish to define it.

Strategic curation votes for posts that will yield the best curation reward, regardless of content substance. Any user of Steem will know that the "best" posts do not always get the highest rewards, and sometimes "undeserving" posts get very high rewards. The existence of betting, games, competitions, porn, etc. etc. shows this clearly.

It makes sense though to consider both aspects, cultural and strategy. They interact with each other. You can do this with the bot.

How does that actually work

Each metric is a numeric value which is multiplied by a numeric weight set by you, to get a part of the score for a post. These scores are added together to get the total post score. If this score is above a certain value, called the threshold, the post is voted on. The threshold automatically moves itself based on the average score of the last few posts, adjusted for how many votes you want to cast per day (and a little higher if you wish).

We call a collection of these weights an algorithm. You can set this in the web app. There are also some white / blacklists that you can add authors that you like / dislike to, as well as tags and categories, keywords and website domains.

Creating a good algorithm

A good algorithm starts with a hypothesis about what makes a good post. This should ideally be a balance between what gets good payout, and what describes "good" content.

Example 1, long form articles

As an example, let's say I like long form articles about interesting subjects, I see they get good rewards and I think they contribute to the culture of Steemit. So I hypothesise that good long form articles have more than 200 words, some images, not too many videos and that the best ones are usually written by someone with a bit of reputation, they've been here a while. Also, they are in English, because I cannot read another language and so would not vote for any in another language naturally.

In that case I might set the following algorithm:

  • post_num_words in range 200 to 600, weight = -0.5
  • post_num_links_image in range 0 to 5, weight = 10
  • post_num_links_video, weight = -25
  • author_repuation, weight = 1
  • post_has_english_language_use, weight = 50

We're using ranges here in some cases to make the metric values more sensible. We don't care if there are more than 5 images, and don't want to over score based on that. This is similar for number of words and the reputation range. Also note that the post_has_english_language_use is a true or false, so if true then 50 will be added to score, if false nothing will. Metrics like this are clearly marks as Boolean in the list on the Edit Also page.

Example 2, artistic posts

Perhaps we're just interested in artistic posts. The hypothesis is that these kind of posts are pretty much just images with maybe a little text. The more images the better, the more words the less good.

  • post_num_words, weight = -1
  • post_images_only, weight = 500
  • post_num_links_image, weight = 20

So for every word we loose 1 score post. If the post is images only (less than 100 words) then we add 1000, which is a lot but will high skew the score up for these kind of posts which we're mostly interested in. Additionally we add 100 for every image.

Example 3, the sell out

Say we just care about curation rewards and want to maximise that, completely disregarding content. Our hypothesis could be that certain authors consistently get high curation rewards, so with our Steemit account we follow them and then use one metric only:

  • author_is_followed, weight = 1000

If the author is followed, their posts will always be voted on, up to our daily limit.

Reflection on the examples

The first two examples are contradictory in goals and represent two completely different ways of looking at quality. The interesting thing is that both of these kinds of posts might do very well and get good rewards, our hypothesis could be right in either case. What I like about them is that they are also heavily cultural.

The last example is very simple and might be effective, but it is not really intelligent. If the authors I follow stop producing content, or their content becomes poor quality, or even post too much, this will stop working. The problem with that approach is that it cannot react to changes in the system. However the first two methods will always look for different posts which fit the criteria.

In summary, I recommend you try to find out how to translate an idea you have about post quality that is a bit deeper than just who and what is popular now, because I've personally seen that Steemit is very dynamic and changes a lot, even from week to week.

Reflexivity with tests and stats

In order check if you algorithm is working you can perform a test run to get an idea of the scoring your algorithm and other settings will produce.

When the bot has actually run a few times, you can check the stats to see how things are going and adjust the algorithm accordingly. Perhaps your approach is not working at all, or maybe your weights are just off. It could be that one metric is contributing too much or not enough to the overall score. You can check this with the metric break down chart, pictured here:

I've used this and found it to be effective in improving algorithm design.

Okay okay, looks good but I'm not a coder

You don't need to be a coder to use this bot. There are detailed installation instructions, and as long as you follow them you should have the bot operational in half an hour.

To create your bot algorithm, you just need try to isolate what makes a good post and translate that into bot-speak. See the examples above.

Where people do find difficulty, I'd love to see people helping each other and figuring out how best to use it. It's a tool based on sound principles (I hope!) and even I as the creator don't know the absolute best way to use it!

Also, I will be hanging out on the # bot channel on Steemit.chat sometimes in the next while, hit me up if you've a question.

But bear in mind, this software is more or less "as is". I welcome criticism, bugs reports (logged through GitHub), contributions and free Steem / pizza, but do not expect to get customer level support and do not be disappointed when I won't built it for you.

This is a hobby project, a labour of love, and I just hope that it contributes a little bit to the community, puts some power into the hands of the dedicated, and leverages the unique property of Steem's curation rewards for everybody.

Thanks for reading!

As I said at the start of this post, I've been working on this project for about a month, and hope to get back to regular posting now. 🤓 I put in a lot of work so if you like it please consider re-Steeming this post and helping spread the word.

Also keep an eye out for @codydeeds, I believe he'll be looking at using the bot and might even do some tutorials.

And lastly, I've already written quite a lot into the docs for this project, which are located in the repository. I'll probably do a few posts explaining things more if there is an interest in the bot but there's a lot of information there if you can't wait.

😄 ✌️

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

@personz can we deploy the bot in our own computers ?? I believe you have discussed it above that you want to try that, do you make any progress about that ?? advance thanks for the response

Yes! I need to update the docs to make this more clear but you can run it in docker anywhere where docker will run.

Check out this guide: https://steemit.com/bots/@personz/docker-voter-automatic-easy-local-install-for-steem-fossbot-voter

Thanks for the response Sir 😊 appreciated 😊

Hello, interesting app
I'm testing it

give me these errors

setError to "stopped" , No new posts since last post and within MIN_POST_AGE_TO_CONSIDER of 30 minutes

and

at=error code=H12 desc="Request timeout" method=GET path="/run-bot" host=xxxxxx.herokuapp.com request_id=bxxxxxx-3xxx-4xxx-bxxxxd-63xxxxxxxxx fwd="xx.xx.xxx.xxx" dyno=web.1 connect=1ms service=30001ms status=503 bytes=0 protocol=https

in Heroku log

Thans you

  ·  7 years ago (edited)

Try increasing MAX_POST_TO_READ to 1000 or even 1200. You can change it in the Edit Config section in the web app.

It seems that version v0.3.0 works well

Thaks you

thanks, supercool app. Please consider adding it to SteemTools! http://steemtools.com

Done! Thanks, great service by the way 😊 👌

maybe a golos version? :)

Yes I think I'll do that at some point, I'd add a feature request to the GitHub now, you're the second person to ask

ping me when u do!

I've created this task. It won't be in the next days, but I hope to get to it at some point.

https://github.com/Steem-FOSSbot/steem-fossbot-voter/issues/4

Would it be possible to run this bot on Golos? ( it's russian version of Steemit under license of Steemit inc )

Looks like your comment got hit by asshole :/

and yours too :) waiting for my downvote :P

I'm so excited, @asshole finally gave me the honor of their down vote power too. :-)

I just neutralized @asshole's downvote....you should see my latest video that explores this theme in more depth.

Lol. I'll check that out. :-)

just ... don't ... mention .... the ... A ... word ... it's getting ... smarter ...

I thought @seraph was programmed to auto upvote his....

It is possible but not with the current code. It should be an easy migration though, you'd just have to swap the JavaScript Steem API library with the Golos equivalent library.

If there's enough of an interest I'd definitely consider making the change myself, or anyone can do it by forking the project themselves.

By the way, I created the Steem FOSSbot organisation, so any forks which keep the FOSS ethos are welcome to be added to it, to keep free, open software written for the Steem blockchain (and Golos too if that's agreeable) in one place.

🙂

I've created this task to be done sometime hopefully soon

https://github.com/Steem-FOSSbot/steem-fossbot-voter/issues/4

THIS IS COOL.

Did you get a server set up? 🤓

no, i was making a video

Ah sorry, no worries 🙂

W o W ! Thanks :)

No problem! 😁

Yes! Keep and eye out for me, going to be doing some big things! If anyone wants one of these setup and does not want to do any of the technical jargon hit me up in steemit.chat thanks!

technical jargon

😭 yes there's probably a bit too much of it. I'm interested to see how you express it without that.

Also feel free to make a pull request on the docs if you think you can improve the language!

It takes a team to get things done that is for sure, I am still finding my place in this super tech sphere but smack in the middle between common folk and you IT wizards is a good spot I think.

Absolutely! 🙌

@personz, sorry I am dummie about bot. Could I use this bot base on author name. I mean make automatic upvote base on author name ? thank you !

Hi 😁 Yes you can do that. You can whitelist the author and then use the metric called author_is_whitelisted. It would work similarly to Example 3 above.

Good post. I know next to nothing about using bots, but this was an excellent primer.

Are you worried that bots like this will eventually devalue Steem by upvoting content based on metrics instead of quality?

I'm not worried about that no. The main purpose of creating it is to put power into the hands of "normal" people who haven't so far had access to a versatile bot they can call their own. They can call it their own because it's kind of programmable, in the sense of being very customisable.

What I want to counteract are the promenant closed source bots and auto-voting services. A big thing is privacy and data protection (I'm all about that). Most (all?) of the services popular now require you to store your keys on someone else's server. This bot is basically your own server, you control it. It's certainly possible to run it all from your laptop, even with the web app, I just have to write the tutorial for it and work out any kinks.

The value of Steem doesn't even come onto my radar tbh 😬

One more thing, I do think that bots are problematic, and I think that an open source, easily available bot will hopefully spur discussion and either they will become nicer in the community or we'll have to do something about them. Making sure everyone has access to a high quality bot is a good way to promote that conversation.

Also I just like building stuff 😜

But the bots are meant to upvote for tokens, it's like being on a treadmill.

I'm not judging, I think it's awesome you can do this. But I think a system like this needs to find ways around these problems too. It's fine if everyone has a bot, but it does become a zero sum game eventually.

Thanks for giving the source code. I don't know much about coding, but it'll be fun to check it out.

I agree. But one has to be practical, it's fine to talk about the big ideas of the platform, but when there are other users empowered by something only selectively available, I feel like that is something to be opposed. And because it is in my power to do something about it, I did. Tokens can also be earned by witnessing and mining. I see this auto-curation as a kind of cultural mining.

I see it as contributing to the ongoing working out of finding ways around the problems which bot raise. I'd like to see where we go with this, but in the meantime, here's a bot 😊 One which has what I feel are good principles at heart, genuinely.

Well, I stand by my assertion that it brings the site down overall. But I can appreciate your logic. (No pun intended)

I disagree. It's like raising the level to competativeness for everyone, too a baseline that allows people to benefit more than they do now where power is concentrated in a few hands.

But in earnest I'd be interested to discuss ways around the problem of bots too. I believe in this approach for the moment but I'm not beyond convincing .

Thanks for the conversation, hope to continue it. 🙃

Cool. Nice work :) Resteemed.

The higher order of thinking. Upvoted followed and resteemed!!!

Grateful for your comment, thank you 🙂

I have tried to follow your instalation guide, but something make me worry . It's about payment confirmation. So what should I do ? :)

Indeed, it's seems unnessesary but it's how Heroku choose to wall off access to their add-ons. I mentioned this in the installation instructions in the GitHub docs for installation:

Also be advised that you may have to verify your Heroku account to use the add-ons which provide auto bot scheduling and email sending. Verifying can involve a bank card but at time of writing does not require any transaction to be made.

If this is not something you want to do, and I wouldn't blame you if you didn't, I'll be working on instructions to run this locally, and maybe even a standalone desktop app in the next months. You can probably get it working locally as it is now, but you'll have to figure out a few things and if you're not experienced it might be difficult.

Oh... that's okay for me, I will try it. I do not have credit card . I will try with my debit card. :) I will tell you if I success using my debit card ! :)

Cool! Please do familiarize yourself with their terms and conditions, I'm not connected to them in any way, not an employee, etc., it's just a convenient service. I can't be held responsible for anything that happens 😅

Great work dear friend @personz, hopefully people will give it a good use. Never missing a decerebrado using the bot to create internal conflicts in the platform, a few days ago was discontinued one for these reasons.
I inform him in a post Mr. @timcliff.
Congratulations on the great work and thank you very much for sharing

Thank you very much 😃 Glad to share, freedom and openness are fundamental to this 🎉

@personz hi..!! iam trying to run the bot but i occurd this error.!! can you help me?

Untitled.png

Not really without some more information. Do you have the Heroku CLI installed? If not, install it, then type in this command into your terminal:

heroku logs --num 200 --app YOUR_APP_NAME

Obvious replace YOUR_APP_NAME with your app name, and sanitize anything you post here by removing any personally identifying information.

I did what you told me . The error i see in my terminal is about a api key. i have done everything correcly following the guide you made. maybe i missed something :/

I need to see the logs. Please open a support ticket on the GitHub project

This may be the result of the Heroku free quota running out too

The @OriginalWorks bot has determined this post by @personz to be original material and upvoted it!

ezgif.com-resize.gif

To call @OriginalWorks, simply reply to any post with @originalworks or !originalworks in your message!

Great post. Upvoted Resteemed and following. Cheers!

Thank you! 😎

  ·  8 years ago (edited)

This is amazing, thank you so much!

One question I have, it only seems to go back so far in the Steemit history, is there a way to make it go back further, like for an initial run?

Edit: One glitch I found: Since I wanted to try a simple vote bot at first, I followed the steps for author_is_followed, the example says to use 1000 as the threshold, however the bot defaults to 1500 as a minimum so nothing happens.

"1000 < 1500, WILL NOT vote on post"

Thanks! 😄

Any time the bot runs it will fetch a maximum of 100 posts from the newest posts. This goes for the first run too, so this is probably only a few hours of posts. Do you think this should be extended on the initial run?

On your glitch, the reason why the threshold is 1500 is because the number of votes in the last 24 hours is above the limit. You can change the limit in the Edit Config settings, or just wait until there are less votes in the last 24 hours.

There's a bit of a missing feature in the statistics in that it doesn't show you clearly how many votes you've made in the last 24 hours, so it can be hard to know if you're over the limit. This is a planned feature tracked here.

  ·  8 years ago (edited)

Maybe an option for a "long run" ? Like on the main menu or something.

I've increased the number of votes per day to 100 - I don't think I was anywhere near this but even a weight of 10000 wasn't enough.. XD - I still can't get it to vote reliably.

I just wanted to use this as a simple vote bot since I can't read Steemit every day.

I'm sure I have some setting wrong but I don't know what that would be.

Cool, I'll consider how best to implement that long run idea, thanks for it.

I can try to help you figure it out better if you hit me up on steemit.chat in the #bots channel. Should be able to figure it out together 😁

I had a bot voting on my blog yesterday https://steemit.com/art/@todaysrant2017/painting-comes-to-life-big-bess-from-fable-3-sings-feeling-good-happy-valentines

there is a war lol on it said

This is a live attack on my accounts @iloveupvotes right now by @berniesanders

The comment spamming bots are very annoying but I wouldn't worry too much, the reputation of those accounts is very low.

Calling it a war is just a way for them to give it some kind of legitimacy and drama, it's just a temper tantrum 😂

Check out these posts if you want to know more, but it's kind of boring. I assure you, the voter bot I've made is nothing like these (it's up vote anyway, no comment posting)

https://steemit.com/spam/@berniesanders/why-i-m-flagging-iloveupvotes-time-to-bot-flag

https://steemit.com/writing/@klye/why-having-temper-tantrums-and-being-anti-social-in-an-attempt-to-make-your-point-doesn-t-work-on-steem

Lol its all new to me , very big learning process and a very different way of thinking, i love steemit ... thanks the links , i will share them on my steemit blog paper :D https://paper.li/RetroSheep_com/1486947569#/

Amazing skill you have to create the algorithm, and you articulated the concept thoroughly. Thank you for helping our objectives in a strategic constructive way for the Steemit community.

Thank you! Happy to help 😇

It looks a great tool. I will have a deeper look tonight, but from your post, I really like your work! :)

Very kind of you to say 😀

This looks fantastic, thanks @personz

  ·  8 years ago (edited)

Thanks for having a look 😊

Whoa, thanks so much, this works great! :-)

Glad you like it! 😆

Seems interesting! I'll have to test it out later :)

Please do! Let me know what you think

Does a cron job need to be setup to run automatically?

Yep, but you should use the Heroku equivilent which is called Heroku Scheduler. It's covered in the installation guide here

If you've been brave enough to try a different config, on a local machine or different server, let me know too!

Cool, I found where to setup the scheduler. Any reason to not have it run every 10 minutes on the free plan? I've got it auto-voting my followed users.

  ·  8 years ago (edited)

The only reason is that it might be too much processing for a free account. That's 144 computations per day, with plenty of HTTP action to fetch data and make votes. Compare that with 24, I'm not sure if you'd still be inside the free account plan limits.

Scheduler add-on runs one-off dynos that will count toward your usage that you will be charged for each month.

But then again it might be fine, I haven't checked. If you find out let us know! I've tested things a lot but I've definitely missed some areas.

EDIT: referenced the Heroku Scheduler docs

Comment depth limit reached, so posting here in reply: I have your github repo watched / starred / bookmarked. I'll contact you via the 'issues' tab after I have some feedback to send your way. Thanks again! :-)

Perfect, that's exactly the workflow I prefer 🙂 Thanks again

That's what I figured. Thanks for the quick responses. I think I'll upgrade to hobby level, and keep you in mind next time I fire up my clean machine's steemd instance for a tip! :-)

Always appreciated! I'd be really interested to hear how you get on at hobby level, will look out for a post or some such in case you're inspired 😜