Adding Retry Logics and Failover in Voting Services

in steem •  9 months ago 

I haven't revisited the code of voting services for a long long time since most of the time - it works. Today, there are some problems of the nodes, and that is why I decided to take some time to improve/fix it.

The voting services consist of several components but as you can imagine, it is decoupled, it works as something like this:

  1. there are several processes (components) that monitor the posts of delegators and good authors. Once criteras are met, the posts are pushed to database.
  2. the reader process checks periodically the database for new posts to vote, and then vote them one by one.

The problem is that sometimes the node API call failed, like this:

image.png

The error is Internal Error which doesn't give enough information of what went wrong here.
** getDiscussionsByAuthorBeforeDate Error [RPCError]: Internal Error **

Since this is scheduled to run every 10 minutes (as a crontab job), usually a next retry would work. But if a node is down for some time, then it will cause the problem.

Thus, i have added the retry - and also failover logics, so it will choose next RPC node and then try again 3 times before skipping.

for (let tc = 1; tc <= 3; tc ++) {
    console.log(`Try process counter ${tc}...`);
    if (await processForUser(user, totalDelegated)) {
        break;
     }
     failover();
     await sleep(3000);
}

It is not perfect, but it is better, and it works for now.

Steem to the Moon🚀!

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  
  ·  8 months ago 

sure, voted you here !thumbup

Hello @dorian-lee! You are Magnificent!


command: !thumbup is powered by witness @justyy and his contributions are: https://steemyy.com
More commands are coming!

Thank you for the job and for working and improving the code when needed :)

Steem to the moon

I have just delegate 5,000+ SP to @justyy

  ·  9 months ago 

Yeah, I noticed that some problems with the nodes, I am still investigating, but voted you already on the post. Thanks !thumbup

Thanks!!

Hello @puncakbukit! You are So Nice!


command: !thumbup is powered by witness @justyy and his contributions are: https://steemyy.com
More commands are coming!

Awesome that you updated the code, but doesn't seem to be working properly just yet, at least you haven't made any upvotes for several hours, and my most recent post didn't get one either...

  ·  9 months ago 

Fixed.

Thanks ! :)

  ·  8 months ago 

Fixed