Background
A blockchain is a type of distributed database, where all of the nodes in the network use a set of rules to agree on what data should be in the database. These rules are called "consensus rules". These rules are important, because they prevent the nodes in the blockchain from having inconsistent sets of data.
A common example of why inconsistent data would be bad is called "double spending". If user A sent 1 token to user B, and then user A tried to send the same token to user C - it would be really bad if the nodes did not all agree on who had the token. Both user B and C could be tricked into thinking they had the token.
Blockchains are state-based databases. A "block" is a set of transactions that update the state of the database. The data in the database only changes when a new block is produced. During the time between blocks, the data in the database is static (it doesn't change). Whenever a new block is produced, it gets distributed to all the nodes in the network - which then validate whether or not the block is 'valid'. If the nodes consider the block valid, then they update their state.
Every new block that is produced is connected to all the previous blocks. This is why blockchains are called blockchains (chain of blocks). Whenever the nodes in the network validate a new block, they also validate that the previous blocks in the chain are valid too.
Forks
When two (or more) nodes in a blockchain do not agree on which data is correct, this is called a fork. In order to preserve the integrity of the network's data, the nodes must reach an agreement on which set of data is correct.
Typically the way this is handled is by a method called "longest chain". Each version of the inconsistent data will have a set of one or more blocks that makes it different than the other version(s). The next time a new block gets produced, the block producer will have to choose which of the chains to append the next block to. Eventually one chain will become longer than the rest. The longest chain becomes the 'valid' chain, and the rest are considered invalid.
Miners vs. Witnesses
Different blockchains have different protocols to determine who produces the next block.
In Proof of Work (PoW) blockchains (such as Bitcoin and Ethereum), the block producers are called "miners". Miners compete to solve complicated math problems that can only be solved with heavy computing power. Whoever solves the math problem first, gets to produce the next block.
Steem uses a protocol called Delegated Proof of Stake (DPoS), which is much more efficient than PoW. In DPoS, the token-holders (users who hold Steem Power) get to elect the block producers. They do this by voting. The block producers are called witnesses.
Steem Block Production Assignments
The Steem blockchain automatically assigns blocks to the witnesses. Whichever witness is assigned the block is responsible for producing it within the given time window. Blocks are spaced 3 seconds apart. Every 63 seconds, the Steem blockchain will produce a "round" of 21 blocks.
20 of the 21 blocks are produced by the 20 witnesses who have the most votes (measured in Steem Power). These witnesses are called the "top 20 witnesses".
The remaining block is randomly assigned to one of the remaining witnesses (called "backup witnesses"), with a probability equal to their percentage of votes (again measured in Steem Power) relative to the other backup witnesses.
Double Producing
The term "double producing" refers to a block producer creating two different blocks, and then distributing them both to the nodes in the network.
Double producing is really bad, because it creates a fork. Until all of the nodes in the network can agree on which of the two blocks is to become the 'longest chain', there will be two different versions of data floating around.
Double producing may also cause other witnesses to miss blocks, if the block they produce ends up on what becomes the shorter chain.
If an application or end-user makes a decision based on data that ends up being in the shorter chain, that data may become invalid after the longest chain is formed. This can be really bad if important decisions get made based on this invalid data.
Malicious users could potentially exploit this by attempting to double spend.
Punishment for Double Producing
In the 'early days' of Steem, the consequences for doing this were extremely severe. Anyone who caught another witness double producing could literally take all of the Steem Power from the double-producer's account.
The method that allowed this ended up getting turned off because it turned out that there was a potential exploit, but the fact that it was there should indicate how serious this is.
Also, witnesses should be extra vigilant when doing maintenance, migrating/adding servers, generating new key pairs. Although they may not be malicious, the common mistake is to use the same key pair on two servers, which results in double producing.
So, always disable your witness when doing maintenance by broadcasting this key
STM1111111111111111111111111111111114T1Anm, and triple check that the keys are different on every node. Re-enable the witness with the main server node's key when all is good and keep an eye on the logs until the next produced block is OK.Here's an example of a double production (collision):
1560475ms th_a database.cpp:538 _maybe_warn_multiple ] Encountered block num collision at block 22296845 due to a fork, witnesses are: [["steem-bounty","2018-05-10T03:26:00"],["steem-bounty","2018-05-10T03:26:00"]]Note that if the collision happens between two witnesses, it's often due to a temporary lag, for example:
28220ms th_a database.cpp:538 _maybe_warn_multiple ] Encountered block num collision at block 22214744 due to a fork, witnesses are: [["netuoso","2018-05-07T07:00:24"],["xeldal","2018-05-07T07:00:27"]]Notice the timestamps with exactly 3 seconds difference.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is an excellent explanation of "Double Block Producing" and much easier to understand now..
I learned something new from this also, I never knew in the early days that SP could be taken from a witness for double producing.. Very interesting..
I can see how that illustrates just how important it is to not double produce blocks.
Thanks for this @timcliff
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
First of all, I have to say that I am extremely happy that I decided to follow you. The first time that I met you was during your post on raising the dust vote amount. So, I'll just come out and tell you the truth, I really only followed you so that I can keep up with any more crazy ideas that you might have. Now, however over the course of the last week or so, I am really beginning to realize that you're doing the best you can try to think of ways to solve problems here. Not only that, but you're probably one of the smartest guys and I've met here and my short time. So, I would like to thank you once again for what you're doing. And like I said last time, you deserve to be paid for your work, so I am hoping that you will accept payment for this post. And I have been thinking about this since the last time. You could take, say half, or any amount you decide, and just hand it out to people in the community who you deem Worthy or that need it. Or however you would seem fit, it's your money. Just an idea to throw out there. Now, on to this post. This is probably the best, simple, explanation of the blockchain that I have ever heard. And I appreciate you for the simplified explanation that you offer. I also think that it is a great thing that you explain the potential issues with double producing. And as far as how easy it is to be able to become a witness due to the simplified processes, I never realized that this was the case. To me, I agree with you, and I think that this is a serious potential for abuse. And while they removed the old punishment, have they not installed any new form of punishment for double producing? And also, is there any way that you guys(witnesses) keep track of this, so that if it does begin to happen it can be stopped quickly? Or is it something that potentially could be hidden until it caused a serious Fork? I ask you these questions because lately as I have gotten deeper into the steemit "behind the scenes" there are a lot of things that concern me. When I first arrived here, I thought everything was happy-go-lucky. I now know that this is not the case. I have enough concern that I have stopped investing my own personal money, and have only been gaining money and SP here through authoring and curation. I appreciate any response that may be given in regards to my questions. And I would like to tell you to keep up all of your great work. I think that you were doing good stuff here and I believe that you deserve encouragement. So thank you so much again for what you're doing and I hope that you have a wonderful week. As always I send you and all of yours all my love until we meet again.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Haha, thanks :)
As far as the punishment for double producing, it is an easy way for a witness to loose witness votes. It is pretty easy for other witnesses (and anyone else who is running a node) to detect it, as it shows up in everyone's logs.
The majority of the time, it just ends up with a "micro fork" - which is basically a fork that lasts one block. As soon as it is the next witness's turn, they pick one of the blocks, and everyone follows. While it is still possible for someone to get tricked in this short period of time, it is a fairly common practice in cryptocurrency to at least wait a few block confirmations before considering something "final" - which should protect against this as long as users/apps are following this best practice. (Some still might not though, which is why it is still best to avoid even a micro-fork as much as possible.)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you sir.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
The whole concept of Witness work involves not only votes and SP weights, but some common ethics to help the platform rather then acting maliciously. The unfortunate event of 'trading / begging' etc for Votes makes a ground for ill intended people to achieve high positions in Witness list.
Lately I noticed many votes coming towards my Witness, that simple got removed after a couple of days. The way i see it, is because of not approving back. (and I am not approving back, because too many missed blocks / compared to position as a main reason).
I would only say it's our mistake for offering tons of simplified solutions suck as docker allowing everyone to become a witness, then execute different strategies (of questionable ethics) to claim high.
As simple as that, someone with enough knowledge to compile and run nodes from scratch is assumed to have enough knowledge to make money outside the scope of abusing the platform.
Witness work is not for everyone. Making it easy for everyone to join the race will only bring trouble.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I would agree in a sense, being a witness is not for everyone but it should not only be for the "tech savvy either" we just recently set up our witness as a project because we are 2 steemians and someone technologically inclined and we are a fan of steem... do we really know the ins and outs? no we do not but we are learning... We had our first block yesterday, and missed it but I do not know if it was due to this double producing. So now we have shut off the server and rebuilt the whole build from scratch instead of following a manual.
So I agree for you there... the votes should come from community work, and running a good service and not from tit for tat...
We are learning on the way. But that would also be why we start lower and then have a chance to build up votes
our witness is @swisswitness
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Well, that's something totally different and totally ok, if someone is learning with a goal to achieve full understanding of the platform and provide great service.
Few months ago, I offered "paid witness setup", but these who contacted me willing to pay for such work, got it totally fee (along with support). I am glad some of them are successful, still doing their work. The 'payment' was just a trick to distinguish who is serious and who is not. They got a full training and explanation to any questions they had.
I have nothing against a total newbie who is willing to learn, and I will always support that by all means and sure others will too. But the real problem is that some people are pushing way more into campaigns / trading votes / begging for votes, rather then understanding technology. At the end of the day, they do rank better, and platform suffers.
I don't think anyone needs to have PhD in computer science to be a witness, the only requirements is to take it seriously, act ethically, learn new things, and support others.
Lucky enough most of witnesses we have are great people. But if you want to distinguish others, just check how many of them become inactive whenever SBD goes down on the market, and then get back once it raises.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is why i follow you, i will have to read that again to fully understand it though lol...
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Woah ! I definitely have to read this more than once. Cue airplane over my head :) So much to learn. What a layered beast all of this is.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
hi @timcliff, as a new witness I have so much to learn and thankfully people like you educating. found the post very informative and easy to understand (for the non tech. like me) and some of the comments also added value
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you @timcliff, you have described a very important issue in a way that everyone understands, it is very valuable. I would like to ask for more ;-)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Wow... I just can’t hold that wow. This is amazing , this is comprehensive , It felt like reading a white paper . Thanks for this much education and information.
I like the sound of this. I look forward to further courses
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That was somehow a good recap of the blockchain consensus and double spending problem, and same time a good lesson of history of Steem.
I just tried to contribute to Steem github and saw your comment, thanks for the review! If you have time, could you give me some information about how (or if) should jenkins be ran on external branches?
Sorry to bother you, and thank you so much!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Replied to your question in a different thread, but that is not something I am familiar with.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Oh, thank you so much, sorry for asking twice, I didn't thought you will be checking that comment with all the activity here :D
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
No worries :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Anyway, I have updated the PR after your comment, if there is anything you see wrong about it, I appreciate your input!
And since in your PR about guidelines you introduce the branch stable, I wonder if the new PR should be opened against develop or master.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
It is a good question, and I’m not entirely sure. I always submit my PRs to master, but maybe I’ve been doing it wrong all this time :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Okay, I will follow your example for the moment, if I will be able to open any other PRs and same time I will keep an eye on the guidelines. Thanks a lot for your answers, I was worried about both of these things pretty much, now I feel better!
In time I will ask you other questions about how do you test the developed features, but I don't want to take all of your day/night so for the moment, thank you again!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I should mention, I'm not one of the 'official devs'. You may want try asking your questions in here: https://discord.gg/B29Bbng. I know some of the Steemit developers hang out there and answer questions on a periodic basis.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you @timcliff for such a comprehensive explanation focussed on Double Producing.
Few questions,
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
There is a method for it in the code. It is deprecated though.
I believe PoW has the same problem, but it is less of a big deal because pretty much everyone waits for multiple confirmations with PoW tokens.
Each node just doesn’t make any changes to the data in its own database until it receives the next block.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank You @timcliff,
Sorry, an off the topic question,
what does it mean by single-threaded and multi-threaded in blockchain and how it affects the tps?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
In simple terms - single threaded means it can only do one thing at a time. Multi threaded means it can do multiple things at once. Multi threaded obviously allows more transacting.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you and please keep on writing more about blockchain, especially about graphene and DPOS .. I have set GINA to stalk you, looking forward to more info @timcliff :)
And I have gone through your blog, you seem to have very active participation in STEEM, I just wanna know if 'one account, one vote' will be a thing just for SMT or we will be seeing a fork for STEEM. Cause if it's just limited to SMT one of my posts becomes obsolete :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Currently it is just a concept being discussed for SMTs.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you for the indepth yet easy to follow explanation of blockchain and witnesses. I have a much better understanding after reading your post. Double producing definitely sounds like it could be a serious issue. If the ability to remove the SP of someone one guilty of this is turned off, how is a witness who gets caught penalized?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
They can loose witness votes if people unvote them for double producing.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That would certainly be the obvious answer, though that alone does not seem like enough of a discouragement to stop some witnesses from attempting to get away with it anyway..
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
You have a minor grammatical mistake in the following sentence:
It should be to choose instead of to chose.Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This was extremely well written and explained lay-person style. Not a lot of devs can do that, nor do they take the time to try. Thanks!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Very enlightening I must say. The system is way more complicated than the "blogging platform built on a blockchain" front and there's a lot of research and programming skill required. Does the fact that steemit blockchain has a more advanced mining protocol than most mean that the value is inevitably going to rise sky high?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
It is technically the "Steem blockchain" (Steemit is built on top of it). I think there is a huge amount of potential here, and I think we have one of the best blockchain projects in existence. Only time will tell if the market will see value in it.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
woo this is very educating, to get the complete information i think i have go through it over and over, to many sheet story about this mining this this is helpful thank for the information.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Very interesting information with very good graphics thank you friend for this great contribution good greetings
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Good and short explanation. Thank you @timcliff
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Wow, very explainable and easy to understand. Thanks for sharing
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
hello, sir how can I catch you on discord channel? we have one channel with 700 members we are trying to make nice project and help to steem community and we have the nice idea for it and we need you to be the witness of our project and we need your support is our discord channel https://discord.gg/3HZdaGk we will explain for you about our project and we are trying to make introduction post about and we need your advice
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I am user TimCliff#9168. Just send me a message :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
sir i couldnt send message its block because im not your friend in discord you i need to your accept my friend request thanks sir
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Really? I get messages from people that I have not friended all the time.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Can you join the PAL discord channel, so then we are in a common sever? It should let us chat after that.
https://discord.gg/nbvTah
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Hi Tim, I know I.m on steemit for quite some time now but it.s first time i saw your interview and I just want to say thank you and i agree 100% with what you said because that.s what I discovered too in my steemit journey. Here is the link towards your interview which I saw https://steemit.com/blog/@terrybrock/steemit-success-series--timcliff-on-who-to-target-with-your-content-1512178826-124805 , thank you again and i hope our paths will cross one day. I wish you a great day and much luck! Alina
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Glad you enjoyed the interview :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That's interesting, it's really cool to see how different steem and bitcoin are! I really feel like steemit, and the steem blockchain is much better than bitcoin's.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Hello, I have followed you and got votes on Steemit, be kind enough to follow me. Let's create a great community. Thanks & Best Regards
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit