A few months ago, I decided to take the plunge and start getting involved with development for the Steem blockchain. I have learned a lot through the process so far. In this post I will share some of my "lessons learned".

Blind Leading the Blind ;)

Please keep in mind that I am still very much a "newbie" when it comes to programming for the Steem blockchain. To a large extent, this post will be an example of the blind leading the blind :)

Still, I hope that there is at least enough useful information in here that anyone planning to work on changes themselves can learn from my experience and have a slightly easier time.

SBD Changes

I decided to work on two issues that were a high priority for many in the community, but Steemit was not working on due to other priorities. The changes I decided to tackle were: 2140 and 2022. The specific details of the changes are out of scope for this post, but for those interested you can read about them here: SBD Print Rate (2140) and Beneficiaries Payout (2022).

Building Consensus

Developing code for the Steem blockchain is quite difficult. I would go as far to say that building consensus around an idea is even harder.

If you are planning to develop a change, it is extremely important that you have sufficient support for your change in order for it to be accepted. If you develop something and there isn't support to accept it, then it will have been a waste of time.

Assuming the change will require a hardfork, then you will need at least 17/21 of the witnesses (20 primary plus one backup = 21) to vote "yes" for the change.

A good place to start is with a post (or series of posts) to discuss the idea with the community. If there is sufficient support, then a good next step is to reach out to various witnesses and let them know you are considering coding whatever the idea is, and find out their level of support.

If you want your code to actually make it to the "finish line" then it is your responsibility to get it there. That includes doing whatever convincing needs to be done in order to get it accepted.

Getting Steemit Development Team Buy-In

I know many people dream of a day when the community will be self-sufficient enough to code up our own hardfork changes, get them accepted by a majority of witnesses, and then get all the exchange + developer nodes to upgrade to the new version without Steemit's support - but the reality is we are not there today.

If you are planning to develop a change for the Steem blockchain, you are most likely going to need a lot of support from the official dev team. If you don't get their help, it will be very difficult (if not impossible) to get your change to the finish line.

A big key here is that you have to understand their role in the process, and be respectful of their time. They are already super busy working on the changes that they are required to do (Hardfork 20, SMTs, etc.) and whatever change you are planning to do will take time away from their required duties.

A good place to start is to open a GitHub issue to describe what you are planning to work on, and make it clear that you are planning to do the coding work. It is also good to demonstrate the amount of support that there is for the change.

Before you spend too much time coding, you should try to get some assurance from the dev team that they are on board, and will accept the change (assuming you code it properly).

Hardfork vs. No Hardfork

An extremely important part about developing changes for the Steem blockchain is determining whether a particular change will cause a "fork" - i.e. a change in consensus.

One way to think about this is that the blockchain data is basically split into two parts: there is the data stored in the blocks, and then there is the state data that is derived from the data in the blocks.

If you are changing the data that is stored in the blocks, then it is likely going to be a hardfork change. If you are only changing the state data, then it is likely not a hardfork change.

Handling a Hardfork

If your change is going to result in a hardfork, then it is important that it is coded in the correct way.

First of all, there are several changes that need to be made (in addition to the "actual" changes you are planning to make) just to make the hardfork to occur properly. This includes incrementing the version number (i.e. 19 -> 20), setting the time that the hardfork will occur, and including all of the necessary logic to trigger the hardfork at that time.

If you look through previous hardforks in GitHub, there are plenty of examples where this is done. (For the changes that I worked on, I did not complete this step myself - since I coded them to rely on the hardfork 20 logic that Steemit was already implementing.)

Once the framework for the hardfork has been setup, then all of your changes to consensus (what gets written in the blocks) need to be wrapped in proper conditional logic. What this means is that when the code executes it tests to see if the hardfork has occurred yet. If it has, it runs the 'new' code; if it has not, it runs the 'old' code.

Here is an example from my pull request that updates the print rate start/stop percentages after hardfork 20 occurs:

Here is another example from my pull request to pay out beneficiaries using the author's payout setting after hardfork 20 occurs:

Testing Changes

I found that testing to make sure all my changes actually worked as expected was actually significantly more difficult than coding them up. I won't be able to give sufficient information to explain how to test every possible change, but here is some general guidance on how to proceed.

Test Basic Node Functionality

Verify that a node can successfully resynch up to the current block, and continue processing new blocks with the changes applied.

Test ALL Paths

Find a way to get it to run through all the possible paths of the code. When coding a hardfork change this can be quite challenging without a dedicated testnet, as there isn't an easy way to test what happens after the hardfork takes place - which is likely the most important code to test.

One option is to launch a testnet. An alternative is to create "parallel code" that runs in addition to the "actual code" and compares what is currently happening (pre-hardfork) with what will happen (post-hardfork).

To give an example of parallel code: when I was testing the print rate changes, I setup additional variables to hold "temporary print rate" values. I copy and pasted the code that would run at hardfork 20, and had it set my "temporary print rate" variables to the "post-hardfork" values when the "pre-hardfork" code ran. Then while the pre-hardfork 20 code was running I could see what would happen once the hardfork occurred by looking at what was in the temporary values.

Debug Output is Your Friend

One of the main ways that I tested to see what was happening as the code executed was to add a whole bunch of extra debug output into the code. I set the debug output to whatever values I needed to see, and then scanned through the log file to see what the values were as the different portions of code were executed.

Here is an example of the logging syntax. It is fairly easy to copy/paste this, and then tweak it to output whatever information you need:

Automated Testing

In addition to the "live node" testing, it is also important to setup automated tests for your code as well. There is some information on their automated testing system here: https://github.com/steemit/steem/tree/master/tests with a bunch of examples inside the "tests/tests" folder.

You can run the automated tests by using the sudo docker build -t=steemit/steem . command.

It is extremely important that you run the automated tests and verify success before submitting your changes. If you don't and your change ends up causing a test to start failing, then your PR will not be able to be merged into the official repository, because it will show failing tests.

The first thing you will need to do is update any tests that may have broken as a result of your changes. After that, you should setup any new tests that are needed in order to verify that the outcomes you expect actually occur after the conditions that cause them to happen occur.

Here is an example of the tests that were run to ensure that beneficiaries payouts are split properly:

When you are running the automated tests, you may want to see debug output from those. The way to output debug info from the automated tests is using BOOST_TEST_MESSAGE. (There are plenty of examples of this in the code.)

In order for the 'boost test' debug output to actually show up though, you will have to edit Dockerfile and add a line: ENV BOOST_TEST_LOG_LEVEL=message.

Converting Values to Strings for Debug Output

One of the challenges that I ran into was trying to figure out how to get values that I wanted to see output to the debug logs converted to strings. Special thanks to @blocktrades for helping me to figure this out!

There are a few different tricks to get things to convert:

1. You may need to add .value to the end: db->get_account( "alice" ).reward_vesting_steem.amount.value
2. You may need to use the boost::lexical_cast conversion tool: boost::lexical_cast<std::string>(db->get_account( "alice" ).reward_vesting_steem.amount.value)
3. You may need to use the fc::json conversion tool: fc::json::to_string(gpo.current_sbd_supply)

(This may not cover every possible variable, but it at least worked for all the values that I wanted to output.)

Verifying You Are Running the Right Code

A few times I ran into a situation where I made changes and compiled, but then when I ran my code to test - it wasn't running the latest version.

There are steps that I used to ensure that I knew I was running the latest code:

• Make sure all instances of steemd are killed
• Delete the compiled executables that are there from the previous version:

sudo rm -rf /usr/bin/steemd
sudo rm -rf /usr/local/bin/steemd

• Update the capitalization of the "Transactions on block" debug output in libraries/plugins/p2p/p2p_plugin.cpp.
• Compile
• Run
• Verify that the Transactions on block debug output in the log has whatever weird capitalization I used (i.e. TraNsacTioNs On blocK).

I'm not sure if this is the right/best way to do it, but it got the job done :)

Miscellaneous Items

These are a few other random things that I learned.

Global State Variables for Parameters

Steemit is moving towards having a lot of the global constants defined in dynamic_global_property_object. Here is an example of how a variable is transitioned into that implementation: https://github.com/steemit/steem/pull/2571

Producing Blocks in Automated Tests May Break Pointers

When I was working on the automated tests, there was a line of code that I thought could be removed:

It turns out that it is needed, since when the db_plugin->debug_generate_blocks call is made, the pointers may become undefined.

Conclusion

Well, that's it :) I can't guarantee that anything in here is actually "good" advice, but I hope that if anyone else is planning to write code for the Steem blockchain, this guide at least helps.