STEEM Internals #13: How to increase STEEM test coverage. Full example: escrow payments.steemCreated with Sketch.

in steem •  8 years ago 

Escrow payments are very important for anyone that would like to do a serious business over blockchain that includes a third party protection. There are numerous advantages of the solution and I can see it being applied in many reliable applications that must include escrow agents.

The concept and official 'STEEM improvement proposal' was opened by @dan at https://steemit.com/sip/@dan/escrow-sip-steem-improvement-proposal and resolved at Hardfork #14.
I noticed there were a few issues still open around that area on STEEM GitHub page and had a look at tests. The STEEM tests around escrow implementation have one the best coverage in STEEM codebase. Whoever tested it did a good job and with a little effort we will achieve 100% statement coverage and later can think of state transition coverage.

In this post I will show you how to build and unit test STEEM blockchain using a clean development environment and extend the test coverage.

Preparing clean build and test environment for unit tests.

Clean build and test environment is an important prerequisite for reliable test results. You must make sure that you have the same results no matter if you are on Mac, Linux or Windows. I will be using Docker containers for that. Feel free to use a solution that suits you best. In scope of this post STEEM will be unit tested on Ubuntu 16.04 based docker image which is the default for STEEM.

Dockerizing build and test.

The docker image I will use contains vanilla Ubuntu 16.04 LTS and STEEM build dependencies.

FROM ubuntu:16.04

RUN apt-get update
RUN apt-get install -qy build-essential git libssl-dev m4 yasm autogen \
                        automake libtool doxygen python3 python-dev \
                        autotools-dev libicu-dev libbz2-dev cmake g++ \
                        wget pkg-config libgmp-dev libboost-all-dev lcov

ENV cname_flags=-DBUILD_STEEM_TESTNET=ON -DLOW_MEMORY_NODE=ON \
                -DENABLE_COVERAGE_TESTING=true -DCMAKE_BUILD_TYPE=Debug

WORKDIR /usr/local/steem/src

COPY build.sh /usr/local/steem/
COPY test.sh /usr/local/steem/

As you can see I created two shell scripts and uploaded them to the image. First is used to build STEEM in debug mode with test network, second one to execute unit tests. More on that below.

build.sh

Build.sh builds STEEM in testnet mode with flags provided by the container.

#!/bin/bash

set -e

cmake ${cname_flags} .
make -j4
make install

test.sh

Test.sh runs unit tests.

#!/bin/bash

set -e

lcov --capture --initial --directory . --output-file base.info --no-external
tests/chain_test
lcov --capture --directory . --output-file test.info --no-external
lcov --add-tracefile base.info --add-tracefile test.info --output-file total.info
lcov -o interesting.info -r total.info tests/\*
mkdir -p lcov
genhtml interesting.info --output-directory lcov --prefix `pwd`

Building the Docker image

Is easy as it only requires to run...

docker build -t steemd-tests .

Workspace checkout

This seems to be a tricky part to dynamically be able to update tests and use any IDE outside the container but it becomes ease with Docker volumes. I am preparing workspace on host OS and exposing it to the container as a Docker volume.
First checkout the workspace locally including all submodules and then bind it to /usr/local/steem/src folder. Docker run -it command...:

$ docker run -v /sandbox/steem:/usr/local/steem/src -it steemd-tests
root@0235753685df:/usr/local/steem/src#

Brings you to the Docker container shell. The workspace on /sandbox/steem is exposed as a volume at /usr/local/steem/src.

Running tests and evaluating results

By having the scripts in place build and test steps are extremely easy.

Build step

root@0235753685df:/usr/local/steem/src# ../build.sh 
-- BUILD_STEEM_TESTNET: ON
--      
--              CONFIGURING FOR TEST NET             
--      
-- LOW_MEMORY_NODE: ON
--      
--              CONFIGURING FOR LOW MEMORY NODE      
--      
-- Using custom FindBoost.cmake
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   thread
--   date_time
--   system
--   filesystem
--   program_options
--   signals
--   serialization
--   chrono
--   unit_test_framework
--   context

...

Test step

root@0235753685df:/usr/local/steem/src# ../test.sh

...

Processing file libraries/wallet/wallet.cpp
Processing file libraries/wallet/api_documentation.cpp
Processing file libraries/wallet/include/steemit/wallet/api_documentation.hpp
Processing file libraries/wallet/include/steemit/wallet/wallet.hpp
Processing file libraries/wallet/include/steemit/wallet/reflect_util.hpp
Processing file programs/build_helpers/cat-parts.cpp
Processing file programs/cli_wallet/main.cpp
Processing file programs/js_operation_serializer/main.cpp
Processing file programs/size_checker/main.cpp
Processing file programs/steemd/main.cpp
Processing file programs/util/inflation_model.cpp
Processing file programs/util/sign_digest.cpp
Processing file programs/util/get_dev_key.cpp
Processing file programs/util/sign_transaction.cpp
Writing directory view page.
Overall coverage rate:
  lines......: 10.3% (3188 of 30945 lines)
  functions..: 15.2% (4049 of 26669 functions)
root@0235753685df:/usr/local/steem/src#

Results

Open browser and point to your workspace root / lcov/ index.html

Escrow code coverage.

Let's check escrow...looks really good but I found an empty spot:

coverage_gap

As you can see tests only execute scenarios where the escrow fee is denominated in STEEM, let's add a new test that checks STEEM Dollars as well...

--- a/tests/tests/operation_tests.cpp
+++ b/tests/tests/operation_tests.cpp
@@ -3729,6 +3729,15 @@ BOOST_AUTO_TEST_CASE( escrow_transfer_apply )
       tx.sign( alice_private_key, db.get_chain_id() );
       STEEMIT_REQUIRE_THROW( db.push_transaction( tx, 0 ), fc::assert_exception );
 
+      BOOST_TEST_MESSAGE( "--- failure when fee in SBD and account cannot cover cost" );
+      op.fee = ASSET( "10.000 TBD" );
+      tx.operations.clear();
+      tx.signatures.clear();
+      tx.operations.push_back( op );
+      tx.sign( alice_private_key, db.get_chain_id() );
+      STEEMIT_REQUIRE_THROW( db.push_transaction( tx, 0 ), fc::assert_exception );
+      op.fee = ASSET( "0.100 TESTS" );
+
       BOOST_TEST_MESSAGE( "--- failure when ratification deadline is in the past" );
       op.steem_amount.amount = 1000;
       op.ratification_deadline = db.head_block_time() - 200;

Rebuild...

...
Scanning dependencies of target chain_test
[ 95%] Building CXX object tests/CMakeFiles/chain_test.dir/tests/operation_tests.cpp.o
[ 95%] Linking CXX executable chain_test
[ 98%] Built target chain_test
[100%] Built target plugin_test
root@0235753685df:/usr/local/steem/src#

Rerun and verify...

./chain_test -t */escrow*
Random number generator seeded to 1475007089
STEEMIT_TESTING_GENESIS_TIMESTAMP is 1431700000
Running 12 test cases...

*** No errors detected

Rerun with lcov and check the coverage...

Mission accomplished.

How to extend tests in general.

One approach is just by looking at tests and the statement coverage and trying to increase it. Statement coverage is a small start, but better than nothing. If you have ever worked with safety critical stuff like ASIL D, aerospace, avionics or medical projects you probably know about branch/decision coverage, modified condition/decision coverage (MC/DC) etc. Another approach from a functional side would be to treat escrow functionality as a finite state machine and design a set of tests that cover at least 0-switch state transition coverage. Well, there is no real escrow state machine but one can be reverse-engineered. Hint: this is just my view on this. But I know 'a bit' about testing...There are tons of methodologies out there for you if you are interested.

Summary

Even if you do not know how to bring in new features you can always help with testing. Since STEEM is an Open Source project it means that you can also take part and improve it. Why tests are important?

  • they are a safety net for any feature / code changes
  • they are to protect and mitigate risks
  • they show how things work or are supposed to work

Don't forget to upvote the post if you liked it!

though to be honest sometimes I have doubts that anyone reads this...
I will begin to post about 22 pushups a day, origami or new steem t-shirts designs if this one proves to be a failure :p

DISCLAIMER: THE INFORMATION IS DELIVERED FREE OF CHARGE AND 'AS IS' WITHOUT WARRANTY OF ANY KIND. I HOPE IT IS ACCURATE AND FREE OF ERRORS AND YOU FIND IT USEFUL.

steem steeminternals
8 years ago by

Downvoting a post can decrease pending rewards and make it less visible. Common reasons:

  • Disagreement on rewards
  • Fraud or plagiarism
  • Hate speech or trolling
  • Miscategorized content or spam

Submit
$0.00
    22 votes
    2
    Authors get paid when people like you upvote their post.
    If you enjoyed what you read here, create your account today and start earning FREE STEEM!
    Sort Order:  
  • Trending
    • [-]
      ·  8 years ago 

    nicely done man.

    Downvoting a post can decrease pending rewards and make it less visible. Common reasons:

    • Disagreement on rewards
    • Fraud or plagiarism
    • Hate speech or trolling
    • Miscategorized content or spam

    Submit
    $0.00
      [-]
        ·  8 years ago (edited)

      Thank you sir!

      Downvoting a post can decrease pending rewards and make it less visible. Common reasons:

      • Disagreement on rewards
      • Fraud or plagiarism
      • Hate speech or trolling
      • Miscategorized content or spam

      Submit
      $0.00