[Steem on RPi] Downloading block files...

I have succeeded to build the steem source for arm and made it a docker image.
Then, I could run a witness node on Raspberry Pi 4B (8G RAM).

The final question is whether or not the RPi4's hardware can sync and run a witness node.
To do so, I need a latest block file and have been downloading the files here:
https://files.steem.fans/ (thanks to @ety001).

Hi, I have been downloading latest block files to quickly sync on RPi4.

There are two ways to download block files.

1 Stream way

As @ety001 guided, you can download the compressed file and decompress it to a destination which will be data/witness_node_data_dir.

For the disk less space users, you could try the stream way to download and decompress backup data. Like this:
wget -c -qO- https://files.steem.fans/data/steem_witness-20210806.tar.lz4 | lz4 -d | tar x

I tried this but failed because my home internet connection was not stable. The tar command always overrides the existing file. So if the download stops, then you have to download the file from the start, which was not possible for most of case because the file size is too large ~ 200GB.

2 Download first and decompress way

The other option is to first download the file and decompress it in the right directory.

wget -c command allows continuous downloading. This is more reliable way.

steem_witness-20210806.tar.lz4       55%[++++++++++++++++++++++++++++++++===>                             ] 117.71G  2.62MB/s    eta 11h 32m

But my problem is slow network. As you can see the speed is 2.6MS/s. I was much lower yesterday. It should be around 10MB/s but recently the speed became so low. And my USB to Ethernet adapter was out of order recently. The cheap stuff always cause trouble.

After downloading the file, do the following to decompress it in the right folder.

go to the witness_node_data_dir~~
unzip the lz4 file and untar the result using the pipe

$ cd ~/steem-docker-ex/data/witness_node_data_dir
$ lz4 -d /media/ssd/steem/steem_witness-20210806.tar.lz4 | tar xv

~~$ lz4 -d steem_witness-20210806.tar.lz4 | tar xv - -C ~/steem-docker-ex/data/witness_node_data_dir/~~

Here I am using external ssd and attached it to my RPi4.

There will be the blockchain folder under the witness_node_data_dir.
The file will be decompressed 20MB per second, which mean takes looooong time. Keep watching the progress!

Update

I have trouble in decompressing the file and copy the files at the same time.
I tried two times the decompressing and copying the files but it failed at 160GB.
I did this command on RPi4 (8G RAM). I guess the RPI's RAM is not enough for this task.

Before, I could decompress (without tar) in the witness_node_data_dir, so I thought there would be no problem.
But I guess it is.

If you have large storage not like me, then you will have not issue here.

This time I am trying to decompress and copy the files on i7 machine which has 16GB RAM. I attached both the ssd having the lz4 file and RPi's SD card to the i7 machine in which the decompression and copy occur.

Let's see the result. If this would be not successful, I have to erase lots of data to secure free storage. Wish me luck.

Oh my god, the source file has an error. The decompression stopped at the exact same place: 160GB. oops

Decompressed : 162900 MB  Error 66 : Decompression error : ERROR_decompressionFailed
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

So I tried to keep downloading the source, but it is completed. oops.

 The file is already fully retrieved; nothing to do.

I think I have to try other source.. long way to go.

TL; DR

I recommend you separate downloading and decompressing if your network is not stable.

When the downloading and decompressing finish, I could start lauch steem docker to sync block data.

Wish Me Luck, It is just before the new era of witness nodes

Resteem / Vote / Follow / Comment / Support

cc.
@steemcurator01
@steemitblog

Does the foundation have some interest in this project? Want to make it official support? Just let me know.

Sort Order:

Trending

[-]

afrog (70) · 4 years ago (edited)

Very good move, @etainclub. I asked another developer if he can do this a few days before. I'm very enjoyed to read that it is already done and of course I will try the installation soon.

$0.05

1 vote

etainclub (78) · 4 years ago

hi! i just followed how the blurt did. Thanks to you I came to know a witness node on rpi.

I am trying to sync. I need to copy the recent block data to rpi but it failed due to the unstable network and some other reasons.

$0.04

moecki (72) · 4 years ago

That sounds very interesting. I think I'll have a look at that, too.

$0.00

I am struggling to copy the large block data to RPi's SD card.
But I will solve this issue!
Thank you.

I can imagine that.
When I read the system requirements, I had doubts if this can work with the RPi.
I look forward to your further steps.

ftz (60) · 4 years ago

great!!!

Thanks. It is tough but I will do it!

I wish you good luck. @etainclub

joviansummer (75) · 4 years ago

If it syncs up correctly, this will be a significant milestone for STEEM blockchain!

행운을 빕니다!

etainclub (78) · 4 years ago (edited)

I am stuck in decompressing and copying the block data files due to storage problem.
See the Update in the post.

If this finishes, I can try to sync quickly. Thanks.

ladynazgool (58) · 4 years ago

I can't login to playsteem, is this app dead?

nope. more details please.

I've installed Playsteem app on Huawei smartphone, app works, but I can't log in.

no response in login? no error? screenshot plz.
if you are in china, it might happen because the app uses google service.

Screenshot_20210809_132207_club.etain.playsteem.jpg
Here is screenshot, I'm not in China, but my Huawei smartphone can't use Google SERVICES)) Regrettably...