Enhancing EOS.IO's GNU/Linux support with Yocto Project

in utopian-io •  6 years ago  (edited)

eos.io+yocto

Intro

In this article I explain my recent work on meta-eosio, present the current state of the project, as well as the next development steps. My intention is to get developers interested in contributing to meta-eosio, in order to accelerate its development.

  • Package Management: Yocto Project's tools enable the creation of GNU/Linux packages (e.g.: .deb, .rpm. .ipk). The average Linux user is not necessarily comfortable with fetching repositories and building stuff from source. Having package feeds that people can install EOS.IO with a package manager (e.g.: apt, dnf, yum, opkg) would accelerate EOS.IO adoption and enable it to be run on popular Distros like Ubuntu, Fedora, and others.

  • Live USB: meta-eosio opens up the possibility of building an EOS.IO Live USB Image to be flashed on USB Flash Drives and boot up as a Live Operating System, making the task of running a node a lot simpler.

  • Hardware Architecture: Although EOS.IO's main repository provides a few shell scripts to help developers build EOS.IO inside many x86 GNU/Linux Distros, there's still no straightforward way to get it running on different hardware architectures (e.g.: ARM).

In order to contribute to meta-eosio, you should be familiar with EOS.IO and the Yocto Project. Throughout this article I make a brief introduction to both and provide references in the form of embedded links.


EOS.IO

EOS.IO is Daniel Larimer's new Blockchain platform for the development of decentralized applications and decentralized autonomous corporations. The official release is planned for June 2nd, 2018.

EOS.IO's development is being led by block.one, and the official website is eos.io. The official white paper is available here.


Yocto Project

The Yocto Project is a collaborative umbrella project coordinated by the Linux Foundation. Its main goal is to help developers create custom GNU/Linux-based Distributions for embedded and IoT devices.

Yocto's main components are BitBake, OpenEmbedded, and Poky.

  • BitBake is a python based make-like build tool specialized in cross compilation, although not limited to that. BitBake is inspired by Portage, Gentoo's package management system. BitBake uses recipes, which are metadata files with information about the packages to be built.

  • OpenEmbedded is a cross-compilation and build-automation framework focused on creating Linux distributions for embedded devices, adopted by the Yocto Project in 2011. OpenEmbedded modularizes BitBake recipes in form of layers. Each layer is called meta-layername.

  • Poky is the reference distribution of the Yocto Project. It contains the OpenEmbedded Build System (BitBake and OpenEmbedded Core) as well as a set of metadata to get you started building your own distro.

For more information, refer to the Yocto Project Mega Manual. Yocto is famous for its solid documentation. You can find a neat "getting started" style section here.


meta-eosio

As mentioned above, meta-eosio is the OpenEmbedded Layer I created to support EOS.IO. It's hosted at github.com/bernardoaraujor/meta-eosio, and it's indexed at the OE's Layer Index.

EOS.IO has many dependencies. The most important ones are:

To name a few other dependencies, there's doxygen for documentation and mongodb for databases, although they're not essential to basic compilation so I'm leaving them aside for the moment.

All the essential dependencies are dealt with at eosio.inc (except ChainBase, I'll write a recipe for it soon). Some of them are listed on the DEPENDS variable, and some have their own path variables set up.

The recipe is currently set to build Dawn v4.2.0, which is set in the tag field inside SRC_URI definition.

EOS.IO is built with CMake. The recipe file eosio_git.bb calls CMake to set up the build environment with the do_configure task, and starts the build process with the do_compile task. I didn't bother to write a do_install task definition yet, because do_configure and do_compile's executions are still somewhat problematic and do_install isn't executed in BitBake's workflow until both have run successfully.

EOS.IO's CMake expects a WASM_ROOT variable with WebAssembly's path. I'm using meta-clang's recipes to build WebAssembly. To do that, I added clang_git.bbappend to the layer in order to force clang to compile with wasm support and added clang-native to eosio.inc's DEPENDS variable.


Current Challenges

At the moment there are a few issues that need to addressed:

  • do_compile (log): EOS.IO needs Binaryen to generate WebAssembly code. I was getting eosio-s2wasm: not found inside do_compile's execution (line 1980 inside the log), so I decided to write a Binaryen recipe, pass the BINARYEN_BIN path variable to CMake in do_configure, and set binaryen-native in the DEPENDS variable inside eosio.inc. However, Binaryen's recipe doesn't seem to be populating eosio's recipe-sysroot-native directory inside its workdir, so the issue persists.

  • do_configure (log): This issue varies depending on which host machine I'm using for the build process, and I haven't been able to nail down it's root cause with 100% confidence. Sometimes I'll get some LLVM compatibility issues during do_configure's execution (line 104 inside the log). I suspect it having something to do with the fact that meta-clang's master branch using LLVM v6.0, while EOS.IO needs v4.0. A dirty workaround could be setting meta-clang to pyro branch (which does provide v4.0), but a whole range of different issues arises from that and maybe there's a better way to fix this. Also, the issue arises when CMake is configuring ChainBase, so writing a recipe for it might be a solution as well.

  • Machine Architecture: I'm sticking to qemux86-64 during the initial development stages of meta-eosio. Additional adjustments to eosio.inc and/or eosio_git.bb are needed to support different machines.

I've been working really hard on meta-eosio. Unfortunately, my time and energy are limited 🙁 The more people are involved, the faster meta-eosio will come to life and people will be able to benefit from it.

New ideas and pull requests are more than welcome, not necessarily limited to the issues presented here. Please post a comment below or send an email to [email protected] if you want to help or have any questions 🤓

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Thats interesting, any guide/reference how to install it in raspi or nanopi?

hey @drsensor, thanks! There's no guide for Raspberry Pis yet, but as soon as meta-eosio's recipes are stable, that's the first thing I'm writing! One of the main goals is exactly to support popular boards like the RPis, BeagleBones, DragonBoards, and others.

Nice. If you don't mind (just personal request, ignore it if its cumbersome), support http://www.nanopi.org board to 😋.

BitBake will build packages for the target machine selected in the MACHINE variable inside the local build settings (conf/local.conf).

Looks like there's already a meta-nanopi layer providing support for the NanoPi board, so (in theory) building for NanoPi should be pretty straightforward.

As soon as meta-eosio is ready I'll make a new article with a guide on how to build and install GNU/Linux packages and images for popular arm boards. I'll try to make sure the guide is generic so NanoPi is also covered :)

Hi, Nice to meet you and Upvoted you :) !
:
自己打败自己的远远多于比别人打败的。

hey @ustd, thanks! :)

Thank you for your contribution. It's always better to write install instructions in the readme.

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

Hey @bernardoaraujor
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Contributing on Utopian
Learn how to contribute on our website or by watching this tutorial on Youtube.

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!