Integrating ElasticSearch on a Private STEEM Blockchain

in eftg •  6 years ago 

Utopian for European Commission (EC):

Utopian.io is running this task request on behalf of the EC Blockchain Competence Center (BLKCC).

GitHub Repository:

https://github.com/scr53005/eftg-steem

Integrating ElasticSearch on a Private STEEM Blockchain

The last task for the EFTG project of the European Commission relates to integrate ElasticSearch on a Private STEEM Blockchain. The current work shows a solution for this task. The principal idea is to have a a bash script who instert automatically according to specific schema in Elasticsearch as soon as a block in generated.

Elasticsearch is a search engine and not just a key-value store. It's also a very powerful analytical engine; all the queries that you would usually run in a batch or offline mode can be executed in real time. Support for features such as autocomplete, geo-location based filters, multilevel aggregations, coupled with its user friendliness resulted in industry-wide acceptance.

Data in Elasticsearch is stored as JSON (Javascript Object Notation) documents. Most NoSQL data stores use JSON to store their data as JSON format is very concise, flexible, and readily understood by humans. A document in Elasticsearch is very similar to a row when compared to a relational database. Let's say we have a Company table with the following information:

Id Issuer_name Home_member_state Financial_year submission_date
1 Fujitsu Technology Solutions DE . 2017
2 Cronos International BE 2018

The meta_data in the preceding company table, when represented in JSON format, will look like the following:

{
"id": 1,
"issuer_name": "Fujitsu Technology Solutions",
"home_member_state": "DE",
"financial_year": "2017"
}

Deliverables

  • Bash script for listening creation of new blocks and inserting in Elasticsearch specific data
  • Documentation on how to deploy and test

Requirements:

Use Elasticsearch 6.X Kibana 6.x stack for storing and querying data
Send curl calls to get data in JSON format

Relevant commit

GitHub 3062a59 in dev branch

How to contribute?
Any type of contribution is very welcome. You can make pull requests to the Github repository, or contact me on Discord (bogdan#72377), steem.chat (barbilian), or steem (@barbilian).

GitHub Account

https://github.com/BogdanVaceanu

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Congratulations @barbilian! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You published your First Post

Click here to view your Board
If you no longer want to receive notifications, reply to this comment with the word STOP

Support SteemitBoard's project! Vote for its witness and get one more award!

Hello @barbilian! This is a friendly reminder that you have 3000 Partiko Points unclaimed in your Partiko account!

Partiko is a fast and beautiful mobile app for Steem, and it’s the most popular Steem mobile app out there! Download Partiko using the link below and login using SteemConnect to claim your 3000 Partiko points! You can easily convert them into Steem token!

https://partiko.app/referral/partiko

Congratulations @barbilian! You received a personal award!

Happy Birthday! - You are on the Steem blockchain for 1 year!

You can view your badges on your Steem Board and compare to others on the Steem Ranking

Vote for @Steemitboard as a witness to get one more award and increased upvotes!