SteemData meets sbds - SQL Users Rejoice

in steemdata •  8 years ago 

What is SteemData?

SteemData offers a database layer for Steem. It allows people to query for large amounts of data very fast, as well as construct complex aggregate queries.

SteemData is using an awesome document based database called MongoDB. MongoDB is the 5th most popular database in the world (source), and is especially favored by younger developers.
Some people have reached out to me on steemit chat, and told me they're learning MongoDB just so they can use SteemData. I think this is pretty cool, and I commend your efforts.

There is however a large group of developers out there whom are more familiar with SQL based databases, such as MySQL. These people were out of luck...until today.

SteemData_Thumb2_small.jpg

What is SBDS?

sbds is the official database layer for the Steem blockchain, developed and maintained by Steemit Inc. It is based on the popular MySQL database.

The credits for the sbds project go to john-g-g and @sneak.

SQL and SteemData

In addition to the open MongoDB database, SteemData now also hosts a publicly accessible MySQL database, populated by sbds.

MySQL Instance

Host: sbds-mysql.steemdata.com
Port: 3306

.
Credentials

Database: sbds
Username: steemit
Password: steemit

.
Example
Screenshot from 2017-05-30 02-16-30.png

Steem Blockchain at your fingertips

Once you connect to the MySQL instance, you can query for any operation that ever happened on Steem.

Screenshot from 2017-05-30 02-19-44.png

Recap

SteemData now offers 3 convenient ways to access Steem blockchain data:
1.) MongoDB
2.) MySQL
3.) API

An API?

SteemData also ships with a powerful HTTP JSON WebAPI.
This option is great for developers whom don't want to use either database solution directly, and just want to quickly integrate SteemData as a data source in their apps.

For more info, please see https://steemdata.com/api

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Yay for more public data :)

I also added your #steemdata tag to the Community Projects section on chainBB. Hopefully it's not too polluted with other random posts about data haha.

errrm @jesta, well-done with the chainBB project, have tried to reach you concerning that but my commments doesn't display on your walls....can i have a link to it please?

What's not working? What are you looking for a link to? I'm confused :)

ChainBB!.....anywhere i can read more about it, whitepaper or something. please don't be confused.

I don't have anything elaborate as a white paper yet, still working on the full plan. I just announced the beta for it a couple weeks ago, that might be a good starting place:

https://beta.chainbb.com/chainbb/@jesta/chainbbcom-a-blockchain-forum-platform-for-steem

Okokokok.......Thanks, this should do mate.

  ·  7 years ago (edited)

ya just go to beta.chainbb.co ! ust tell people to ggo there, sign in with their steemit username and POSTINg key which can be found in that "permissions" tab in ur steemit profile....its just a posting passwordnota transfer password , so ur only giiving tis website ability to post for u....so its not a security issue signing into chainBB

youl make so much more posting to chainBB its not even funny....i hate to say it like that but itstrue! go use chainBB and the whales there will help u out with upvotes

Thanks a million mate....i appreciate

Great work @furion. I am learning MongoDB because of Steemdata but since I am not very good on the backend side I will look into your API's now.

Oh cool! I just finished a course on SQL, time to put it to good use!

Thank you for your explanation.
I am not a computer guy, but your explanation made me understand Steemdata easily.

This is TERRIFIC! Awesome work both on the social aspect of things as well as the technological connection you just exposed the SQL Communities around the world to! ;)

Thanks a bunch, all for one and one for all! Namaste :)

This is awesome! Great work!

Feeling bad i abandoned my coding lessons😢. Few of this terms are very familiar, others are just Wizadry terms to me😣

youre like me, sad u didnt take up prorgamming earlier, we must accept that it would take us way too long to learn and we could do SO much more for steemit by spreading links and being marketeers forsteemit just make youtbe videos about how aamzing steemit is ima make youtbe videos again about how much i make on steemit to eentice teenagers who are all posting to instagram and facebook for free

LOL!.....very well said my friend.....please tell me how to post on chainbb using my phone, i can't a posting page .

Very important article for me. Being well versed in all the 3 technologies i will be checking them out especially the mongoDB. There is a use case for all this and after analysis on the data provided will try to create some relevant apps.

  ·  8 years ago (edited)

I am really excited about this. I would like to use this for a few things i have thought of lately. Another amazing development out of you @furion . Thank You

MySql injection on steemit?

The database is read only (you can only perform SELECT queries).
The data is also from the public blockchain, so feel free to sql inject whatever you like.

Waw thanks!! It helpful so much

Congratulations @furion!
Your post was mentioned in my hit parade in the following categories:

  • Upvotes - Ranked 6 with 495 upvotes
  • Pending payout - Ranked 4 with $ 818,7

good stuff, i need in

Stoked to hear that steem is using MongoDB and even more stoked that more people are learning MongoDB to use it. Database query like SQL is a legit marketable skill! Thanks for sharing @furion

  ·  7 years ago Reveal Comment

Thanks for this post, I'll be following!

I am concerned that all this data is useless until Steemit will start to reward traffic - take a look here https://steemit.com/steemit/@digitalplayer/4-98-of-steemit-search-engine-organic-traffic-and-unfair-rewards-series-1

Great work!

Great article, last month only i joined.. so you article made me understand so many things.. Thanks for sharing !!

nice....your post deserve upvote and resteem....

great work buddy
thanks for sharing
upvoted followed

Buen trabajo

Great job!
Thank you @furion

Great job!

can one mine steem?

As a SQL user (see username, heh), huge thanks. Plus with MySQL, I can import these data into SQL Server. Very awesome of you!

Wow SQL and NoSQL databases on the Steem blockchain! As an avid user of IBM dashDB (a SQL based cloud database) and MongoDB as part of IBM Compose, I'm looking forward to using SteemData to give my apps some much needed firepower!

That's awesome. To be freely able to check out anything that ever happened on this platform. Can't program myself though :o).

thanks for sharing @furion

Great inprovement, hope great performance for the future

Very cool! I need to get involved with a few of these dev projects and contribute.

aha... "sbds" I thought you meant "SBDs"

This is awesome though. More insights and more transparency (where this is practical, privacy in other areas).

:thumbsup:

Keep up the good work, @furion. Your data collection skills are superb!

SELECT
a.yayyyy,
b.yay
FROM yays.yayyy a
LEFT JOIN yays.yayyay b on (b.woot=a.woot)
WHERE a.thumb="👍"

Very cool.

I am not a IT guy but still you have cleared many things.
thanks for the information

It's always good to hear about advancement updates for steemit whether I understand or not. Anyway thanks for everything.

Great Article

As a developer and MySQL database user for 20 years.... THIS. IS. AWESOME.

OMG Im more impressed at this comment section! I can tell Furion's Upvotes must be worth $1.20 because every comment In here is up over $1.20 like furion made sure to hook up all his commenters with some free upvote love! When i show my instagram an facebook friends how just a freakin COMMENT can earn them a dollar they immediately singup!

I thinl I have found a new strategy to gain followers! You go tell ever personu see on the street every person you know in real life to go singup for steem and to follow you, you tell them if they follow you, youll follow them and upvote their posts, maybe send em a dollar in steem, i barely ever have to actualy send that dollar in teem as noone asks for it, people just singup and immeditaly get hooked and make thei own money! But I would have no problem sending $1 in steem or SBD to every new person I got to singup in real life! They will end up sharing your posts getting you more than that dollar back!

you can make yOUR OWN followers ! dont get JELOUS of @furion or @jesta or any of these amazing whales making thousands of dollars per day! They are basically all working for that money as developers or programmers or marketers! And AND just remeber! by upviting theese whale posts YOU get a cut! and comment something helpful and theyll notice you!

BUTREMEMBER steemit is only 172,000 members! so imagine if you yourself, got 1 MILLIOn instagram foollowers to follow you here on steemit! youd end up making MILLIONS of dollars! youd make more than EVERY whale NOW here on steemit!

we dont even have 1 million users yet! and our marketcap is alreay about 300 MILLIOn us dollars! and people laugh when they say steem marketcap will never hit 2 billion? are u kidding me? if facebook is worth 200 billion we are worth 2 billion at least! And were already .3 billion is it so crazy to magien us hitting 1 billion by this year?!?!?!?!

There is more growth for steemit on the way than we have ever seen so far!

There will be people making more money here on steemit thanANY otheer socia media platform

people ALREADY make thousands a day off facebook and instagram you just NEVER SEE it! here onsteemit its all out in the openand people are amazed!

Imagine if everyone could see your bank account, lol dating sites would be simpler, steemit will become ultamite dating site because youll be able to see how much money soeone has right off the bat

Steemit is like the internet in 1999 S much potential so much money to be mae BUT ONLY A SMALL GROUP has the vision to make it possible AND EVEN THEy dont realize just HOW BIG this internet thing would turn out to be!

steemit will assimilate all

Great work @furion, what app/tool do you recommend for connecting and querying mysql/SBDS on windows?

Had no idea about sbds, congrats to all the people involved! I successfully connected to the MySQL instance but it seems data is one day behind. I understand the constraints of such a tool but I just want to know if this is by design (you're syncing only 24 hours) or it's a temporary glitch. It would be awesome if SteemData could also function as a high availability node for the Steem blockchain ("high availability" can mean even up to 5 minutes lags, IMHO).

Congrats and nice to see you active again (after all those talks we had at d10e, back in February ;) )

It should catch up to the head, there is a little bug and it falls behind from time to time.

Thanks.

Interesting publication congratulations @furion If you want you can go through my blog and observe my publications of much interest of what happens now in Venezuela

Incredible! I'd love to collaborate on a project using the steemdata API. Does anyone have any ideas about some data mining / visualization projects that could be interesting to get up and running?

This is super usefull! MySql give great power, can't wait to use it. Nice job!

As third place witness I wonder if you can help me please? I can't upvote anyone as whenever I press on any part of the tool bar nothing happens. I have 💯 % of my votes left (just checked). Any ideas? I'm following you now too. Thanks

Congratulations @furion! You have completed some achievement on Steemit and have been rewarded with new badge(s) :

Award for the total payout received

Click on any badge to view your own Board of Honnor on SteemitBoard.
For more information about SteemitBoard, click here

If you no longer want to receive notifications, reply to this comment with the word STOP

If you want to support the SteemitBoard project, your upvote for this notification is welcome!

thanks!

  ·  8 years ago (edited)

Hi @furion, how much is the current size of Steem's blockchain including all the plugins? I'm planning to have my own local copy via MySQL. Thanks!

Hi @furion

I can not connect the MYSQL database now. (It can be accessed some days ago)

Is there something wrong?
Thank you.

Hello @furion! Thank you for your work! It is wonderful! I have a question: I connected to mysql and see only old data there. So it should be? And how often is the mongodb updated? Thank you!

WebAPI: 503 Service Temporarily Unavailable!
Thank you for making access to the steem data easier! I've installed the Robo 3T DB Manager to explore the DB a bit and I think it's becoming a new hobby of mine :)

I want to use your WebAPI for a small test project, but I always get a "503 Service Temporarily Unavailable" error. Are you working on the API or am I doing something wrong with the request. I tried the examples I found on https://steemdata.com/api

I saw this post sometime ago and now started trying to connect to the data and saw something weird. so, I search for this post to asked a question. How old are the records in the database? Doubt here because when I query sbds_tx_comments table with descending timestamp, first one I see is 2016-07-03 08:57:12.0. When I look at sbds_tx_votes in the same manner, the timestamp is 2016-07-03
On the other hand, data from steemsql.com seems to be more recent.

Your projects are amazing,

i (/my project) really can use some of your help and coding magic.

Please consider to help a bit/ give a good push in the right direction to make Steempi work with Steemdata.