SteemShovel.com
Disclaimer : This is pre-alpha, in-development software so expect it to crash and/or be unavailable from time to time while I continue to work on it. Thanks for understanding :)
I'm excited to announce the very first rough draft of my Steem Search engine - Steem Shovel. I was frustrated with not being able to find older posts on Steemit so I set out to build my own search engine. I have some experience with ElasticSearch and it seemed like a great fit for indexing Steem posts and comments.
What's it do?
SteemShovel runs continuously indexing new content from new blocks. It will index both comments and posts and make them full-text searchable. Moving forward I plan to allow you to limit your search to specific authors and also restrict searches to certain tags. Right now it's just a full on mega search against all of the fields/text.
What powers SteemShovel?
The underlying application code is written in Node.JS while the search engine is powered by ElasticSearch. I use the SteemJS module (https://github.com/adcpm/steem) to talk to the Steem process.
The application is load balanced using AWS EC2 servers behind an Elastic LoadBalancer. This gives me the ability to add more servers / horsepower at will if anyone decides to use this app.
Why can't I find the post I'm looking for?
The indexing process is currently making its way through all of the Steem blocks. A second process is indexing new posts as they appear but there is a big gap between the two. Because of this it might be a while before you are able to find every single post you're looking for. Hang tight, they're coming!
What are some improvements coming to SteemShovel?
Well, I'm glad you asked! I have a pretty ambitious set of goals for this project so we'll see how it goes. The main focus now is to work on the indexing and build some more filters that you can use to search. Beyond that I want to improve the look of the site and make sure it's working well on mobile.
Once the basics are covered I have tons of features I want to work on - log in, voting, UI improvements to show threads of comments like Steemit does.
But wait, there's more!
I have also built this to be an API that anyone can talk to. All you have to do is direct an http client to https://steemshovel.com/search/?search={your search term} and you'll get some JSON to do with as you please.
If you have any questions or suggestions as to what you'd like to see this thing do, please comment and let me know.
@kaptainkrayola thanks for the heads-up! This is one of many steem tools.
@roelandp is making a collection of them at http://steemtools.com/
Keep it up!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'll get the form filled out there to get listed directly. Thanks!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Great tool dude!
I've listed it on SteemTools!
It's fantastic that you have added a JSON api, this is really helpful!
Supercool that you are indexing...
Feature request, undoubtly already on the list:
Hats off! Applause!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is much needed! Thank you!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Alright - you totally deserve the upvote and this comment.
Great job providing for the community. I have no doubt you will make:
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
You rock man!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Great!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks for the support everyone. I think I'll use the money from this post to get a proper web designer to work on the site as well as some contracting hours from another node.js dev to help build out some more of the features.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
It already looks pretty great IMO
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Just W O W
Thank you so much for your hard work. Hope you're appropriately rewarded. Come on people, get behind @kaptainkrayola!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I do love your enthusiasm. Thanks for the support!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
The built-in Steemit search is diabolical! You are providing a much needed service. Huge respect and gratitude.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Very nice idea, thank you for sharing
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I love how fast it is, and the results seem to be pretty good. Kudos on the implementation so far.
I only have 1 minor feature request, which is to have the ability to show only results from posts in certain time period (ie. last 24hr, past week, past month, past year...).
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'll add some of those filters to the to-do list for sure. Thanks for the input
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
way to go! good job! just what i need!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Wow well done! I hadn't even considered this angle yet. We may have to have a chat at some point, I could see some of my projects pointing to steemshovel as their search provider :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'd love to collaborate. I'm on skype - same username if you want to chat.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is a fantastic example of giving back to the community kaptain. You definitely deserve some upvotes for all of the hard work and expertise displayed here. Great job, we all hope to see more soon!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Damn, really impressed with the speed and seeming thoroughness of the results. I keep throwing odd search terms at it to see what weird stuff it digs up...sorry.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Nice. Thanks :D
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
ROCK ON!!! Way to go! Can't wait to try it out!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This thing is lightning fast. Great job.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you. I'd love to take all the credit for that but a lot of it is just ElasticSearch is a fantastic tool.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Hello! I am @upvotes and I am here to help you get upvotes!
Just upvote this comment and your post automatically becomes an @upvotes candidate for: @people, @places, and @headlines
Several times a day the most upvoted candidates become the Best of Category post
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Very good job!! I suggest you to add ordering by votes on list.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
https://steemit.com/steemit/@minion/youtube-vs-steemit-enemies-or-siblings
[IMPORTANT] This post covers the Similarity between YouTube and Steemit as well on How to use Steemit and It's
OFFICIAL CONTENT RULES.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is a great idea. Nice work!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This seem very cool ,
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I support the idea, it will be a useful tool for sure.
that logo however looks like a face:
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
The logo seems pre-alpha as well...
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'm open to suggestions on the logo so if you have something better, send it over.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
So...are there any list ranking? Meaning, does it place the most recent post on the top of the list, or are there other factors involved to get to the top..like with Google
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
oh men. what are you try to carry with this steemit shovel? raking all the monies in the communty alone. lol
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
why today is down ?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This is wonderful! How do you guys develop apps on Steemit? Where are the developer resources or did you figure this all out on your own?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That is cool you will be able to find my post about the machine gun festival I went to.
https://steemit.com/guns/@steve-mcclair/the-big-sandy-shoot-festival-for-machine-guns-heavy-artillery-and-gun-enthusiasts
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
The growth of this Network starts to scare me :D !
Amazing Job, Kaptainkrayola !
I wish you the best of success!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Woop woop
Thanks
@bullionstackers new post
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Good initiative
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Somebody better call Mike Mulligan!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Nice initiative. Very needed indeed.
IMO the most badly needed feature is the ability to browse user names and posts' tags.
Right now, it's impossible to find a post you know it exists, if you don't remember the exact spelling for the name of the author or the post's tag name.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That's on the list of things I want to add as well. I'll be posting more updates as I add new features so keep an eye out
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
http://www.google.com
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Good job guy! Exactly what we need!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
i think this will be one of most usefull tools around for an average user! gj
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Wow, this is so cool. We are running our own blockchain project called ElectriCChain and this means that if we make a heap of categories and articles on steemit then we can leverage your API tools for our own project?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Sure, I don't see why not. i'll be adding more granular search options to both the site and the api so there's no reason you couldn't use it for that.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Good news! we really need it. Google search is awkward.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you for using @upvotes! I have submitted your post to be a Best of Category candidate in @people, @places and @headlines
Feel free to upvote and add comments to your own candidate but more importantly please upvote exactly one other candidate to help improve quality, your ranking wont be lowered by this.
Just reply to this with a comment starting with a specific @category to become a candidate
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit