Open-sourcing Vessel: a high-level Ruby web crawling framework

in ruby •  5 years ago 

Vessel is a fast, open-source, high-level, web crawling and scraping framework for Ruby, built on top of Ferrum — a minimal-dependency pure-Ruby driver for running headless Google Chrome instances.

Why would you need a web crawler? Perhaps you’re building a search engine for an intranet or a group of public-facing websites, or just need to mirror a website with finer-grained control than tools such as wget offer.

Learn how you can use Vessel framework in your project

Vessel Evrone.png

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  



HIVE (THE NON-COMMUNIST, NON-CENSORED, JUSTIN SUN EXCLUDING SOCIAL BLOCKCHAIN) IS ALIVE, COME JOIN US!!!

https://peakd.com <--- Log in with your Steem account today!!

Hi! Did you know that steemit.com is now censoring users and posts based on their opinions?
All the posts of these users are gone!
https://github.com/steemit/condenser/commit/3394af78127bdd8d037c2d49983b7b9491397296

Here's a list of some banned users:
'roelandp', 'blocktrades', 'anyx', 'ausbitbank', 'gtg', 'themarkymark', 'lukestokes.mhth', 'netuoso', 'innerhive'
See anyone you recognize? There could be more, they also have a remote IP ban list.

Will you be censored next?