DTube: a decentralized video platform using STEEM and IPFS

I am really excited to finally bring this project to the public. Introducing...

HTTPS://DTUBE.VIDEO

What is DTube?
DTube is an application fully written in javascript, that runs in the browser, that allows you to upload and watch videos on top of the IPFS Network. Moreover, it uses STEEM as a database and enables earning rewards from your uploads.

This might ring a bell for those who remember the SteemQ project announcement, which made almost five thousand dollars in rewards, but never got released and ended up being rebranded - It's still not functional after more than a year, and even the current alpha uses a back-end server for everything and is therefore still centralized. I am sure I wasn't the only person disappointed by SteemQ.

I opted for a different approach. Build something first - talk after. If you are wondering, I did everything by myself (and the help of open source libraries of course) and it took about 4 months to reach what I have now, starting from scratch.

A huge part of this article will be technical and maybe not so interesting to some people, but I believe a big part of my followers are interested in this kind of discussions, so I will explain how it works. Sorry for the incoming walls of text :)

How to use DTube
Logging in

1. Enter your STEEM username 2) Enter your Private Posting Key, you can find it like so:

Multi-user login

Youtube and Facebook have this amazing feature that you can easily have multiple channels linked to your account and switch between them easily. I am sure I am not the only user on STEEM who owns multiple accounts, and having to use different browsers or bots to upvote with all accounts gets boring very quickly. On DTube, you can log in with as many accounts as you want, and switch between them.

I don't know why SteemIt still doesn't have this feature.

Settings

DTube features a settings menu accessible from anywhere in the app. It allows you to adjust your voting power, filter NSFW contents similarly to SteemIt.com, and tune the IPFS Settings. IPFS Gateway is where the files are loaded from, while the API is what we use for uploading.

Uploading

First, you need to be logged-in to be able to access the upload page. Then just simply add a video, a snap (aka thumbnail), and finally fill the texts for the blockchain and submit!

If all goes well, you should be taken to your video after submitting and it should start playing the video between 0 and 30 seconds depending on the gateway and your connection speed.

Browsing Videos

There are currently 3 main ways to discover contents on DTube. You can go to the homepage and discover the NEW , HOT and TRENDING categories. They work the same as SteemIt. You can access a certain user's channel by going directly to https://dtube.video/c/curator for my testing account for example. Finally, you can use the search in the top bar, more information on that later in the article.

How DTube works
Using the STEEM Blockchain as a database
Because we want DTube to be truly decentralized, we cannot have a server running a database and use it to query things. Using a blockchain is a natural solution. STEEM has many advantages over other blockchains. It is fast (3 sec blocks). It is free, anyone can use it without having to deposit some form of currency, and transactions have no fees. Who would want to pay money to upload a video, comment a video, or even upvote? Nobody. As an added bonus, the STEEM blockchain already has a reward-earning mechanism.

Any video uploaded on DTube becomes a STEEM content, that can earn rewards for 7 days. DTube doesn't use either the title or the body of a content to store information, therefore if you don't want your video upload to appear in your feed, you can edit or delete it, the video would stay intact on DTube. While it would be possible to display nothing on other platforms like steemit.com or busy.org, I believe the current behavior to be fine and will help the platform growth and the users rewards.

Using IPFS as a static file storage
IPFS is a protocol that enables decentralized file storage. The principle behind it is called Distributed Hash Tables (DHT). The same principle also powers the BitTorrent network. Just like how cryptocurrency uses asymmetrical cryptography, DHT networks will hash contents in order to be able to identify a file. The hash becomes the identifier of a file, and it's as easy as re-hashing a file and making sure the hashes match to ensure the file that was sent to us is the original.

Why not use Torrent then? Torrent arguably already has more content and is more mainstream. However, Torrent was not built for the web. Some implementations such as WebTorrent are trying to make it work, but it still requires some 'hybrid' clients that would act as bridges in order to transfer files from hard clients such as μTorrent to web clients that run in a browser. Not very convenient, even less efficient. I am sure a lot of you have used PopcornTime to watch some 'pirated' content and have enjoyed it, but building something similar that runs inside a browser without having to run hybrid clients to connect people, is sadly impossible.

IPFS is a younger, open-source, and actively developed protocol. It has a bright future ahead. Using it as the main mean of storage for this project was a no-brainer.

IPFS Storage Cost
IPFS is cool, but there is no magic. Someone needs to seed the files, and your browser cannot permanently store huge files (local storage is limited to 50MB on most browsers), so seeding through the app directly is not possible as of today. While my first idea was to ask some witnesses to run some IPFS nodes, it became clear after a few conversations that this would create more problems than solutions as most witnesses are non-technical persons and running and configuring an IPFS node correctly seems to be a tough challenge for most. I still believe this solution to be the right one, but I would clearly need to setup a docker or something easy for witnesses to actively join the DTube network and start seeding files (and earn a share of the rewards).

Instead, I searched for existing IPFS nodes and contacted the owner of IPFS Store, a website that allows you to pay in Bitcoin to keep your files on the IPFS network. After a few positive and instructional replies from @nannal (steem, twitter/etc), I knew I found my man.

I have recently created the @dtube account. This account will be used to collect 25% of the DTube author rewards. 10% of these fees will be used to pay for long-term storage of the files on IPFS Store. The rate is $0.044 per GB per month. So, for example, let's say you upload a 100MB video, that earns $10 rewards, then $0.25 will go to @nannal and ensure data redundancy for ~57 months. Once this time is passed, users will need to either pay themselves (crypto accepted of course) to keep the files being seeded, or seed it themselves directly on their own PC and connection.

Design
Most developers know the saying 'do not reinvent the wheel'. The same applies to design and user experience. The leading video platform YouTube has been running successfully for years, and copying them makes sense. Especially when I am a sucker at front-end development and UI design. I tried to copy the look and feel of YouTube using the Semantic UI framework. While I am proud of what it looks like, I am sure some more talented people than me would slap me in the face for how bad it looks and plays.

I am very open to suggestions. Or if you are a web-designer, with a good portfolio, and Semantic UI experience, feel free to contact me on the steemit.chat.

Using AskSteem for Search
It might be hard for non-technical people to understand it, but the search is actually the toughest problem to make DTube a viable product. When you have a standard SQL or Mongo database, searching is easy. However here, our database is the STEEM blockchain, and we cannot search by communicating with a steem node directly. I understand that search is a low-priority feature for a blogging platform, especially when Google indexes every content and offers a search API. But for DTube, users will use it a lot.

I contacted @thekyle, a competent developer who posts on SteemIt but doesn't get half as much rewards as he deserves. He recently introduced AskSteem, a search engine for the STEEM blockchain. His service uses an ElasticSearch DB and will index new videos under 15 minutes.

His API was originally not enough for me to display certain information such as the video snap (thumbnail) in the search results, or else I would have needed to make one call for each search result, i.e. a lot of useless calls. @thekyle fulfilled my request to improve his API in under 24 hours, and allowed it to run smoothly as it should. ❤

Arguably, the search is not decentralized. Any search will go through the AskSteem server, search results could be altered, or the search could just stop working if @thekyle decides to close it.

A better solution would be to have a search function directly on the steem nodes. Another solution would be to have yet-another DHT network running inside the app, with a fuzzy search function. I have literally no idea if making a fuzzy search algorithm on a DHT network is possible without creating a point of failure, and I don't think anyone has an answer to this question on Earth. That's why I decided to make a compromise. We will see how it goes.

Permanently stored data
DTube will use your browser local storage to store a bit of information. First, it will store your user accounts and posting keys so that you don't have to relog every time. I suppose that won't shock anyone considering steemit.com does the same.

Secondly, DTube also stores your watched videos in order to display the 'Watch Again' category. Just like YouTube does it. You can notice that post rewards won't show for the Watch Again category, as these rewards can change at any point in time and it would be impossible to trust the data we stored a while back ago.

If you clean your browser sometimes, you will have to relog and your watched videos history will be cleaned, with no way to get them back. Just a friendly warning :)

Why can't I log in with my master password or active key?
Because it's a retarded thing to do. Steemit.com allows it solely because they also have a wallet in the app. XSS attacks are common on the internet, and most devs are rightfully paranoid about it. I would feel terrible if some users of my app lost their accounts and money because of me. Instead, it is much easier to force users to use the posting key and limit the risks.

DTube does not include any feature that requires anything else than the posting key. That means no wallet, no password reset, no internal market trading, no powering down, and way less work for me. If you want to manage your STEEM rewards, I invite you to use Vessel. And only use your posting key on any online platform such as steemit.com, busy.org, etc.

"What about SteemConnect??" I can see this as a comment below this article already. I am very aware of the SteemConnect project, that aims to provide an easy way for developers to enable STEEM login and key management. However, I am personally very much against delegating my keys or sending them to someone else to use, even if it's a trusted 3rd party ran officially by SteemIt Corp. Sorry but I like to remain in 100% control of my keys, and you should too. Remember all the people who cried on Reddit whenever an exchange got hacked? Don't be like them.

The case for user-side encoding
On YouTube, you upload your original file, and YouTube will encode your file and even convert it to multiple qualities, as well as create some fancy gif to preview the video on mouse hovering.

Achieving the same in DTube is hard. Encoding a video file in javascript is harder to code, and would always be slower. Bad On top of that, if you want multiple quality files, then you would need to upload multiple files and multiply the time it takes to upload your video. Bad Bad Finally, it would probably grow the total size of the DTube application quite a bit, and therefore increase the initial loading time. Bad Bad Bad

There are already countless programs that permit video encoding. And that would only be required to do for videos recorded with professional tools such as high-grade cameras. People who own this kind of hardware, usually already know about encoding. My personal smartphone already encodes every video in H.264, and I suppose your iPhone does the same. So why not keep it stupidly simple?

Easy to integrate for other platforms
Any other STEEM based platform can easily integrate videos posted on DTube. The video IPFS hash is stored inside the json_metadata of a post. Then it is as easy as doing

Known Issues
No progress bar on the upload - github issue
Avatars don't resize and load the full-size images - maybe solution
Missing features
Reputation system
Tags browsing
Video suggestions (on the right of the video)
Following Channels
Custom Player
I believe the current state of the product to be viable, even without these missing features. I will add them, slowly. Also because I am unsure which of those is a priority right now, I will let you guys choose and argue about it in the comments!

App Dependencies
DTube is built with Meteor, and uses many open-source packages from NPM. DTube wouldn't exist without them, so they deserve some credits.

ipfs-api - Communication with the IPFS Network
SteemJS - Communicating with the STEEM Blockchain
autolinker - Adding links to descriptions / comments
moment - Displaying times
XSS - For displaying untrusted texts and preventing XSS attacks
Credits to debuggers and idea suggesters
@steeminator3000 @dailydogger @coldbolt @nannal @thekyle @ekitcho @fabien @kalvas @french.fyde @nnnhhh @fayetristan @lukestokes

Cross-links
Steemit
Hacker News
Last words
While I personally believe that decentralization is the future, don't get your hopes too high yet. Competing with YouTube is an heavy task that cannot be done by one man. My intent with this project was to practically demonstrate the possibility of decentralized video on top of IPFS. It works, but we don't know how well it will work in the long-term. YouTube also has a lot of extra features that makes the experience superior. What I built, represents maybe 1% of the work required to have something that can compete with YouTube. I built some walls, but there is no insulation, or painting, and the whole building might collapse at any time.