The situation
Yesterday, @heimindanger reached out to me regarding adding a feature in the new d.tube, that allows the user to choose an upload server of their choice. He mentioned that he wanted to add @vaultec's dtube.network and @onelovedtube's upload servers as predefined options.
As you know, I had chosen to write the uploader website from scratch, including both the client and the upload server side code, mainly for these reasons:
- Compiling the official uploader code is not easy due to the dependencies used, as well as the fact that it doesn't come with a UI that interacts with the blockchain. The user has to clone the dtube frontend, make some changes to connect to the upload server instance and compile separately. My uploader includes everything in one package, all you need is Node.js, a couple of dependencies for sprite generation as well as a running IPFS daemon to get it working.
- I need a whitelisting system to ensure that only paid users are able to upload.
- The queueing system makes everything synchronous therefore hard to scale to large amounts of users. I could fork it and modify it to meet my needs but due to my lack of C# knowledge this is not happening.
Currently, the /videoupload
API has no authentication system in place, with CORS disabled so that the API may be called only from the same domain (preventing unauthorised users from making the call elsewhere). Same applies to /imageupload
API for images in Steem article body.
Implementation
In order to include @onelovedtube's upload servers as a node option to upload videos on d.tube, several changes are required:
- Changing the name of the API call so that it's the same as the ones used in the official uploader (this only requires 4 lines of code changes).
- Enabling CORS so that API may be called from any domain.
- Adding subtitles support (coming in the next code update)
- Adding an auth system so that only certain users may upload to the server (this is the one that we need your help on).
Solution 1: Issue a one-time use upload token for every upload
How this can work:
- [Server] Define a variable called
uploadTokens
as[]
on app startup. - [Client] Call
/uploadrequest
GET API with ausername
parameter, which is the Steem handle of the user. - [Server] Check if username is in the whitelist. If it is, generate a random string (perhaps with GUID).
- [Server] Append the generated string to the
uploadTokens
array. - [Server] Encrypt the generated string with user's public posting key, then return the encrypted message as response.
- [Client] Decrypt the message returned with their private posting key, then send the decrypted message in the
/videoupload
API call, with the decrypted message in a URL parameter calledupload_token
. - [Server] Check if submitted
upload_token
exist in theuploadTokens
array. If it doesn't exist, reject the upload request by returning an error. If it exist, delete theupload_token
string and proceed with the file upload.
Here comes the problem:
This will only work if the user is authenticated with Steem Keychain, or by entering their private posting key on d.tube. SteemConnect does not have a method to decrypt messages with posting keys, so this will break SteemConnect support.
DTube has plans to depreciate SteemConnect auth on their official frontend, which leaves them with Steem Keychain and private posting key auth. @onelovedtube will never ask users to enter any private keys on our own websites, which leaves us only with Steem Keychain, which can be an inconvenience to users who cannot install the Steem Keychain extension for any reasons, if we choose to depreciate SteemConnect auth.
Solution 2: Submit the access_token to server when uploading
Whenever the user logs into our uploader webpage regardless of auth method, an access token is generated and parsed into the URL to detect logins. The access token contains information such as logged in Steem username which can be submitted to server together with the upload API call, and it can be checked against the whitelist before the upload begins.
Access tokens for Steem Keychain authentication can be obtained by calling the /login
API followed by /logincb
API, the same way of the Keychain login system of uploader.oneloved.tube website.
Here comes the problem:
On d.tube, users may be logged in by entering their private posting key, rather than using Keychain or SteemConnect. Therefore access tokens cannot be generated, which means that the user cannot verify themselves that they're in the uploader whitelist file.
Solution 3: Sign a message on client side, then submit to server for verification
Credit goes to @heimindanger for this solution
How this can work:
- Client is asked to sign a message (using requestSignBuffer on Keychain, and equivalent methods on other libraries).
- Signature is sent to server with
/videoupload
API call, then is verified on server with Steem-JS. - If signature is invalid (or the user signing the message is not in whitelist), upload is rejected. If signature is valid and user signing it is in whitelist, file upload proceeds.
Here's the problem:
Looks like there is no way to verify a signature on Steem-JS according to their documentation (maybe it's not documented somehow). There's also no way to sign messages with SteemConnect.
So this is it, the 3 solutions to this auth system to the file upload APIs on #onelovedtube uploader. All of them come with issues that break an existing sign in method, so I need your ideas to decide which is the best approach to this situation. If you have any ideas regarding solutions that is better than the ones specified above, feel free to let me know in the comment section below.
As you stated we do not ask for posting keys thus maybe the best option is solution #2 and I would assume users that wish to use our service would understand.
The technical is by far over my head so I can only go off what information you have provided. If it will work for our existing 2 sign in methods with the uploader I see no reason not to do it for the mean time while working on further possible solutions for posting keys or just continuing to support just the two auth systems. Sounds like some serious development of the main components is required to make every option viable, not sure how challenging that will be?
Posted using Partiko iOS
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Solution 2 will break support for posting key login on d.tube since there's no way that you can generate access token for those. Remember that the API will be called from our website, d.tube and potentially other websites as well.
If we decide to depreciate SteemConnect altogether then solution 1 will work.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Could you not have posting key just doable the process’s used for alternative upload servers? Have it runs as it does now with the added benefit of those using Steemconnect or Keychain able to select destination?
I do agree keeping posting key available is essential.
Posted using Partiko iOS
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
The whole idea is to allow uploading to our servers from their interface, not redirecting to another domain, regardless of auth method.
Another solution would be combining multiples. That could be options of either submitting SteemConnect access token or using a one-time
upload_token
. The API may accept either inputs.Right now an investigation is being carried out to find out how our users are logging into our uploader. Let's see how it turns out here after 7 days.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Sounds like a plan to me 🤙
Posted using Partiko iOS
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'm really happy to see you provide such dynamic solutions here to choose from. Solution two or three seem to be really viable if we want to keep steemconnect. On the other hand, solution one may be able to implement a bit cleaner. And we have ran into a mountain of those sdk errors already.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
If you have better ideas than what I mentioned above just let me know 😃
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
This post has been included in the latest edition of SoS Daily News - a digest of all you need to know about the State of Steem.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit