First of all, let me excuse myself. English is not my mother tongue so I don't sure that ppl can understand what I mean to say. This is just an translation for my korean post but I'm ready to communicate to anyone interested in both coding and english. Thanks for feedback.
Recently, It seems like the fluctuation of the coins are not the same as what it was but still I'm obsessed with the concept of bit coin. so I decide to push ahead running a project because I promised to my friends that I will make a application for em. I'm using nodejs btw.
What I'm trying to make is an simple server-side application that pushes message(that is tweet of certain use) to telegram. My application is working in test phase somehow but I realized that I'm lagged because of the rate limit of APIs. since the logic of my code is getting complicated, I really need to write the logic in somewhere and that's why I'm writing this post in here.
The flow of my logic is like,
- In test phase
fetch screen_name(this will specify a certain user in twitter) -> call twitter API for each screen name and load tweet id and text in object array -> push the message to telegram private channel with telegram bot
In actual applying phase, which is that I didn't code yet, I'll autoficate(don't sure is this word is actually exist) this full flows.
Anyway in this flow, I found some problems in second phase and third phase which is rate limit of each API(twitter, telegram bot)
references
twitter API rate limit : https://developer.twitter.com/en/docs/basics/rate-limits
telegram API rate limit : https://core.telegram.org/bots/faq
First problem, twitter rate limit,is can be handled by create multiple app since twitter API rate limit policy is applied in not user but each application. I crated 3 twitter app judging by the amount of fetced screen name and the amount of expected call ( I made 224+ call in every 20 sec, so basically I need to change the API keys for evey 5 minutes ).
But real problem is that how can I change the API in real time and the first thing came in my mind was create managing process and make the process create another process with different API keys for every 5 min.( I guess this idea came from the influence of the last class of last semester which I taught from OS class by Jung Kuk Kim ) There is a function similar to POSIX fork() in nodejs which is child_process.fork() and I decided to use this.
I will keep post about this part after doing this.
nodejs.fork()
https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_optionsNext problem was occured in telegram API and it was also rate limit which is 30 calls for 1 second. In first, I coded call telegram push API for 34ms interval. ( which is calculated from 29 call = 986 ms , 30 call = 1020 ms, rate limit = 30 call / 1000ms) but there is another rate limit policy for sending messages in certain channel iteratively, don't sure what is the precise amount but I decided to ignore this problem because this problem occurs only in test phase. I don't think that 224 new tweet will occur in same time in real life. I'll handle this problem if I got problem in future.
my current pseudo is like,
bot.on(db.fetch().then(success callback(twitter success, twitter fail), fail callback))
but I need to change the API keys in every 5 min and start new process so basically I need to send start command to the telegram bot in every 5 min and this is not acceptable. I guess I'll change the entire form like this
bot.on(manager procss -> fork() or exec())
I know that many ppl already made similar application like this but I wanna make my own app. I think there is more simpler way to handle this problem that I couldn't figure it out since I'm newbie developer but I guess this is my best for now. If someone did this in the past or have an idea for this, let me know.
Thank you for reading.