bullfrog-system |
The bullfrog-system project creates a clean, complete, and standardized installation of the STEEM Daemon suite on Ubuntu 16. It provides all the tools necessary out-of-the-box to mine, witness, and bot against the STEEM blockchain.
GitHub : bullfrog-system
Background
This is the #mining system that I currently use with @bullfrog et al. I keep roughly 40 accounts to handle the miner witness queue. I mine strategically. When I do mine, I maintain a hashrate of roughly 5.7 mhps. If I'm feeling froggy that day, maybe 11.4 mhps.
I do not maintain mining hardware. I do this all with VMs via Amazon's EC2 service.
This system will be a pre-requisite for the NodeJS bot framework that I'm building here: GitHub: bullfrog. I will announce its release on #steemit once it's ready.
Installation
The target system must be a freshly installed Ubuntu Server 16.04 OS. Only "OEM" 16.04 systems are supported.
Installation is as simple as one command:
curl https://raw.githubusercontent.com/roylaurie/bullfrog-system/master/install-frog.bash | bash
Once the installation has completed:
- Add your miner accounts to the config files found in /var/local/steemd/configs.
- Use ~/bin/config-steemd.bash to select a role (miner, witness, synconly).
- Reboot.
You will know that the replay & sync has completed when you see the bottom two lines as shown. |
What it does
The installation script configures the system as such:
- A new user, 'frog' will be created which you will use for all steem maintenance tasks.
- steemd and cli_wallet -d will be installed as system services, out of /var/local/
- Users 'steemd' and 'steemwalletd' will be created to run steemd and cli_wallet -d respectively.
- Management tools will be installed in /home/frog/bin
- steemd will automatically start on boot.
- The latest blockchain snapshot will be downloaded (hosted by @steemychicken1).
- steemd will immediately begin running --replay and sync after install.
Running the steemd service
To start/stop/monitor the steemd service:
sudo service steemd start
sudo service steemd stop
sudo service steemd status
To enable/disable steemd starting on boot:
sudo systemctl enable steemd.service
sudo systemctl disable steemd.service
To follow the log:
sudo journalctl -u steemd.service -f
Running the steemwalletd service
To start/stop/monitor the steemd service:
sudo service steemwalletd start
sudo service steemwalletd stop
sudo service steemwalletd status
To enable/disable steemd starting on boot:
sudo systemctl enable steemd.service
sudo systemctl disable steemd.service
To follow the log:
sudo journalctl -u steemwalletd.service -f
Management tools
bin/config-steemd.bash
Switches between pre-existing config.ini files. Default names available: miner, witness, and synconly.
bin/replay-steemd.bash
Switches into synconly mode and runs steemd --replay
bin/recompile-steemd.bash
Re-clones the steemit source-code, recompiles, and re-installs.
Configuring mining accounts
To generate accounts with your wallet:
cd ~/wallet/steem
cli_wallet
>>>> suggest_brain_key
The steem wallet will generate a new random private key for you to use:
suggest_brain_key
{
"brain_priv_key": "SUEDE TETRIC FUZZILY BLICK AMANDIN NUMBING BICYCLE BARPOST OUTFLY ...",
"wif_priv_key": "5J8EBoJv42M6Eg2wX9ApwsZQsAoTAEwKfqAqjjZYX3UmK3Pa9XJ",
"pub_key": "STM7NZ9DQXaSRDH9z4MUHkx31axQhngUdwgJ3F5udNQN8aYBCDJy2"
}
Copy the wif_priv_key, that's your miner's private key.
Choose an account name that isn't taken on #steemit. Check the website to make sure.
Edit /var/local/steemd/configs/config.miner.ini. Find the area that looks like:
# name of miner and its private key (e.g. ["account","WIF PRIVATE KEY"] )
# BEGIN MINERS
#miner = ["MINER-NAME","MINER-WIF-KEY"]
# END MINERS
Uncomment the miner = line and fill it in with your account information. It should look like:
miner = ["etherqueen", "5J8EBoJv42M6Eg2wX9ApwsZQsAoTAEwKfqAqjj ..."]
Now edit the /var/local/steemd/configs/config.witness.ini file.
Repeat the exact same steps as above for the miner = line.
Find the area that looks like:
# name of witness controlled by this node (e.g. initwitness )
# BEGIN WITNESSES
#witness = "MINER-NAME"
# END WITNESSES
Uncomment the witness = line and fill it in with your account information. It should look like:
witness = "etherqueen"
Repeat these steps for each additional account that you wish to use. Remember to save the wif_keys.
How to handle a Hard Fork
First, SSH into your machine with your login user. Then ...
sudo su frog # maintenance account
cd # /home/frog
sudo service steemd stop
sudo ~/bin/recompile-steemd.bash # will install as well
sudo ~/bin/replay-steemd.bash # will run replay in synconly mode until killed
You'll see the steemd log as it replays the database then syncs. Once sync'd completely, CTL-C to kill steemd. Then ...
sudo ~/bin/config-steemd.bash miner
sudo service steemd start
Multiple mining accounts
Once your miner finds a POW (proof-of-work), it gets placed on the bottom of the miner-witness queue seen at steemd.com. While on that queue that miner account is on timeout; it cannot be used to mine. So, you'll need at least two accounts. Preferably, the same number accounts as the total number of POWs that you find on average in a 3 hour window.
Once your miner reaches the top of the queue, it's allowed to act as a Witness once and generates 1 block (or misses its turn) for which you receive 1 STEEM in the form of VESTS.
After that, the miner account is available to use again. |
The next miner on the queue will then generate the next block, and so on.
Multiple mining machines
You can run multiple machines with the same pool of mining accounts, with one very important caveat:
Thou shalt not double-witness.
You must have one and exactly one machine that includes the witness = configuration. All other machines cannot include witness configs, and must be purely miner = configs.
You can use the ~/bin/config-steemd.bash <miner/witness/synconly> command to switch between the two.
If you get caught double-witnessing, you will be flayed, beaten, and impaled. No big deal, really.
Changelog
version | date | changes |
---|---|---|
v0.0.0 | 2016-07-21 |
|
v1.0.0 | tbd |
|
License (MIT)
Copyright (c) 2016 Roy Laurie
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
This is very cool!
I'm all set up but not mining.
I edited the 2 ini files as instructed. I am synch'ed to the latest block.
I tried to do sudo ~/bin/config-steemd.bash miner, which I assume will put me in miner mode, but it is asking for the frog password and I have no idea what to enter. I tried the password I created to get into the ssh via putty, but that did not work. So I am out of ideas! Any thoughts on how I can move forward?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
My apologies. At the end of the install script, it gives at list of things you need to do, but I didn't include one of them in the docs and of course now I can't edit.
User 'frog' is created without a passwd to begin with. The first thing you need to do after installation is set the password so you can use sudo with that account.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks for your help!
I have now created a password for frog, after doing ctl-c, using sudo passwd frog
Then I changed to the frog user, using sudo su frog
I went to the directory with config-steemd.bash, using cd ~/bin
I did ls, and I see the file config-steemd.bash in green. If I run it, it says "command not found" both with and without sudo as a prefix. config.ini is in red and the other 3 are in green, when I do ls.
frog@xxx:~/bin$ ls
config.ini config-steemd.bash recompile-steem.bash replay-steemd.bash
frog@xxx:~/bin$ sudo config-steemd.bash miner
sudo: config-steemd.bash: command not found
frog@xxx:~/bin$ config-steemd.bash miner
config-steemd.bash: command not found
Not sure how to proceed. Seems strange to get "command not found" when it is obviously there, but then again, it may be something obvious as I am very rusty at linux!
Thanks for any light you may be able to shine on this!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Similar to Windows, in 'Nix, if a command is not in your $PATH (run "echo $PATH" to see), it will not "find" it. You have to specify a command's path manually if that's the case.
I didn't set up '/home/frog/bin' as a default path as a security precaution for now. You have to include the full path when calling the command:
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks again for your help! I am making some progress I think! This is what happened this time:
frog@xxx:/home/xxx$ /home/frog/bin/config-steemd.bash miner
realpath: /var/local/steemd/configs/config.miner.ini: Permission denied
/home/frog/bin/config-steemd.bash: line 14: cd: /var/local/steemd/witness_node_data_dir: Permission denied
ln: failed to create symbolic link 'config.ini': Permission denied
chown: cannot access 'config.ini': No such file or directory
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
You have to use sudo (Super User Do) on that one. That script is trying to alter a system file so it needs administrator privileges when running, thus the "Permission denied".
You're getting there! Keep hacking at it.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I've added piston.rocks support. It will automatically be installed and available at /home/frog/bin/py/piston.
Also, I've moved the bullfrog-system bin files to /home/frog/bin/frog so that other projects can be added directly to bin without affecting the bullfrog-system project's git status.
Finally, I added a custom MOTD .
The alpha (v0) is now in code-freeze while people try it out. I'll cut v1 a week from now and announce it on #steemit. The master branch will be recreated as a release branch.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
As we all know, killing steemd while it's mining always corrupts the blockchain db. Always. .... Always. Did you hear that devs?! :P
That's what I use the synconly mode for. You can kill steemd in synconly mode and 90% of the time it won't corrupt. You can use this to prep VM snapshots so that the db is clean and ready to go on boot.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I see. Is mining complicated? I'd like to try but i'm not sure I'll be able to.. Can you share a link in your post for mining steem with this. I'm sure many people new to crypto might find it useful to get more information on using your tool to mine Steem.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Mining is complicated, which is why I built this.
This sets everything up for you, then you just add the accounts to the config files, run the config command to tell steemd its role (witness, miner, synconly), then reboot. Done.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you so much!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I added a link that details how to configure everything. I'll add another story on how to use EC2 shortly.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I will follow through this article and try this out. Thanks again. :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I'd upvote this several times if I could.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
i'll do it right now, thanks @kilrathi
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
How'd it go?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I have not yet had time to do, I own all this weekend, maybe I ask you something to you dude 8]
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Sure, I'll be around.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
im looking for a solution to mine independent of pc, ubuntu, os. Something like a Cubieboard with chip to do hashing and block erupters to add to processing....or do I have it all wrong. What I currently have is CCMiner loaded onto Cubieboard running the mining program and each block erupter is 8gh processing power. This only mines, nothing else, Cubieboard chip is dedicated to CCMINER
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Can you add automatic download of blockchain from http://www.steemitup.eu/ please?
cd /var/local/steemd/
wget http://www.steemitup.eu/witness_node_data_dir.tar.gz
tar -xvf witness_node_data_dir.tar.gz
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Yes! I've been looking for a good link all day for that very purpose.
Thanks, procto. I'll update the install script to grab that and I think I'll add a standalone script to import it at any time as well.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
recompile-steem.bash
missing line
rm -fr /home/frog/project/steemit
Guide says bin/recompile-steemd.bash
file says
bin/recompile-steem.bash
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks. Fixed. https://github.com/roylaurie/bullfrog-system/commit/f3649775818eb25d290b84ab59c5b7dad6da31ef
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Nice to help out
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
echo -e "${GRN}|=== Configuring steemd for 'synconly' ... |${NC}"
sudo /home/frog/bin/config-steemd.bash synconly
is obsolete now
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Fixed and tested.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Good catch. bin/replay-steemd and recompile-steemd do that on their own now ...
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
also
echo -e "${GRN}| * Configure files in /usr/local/var/lib/steemd/configs. |${NC}"
should be
echo -e "${GRN}| * Configure files in /var/local/steemd/configs/. |${NC}"
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thanks! Fixed.
Testing the download now ...
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Done. Implemented & tested. Thanks, again.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I've added a changelog section so that people can follow releases as well as future enhancements.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
i've tried to but it seems that i have no skills in this ) is it possible to make a video tutorial on this article of yours?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
I intend to eventually.
That ... looks like a Linux problem. Is that in a VM or directly on hardware?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
vm 12 workstation player. pfunk's guide finally helped me out, but I'm curious one about your miner as well)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Yeah, I used @pfunk's guides to get started. They're awesome.
My setup is basically just the result of trying out everyone's guides. :)
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Can't thank you enough for this. I have been scouring steemit for ways to create new accounts. I knew there was an easier way especially since new accounts are by invitation only.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
hehe I'm not sure if it's easier.
The catch there is that you can't login to steemit.com with a WIF key, so you're going to be limited to using @xeroc 's piston.web
Also, you have to mine a POW before it creates the account, so technically you're paying for the account whereas an invitation is not only free but you get hooked up with some faucet money.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Ok. I need to provide a disclaimer before I ask this question. This is my first attempt at mining any coin.
With that said, my question is concerning WIF keys and user accounts. As I understand, once I create the WIF key, I need to pair it to a username. So in my config file would it look something like
["newusrnm","newWIF1"]
["newusrnm2",newWIF2"]
etc, etc?
Also, if the blockchain is running and I decide to add a new username, am I able to adjust it on the fly or do I need to stop the mining process, adjust the config file, and then restart?
Thanks in advance!
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Correct on the first question, that's how your config would look.
You need to restart the steemd service for the changes to go into effect.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
What size of instance do you use? I'm trying on a free instance which is 1GB - 1 CPU. My guess is this is not enough horse power? What would you recommend?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
will try it tomorrow 'cause there's no luck with windows. appreciate ur work.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
do i really need freshly installed ubunu version? and Why?
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Thank you! I'm sitting here just got my rig mining steem two days before the hack since then I haven't been able to get Ubuntu running. Pain in the ass, I'm gonna try this and see if I can get it going. If it works you deserve my upvote and much more lol.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Let me know if you have any trouble either on here or on the official #mining chat channel; https://steem.slack.com/messages/mining/
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
P.S. I'll add a screenshot of my mining results tomorrow morning.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
how do you deal with forced shutdowns and the reindexing that follows???
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
That's a huge annoyance.
After a shutdown, I config-steemd and restart it in "synconly", which doesn't include any miners and usually doesn't corrupt the DB when killed. Then, after it's has re-indexed and re-synced, I shutdown again and my DB is rebuilt and ready to go.
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
Here is an Archive of Cryptocurrency App building Code on Github for anyone creating a Steemit app
https://steemit.com/steem/@marsresident/github-cryptocurrency-app-creation-archive
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit
a is abc
Downvoting a post can decrease pending rewards and make it less visible. Common reasons:
Submit