Many newcomers often ask themselves: what is a wallet? what is an address? what's a private key? and how can I keep them SECURE form theft?

Addresses and Private Keys

Let's put it simple. An address is a public identifier and you can prove that you own that address with your private key (which only you must possess!).

You may wish to send X coins to someone else and use your private key to digitally sign the transaction for a destination address.
Everyone else can confirm the signature is yours by using your public key.

This is called asymmetric encryption. Let's not dive too much into this though, as I just want to explain the generic meaning of each concept. The example below explains more or less what I've just described.

Wallets

A wallet can be seen as a key-ring, i.e. a group of private keys. A wallet is usually encrypted using a password, and stores a group of private keys. The public keys can be easily computed from the private counterparts by the wallet software.

How and where to secure them all?

Software Wallets

Most wallets are software-based, i.e. you install them in your computer or mobile device, and you can control everything there (e.g. send coins to someone). They're usually password protected and may or may not require a passphrase for creation (which must be stored securely as it can be used to restore your wallet).

I go a bit technical below, you can skip it:

You may ask yourself: how can my private keys and addresses be re-generated from a passphrase? Those wallets use key derivation algorithms that use your passphrase as input and generate private keys as output.

Web-based Wallets

Alternatively, you can delegate those responsibilities to a web-based wallet, such as Blockchain.info and CoinBase.com. They normally use your password to encrypt your data, so that not even them can access your private keys.

This, of course, requires you to trust them (and their encryption algorithms), as well as hope that their system is always up and running.

Hardware Wallets

A hardware wallet can be used to securely (supposedly) store private keys in a hardware device.
These devices are normally safe from viruses and can be connected to your PC via USB.

Examples include TREZOR, Ledger HW.1, Ledger Nano, KeepKey, CoolWallet.

The idea behind it is simple: generate and store the private keys inside and don't let them out.
This should work for most users, and as long as you keep it in a safe place and don't lose your passphrase (a list of words that is used to re-generate your wallet), you should be able to restore a lost or damaged hardware wallet.

I go a bit technical below, you can skip it:

The "software security" they state is misleading though, because nowadays, hardware attacks are becoming more and more easy to perform - even without any access to the software within it (also called a Side-Channel Attack).

For example, according to https://jochen-hoenicke.de/trezor-power-analysis/, the first TREZOR versions were actually subject to Side-Channel Attacks, which would allow an attacker to retrieve your private keys, should the attacker have your TREZOR in their hands.

Apparently, they patched it by requiring a PIN to compute the public key. The hardware itself should still vulnerable to power analysis though as (according to that article) only the firmware was patched.

However, this is the kind of vulnerabilities hardware wallets may have.

How do you keep them secure?

I use a software wallet

Keep your wallet in more than one place so that you can ensure redundancy (you never know if you're going to lose your passphrase and the original file gets damaged). Make sure the wallet software supports encryption and choose a large password (>256-bit). Store it, along with the passphrase, in a secure offline location (or even paper...).

I go a bit technical below, you can skip it:

I'd suggest having a laptop, desktop, mobile phone, or any other device that is not connected to any network and comes with a fresh installation of its operating-system. Install True Crypt 7.1a (https://www.grc.com/misc/truecrypt/truecrypt.htm) on that device and create an encrypted volume (e.g. with AES-Twofish-Serpent). You can even password protect the devices as well.

Then, store within it, a file containing the wallet encryption password and wallet passphrase. If possible, do this across at least two devices to ensure you have a fail-safe. Now you need to store your volume encryption password in a safe-place (or simply remember it).

I use a web-based wallet

If you have a lot of funds, consider moving them. You never know when the web-based wallet might be hacked, shut-down or simply lose your data. In any case, if you still want to use it, make sure to store all your account data (passwords, passphrases, ids, etc) in a secure offline location (or multiple, in case the first gets lost or damaged).

I use a hardware wallet

Probably the wisest choice. Still, if it gets stolen, a hacker might still perform certain types of attacks and retrieve your private keys - therefore gaining access to your funds. You may still need to password/PIN-protect it, in which case you'll probably need to store it somewhere safe as well.

Be cautious when buying one: read reviews, check for possible exploits available on the internet, analyse the security features of each product and finally make your decision.

Summary

Hardware wallets might be your best bet. In case you can't afford them, try to focus on software-based wallets rather than web-based (unless you fully trust them and find them ultra reliable) - I'm probably gonna get crushed for saying this! A good wallet that doesn't need the whole blockchain in your hard drive is MultiBit - and it can integrate with TREZOR for example. Armory offers the most security features, but it requires the whole blockchain to be stored in your computer. The choice is yours :) You can find a list of wallets and their details here: https://www.weusecoins.com/en/find-the-best-bitcoin-wallet/

If you really want to go for web-based wallets, I currently use Blockchain.info and CoinBase.com (they integrate with Exchange websites).

The key concepts to remember are: strong encryption, backups, password and passphrase stored in secure (offline) location(s).

The future of wallets?

I'd say with the new FPGA and System-on-Chip models, especially from Microsemi (https://www.microsemi.com/products/fpga-soc/soc-fpga/smartfusion2), it would be interesting to see a hardware wallet built with that. They are now providing moderately cheap and highly secure chips, that are protected against hardware-based attacks, allowing for hardware designers to create extremely secure applications for its end users.

I wouldn't be surprised to see something built with this in the near future.