The great RAM debate

EOS is a blockchain-based infrastructure allowing anyone to deploy decentralized applications. EOSIO software provides access to three types of resources : CPU (processing power), bandwith, and RAM.

End users on EOS don't have to pay any fees to make transactions on the blockchain, but a developer who wants to deploy a decentralized application has to pay for informatic resources he needs. Random-access Memory is needed to store account informations, and especially to store contracts' states. We have to keep in mind that RAM is one of the main factors which allow EOS to process a very high transaction volume per second : if those datas were disk-stored, the time to access disk would limit the throughput.

Every EOS full node has an amount of RAM dedicated to store the data of this smart contract infrastructure. The amount of RAM given by every full node is coded in the software. RAM is currently a scarce resource on EOS, as it is limited today to 64 GB. It is therefore necessary to allocate it in an optimal way.

How does Bancor Relay works ?

The system which allows EOS users to buy and sell RAM is Bancor's algorithm, known as Bancor Relay. It was implemented on Dawn 4.0 release of EOSIO software : at first, EOSIO System Contract was allocating RAM to users who staked tokens - it was abandoned because of high risks of shortage in resources; Block.one decided to switch to a market-based allocation (yes, we don't talk about a free market as RAM supply is controled by BPs).

Bancor Relay facilitates price discovery and provides liquidity in the EOS RAM market, by allowing users to buy and sell RAM for EOS tokens at previously established market prices. Bancor Relay is basically RAM market maker's algorithm. Prices are determined through the following parameters : RAM outstanding supply, connector balance (in EOS) and connector weight (the ratio between RAM total value and its connector balance).

We won't enter in the details here, but Bancor's pricing formula is detailed in Bancor's whitepaper.

The issue with this algorithm used in a scarce market like RAM in EOS is that hoarders can make the price infinitely growing for little costs.

Speculation and issues with RAM market

A substantial amount of RAM is needed by DApps developers. If developers have to pay a price which is far too high, deployment of decentralized applications becomes unaffordable.

Traders, on the other side, don't really care about resources allocation or fair price : they care about making money, and by speculating on the future demand for RAM when EOS ecosystem will be widely used, they are pushing prices higher.

At the time of writing, the cost of RAM is ~ 0,18 EOS per kilobyte. It is better than the 0,9 EOS / kB peak that came in early july, but it is still very expansive if you want to deploy a large scale DApp or make an airdrop to all EOS accounts.

RAM seems expansive for the average user, but he only has to pay for it once, during account creation. For the moment, an account needs 4 kB of RAM, but the system contract will be updated soon to lower this amount to 1,5 kB. After the initial payment for RAM allocation, all the transaction fees are free : paying less than 10 dollars to create an account allowing to make a lot of transactions without fees is not very expansive.

From the developers point of view, however, it becomes very costly to deploy a DApp which will be widely used. Moreover, many accounts created will be unused : it's basically spewing money for the developer.

One of the promises of EOS is financial inclusion at large scale. But who can afford those costs in a poor country where the account creation cost is equivalent to a weekly income ?

RAM prices have of course a very big impact on airdrop costs. The cost of an airdrop to all EOS accounts is something like 10 000 EOS, which is a lot of money. But is it a bad thing ? We can assume that if the costs are very low, EOS network will be spammed with projects and tokens which have no meaning or no value. On the other hand, if the costs are too high, airdrops, which are an essential part of EOS business model and marketing, will basically become impossible. As a solution, we could pass RAM's cost on users, or let BPs provide account creation services.

The problem with RAM market is that you can only buy or sell RAM. If the users had the ability to lease it, it will mitigate the problems mentioned above. It is a proposal supported by many BPs. As LiquidEOS said, just imagine a real estate market where you can only buy or sell houses, without any possibility to rent them : there would be plenty of empty houses on the market, only detained by speculators driving the prices artificially high on the market. A potential solution is to make sure the resources utilization as high as possible, but without hurting the free market principles. One proposal could be to establish a kind of property tax, making RAM holding a bad long term investment.

Another issue rely on the fact that there are very few costs for speculation. When allowing speculation on a finite and usefull resource, speculators should pay fees to discourage them to buy the RAM in advance, thinking that developers will need it in the future and will pay a higher price to access those resources. EOS community should choose what to do with RAM trading collected fees. However, pay directly for RAM in EOS from user's account is not a solution which goes with EOS' principles.

Potential solutions and proposals :

Dan Larimer made a first proposal in early july :

EOSIO RAM Market & Bancor Algorithm

First step is to increase RAM supply. He suggested a 64 GB / year rate (that's 1 kB per block). Block producers can modify this rate of increase.
Second step is to lower account memory usage : after auditing RAM usage per account, it appeared that the 3 kB / account was "overly conservative". The future update to the system contract will create 1.5 kB of RAM per account - this will cut in half the cost of account creation.
Third step depends of the iOS wallet Block.one is building : the team plans on offering an account to each user.

Dan Larimer is right when saying that RAM is a scarce resource, but Moore's law isn't the cause of this scarcity. Real reason is the current market-based model, which fails to allocate RAM in an efficient way. Moreover, 64 GB is a very low amount to start with.

To fix RAM issue, we have to create an environment which strongly discourage speculation. Several solutions have been proposed by EOS block producers and members. There are two main areas for improvment :

Discouraging speculation by putting financial mechanisms in place which make RAM speculation less attractive ;
Establishing a leasing market for RAM which allows users to rent this resource.

One of the solutions is to allow RAM futures and split RAM on two different markets : a buy/sell market and a rent market. If RAM becomes rentable, there will be far more utilization and it will be pushing RAM price down.

Another solution could be to give their own place to speculators and treat RAM on an exchange. It could list several trading pairs on RAM, and one could buy and sell RAM as easily as a token. It will lower the entry barrier, but it will increase volatility.

Bancor's proposal :

A Proposed Solution to EOS RAM Allocation

There are two key points in the solution proposed by Bancor to this unaffordable RAM price problem.

Changing the RAM contract to burn EOS tokens at a constant rate, let's say 10 % per month. This will cut the incentive to hold RAM for long periods.
Enabling futures contracts on RAM price for the next 64 GB to come, allowing speculators to bet on future price without preventing developers from accessing this resource at an affordable price.

A RAM leasing market :

There are some difficulties to overcome with RAM leasing : what happens to the data if the rent isn't paid ? It could be very damaging to end-users if the system wipes it out in case of a default of payment. So we have to put in place strong incentives to pay the rent. There are two ways to incentivize users :

Locking up tokens that will be used to pay the rent if it’s not done voluntarily, while indexing rent cost on the size of the "bond". The more tokens are locked, cheaper is the rent ;
Account freezing in case of default in payment.

Greymass' proposal : separating hot and cold data

One way to increase RAM supply while having an optimal allocation would be not to store cold data directly on physical memory, but by using disk space. This a very elegant and interesting solution proposed by Greymass (who are doing an amazing work for EOS), but it implies very substantial changes in the system contract.

The idea is to have a feeless, automatic and transparent RAM leasing system which enable users to switch data from "cold" to "hot" depending on their needs and on their resources. What is very interesting with this design is that even with the actual 64 GB RAM limit, less than 10 TB of disk space will be enough to store more than a billion accounts' data.

As mentioned by Greymass, increasing physical RAM can be dangerous, as it impacts every single node on the network, and not just the block producers. This could raise a lot the barrier for new nodes to enter, due to very high technical specifications needed.

Greymass suggests to separate liveness from persistance. The idea is to have a system where "persistence is guaranteed, but liveness is not" : unused data will move from physical RAM to disk.

Live storage will have read/write permission ;
Persistent storage will only have activation permission.

While disk will be a resource available through staked EOS, RAM will be acquired through leasing and backed by disk. The leasing system would be based on an auto-renewing resource, like fuel. The leasing prices would be correlated to system utilization. In case of high demand, Bancor algorithm would be pushing fuel price, and it could become costly to activate disk resources into RAM. Users would have to wait for less utilization or to add stake more EOS to increase the total amount of available fuel they have to use.

Assuming the user is willing and able to pay the fuel for the lease, which may be cheap if few leases are outstanding, or expensive when outstanding leases approach system RAM capacity. However, as leases will expire, a user may opt to wait if prices are prohibitively expensive for their use case.

With automatization, the lease would only be possible if user is able to pay for it. The data would automatically go from RAM to disk and from disk to RAM depending on user's activity and staked EOS tokens.

It would be still profitable to speculate a little bit on RAM leasing prices, but it will be much less damaging for the accessibility of this resource :

The market for leasing may have some slight speculation in it -- a user might speculate it is better to acquire a lease now, if they anticipate they will use it soon and the price will go up soon (as leases themselves can still use the Bancor algorithm to determine the right price for a given lease). However, not using a lease is simply a waste of time for a true speculator, as there is no longer any concept of selling. You merely acquire the right to use the liveness of RAM for your data for a certain amount of time.

The performance implications could be mitigated by disallowing transactions updating their data for an amount of time depending on the data size.

For more details, read this excellent post by Greymass :

Leasing EOS RAM with Zero Fees and No Forceful Memory Frees

Filipe Alonso's proposal :

Filipe Alonso made on Coinmonks a very relevant proposal regarding RAM issue, divided in two major articles :

We won't describe in details this proposal here, because those two articles are certainly the best available on the subject and must be read closely by every EOS member interested in this topic. The idea is :

Driving RAM accessibility costs per usage ;
Controlling RAM distribution with inflation/deflation mechanisms.

The function of RAM production, then, is solely that of providing the network with enough resources to grow in a sustainable manner. Unlike collateral supply, which can increase and decrease according to memory requirements through inflation and deflation, RAM production can and should only increase for obvious reasons related to space usage.

franceos' favourite solution :

We don't want RAM to become a scarce and costly commodity for EOS users, but a technical resource with as little incentive to speculate on it as possible. We are in favour of a leasing market for RAM. From our point of view, it is the best way to allocate efficiently this resource : speculators won't be incentivized to hoard RAM and its distribution will be driven by the real technical needs of the network. Developers as well as users will be able to rent the required amount needed; it will reduce costs, and mechanically ensure a way better allocation.

As such, we strongly support Greymass' proposal for a RAM leasing market.

The debate is not over and we are sure EOS community will agree on a solution to fix this issue which could be very domageable for financial inclusion in our nascent crypto-ecosystem. Stay tuned !