4K 60FPS video is coming but where are we going to store it? The unexplored potential of optical discs

Granted, DVD-writing capacity came a few years later, but at that time, the optical disc was the undisputed king of carrying video content. 

We know what happened next: For the last 20 years, the DVD with sizes of around 4.5 - 9GB has been one of the most widespread mediums for storing standard definition videos and movies, commercial program and games, even user data as a storage medium.

By the mid 2000's, buying 50 blank discs (4.7GB), would get you to near 1TB of storage but with just a fraction of the cost required for the equivalent 1TB storage from a hard drive.

The inexplicable "halt" in progression

At that point something weird happened: The evolution of the optical disc practically stopped in its tracks. While there were constant news about new optical technologies that were going to take us to the moon (in terms of storage capacity), we "somehow" got stuck with the DVD for the desktop PC. 

Research and experimental implementations continued even after the Blu-Ray and up to this day:

2004: "Pioneer has technology to develop 500GB Blu-Ray discs" 

2006: "How to burn a three terabyte CD"  

2007: 1 to 5 TB of data in a CD/DVD-size disc  

2010: TDK playing around with 1TB discs 

2013: Swinburne University’s Center for Micro-Photonics get capacity up to 1 Petabyte in DVD-sized discs, through the use of dual lasers

2016: Tiny Quartz-based optical disc stores 360TB with practically infinite lifespan 

...yet despite all these technological advances, we have seen none of it in the affordable consumer market. 

We still use CDs, DVDs and to a lesser extent Blu-Rays, which are all obsolete technologies (16-30+ year old techs). 

The format wars

There was a protracted period when HD-DVD and Blu-Ray were battling it out for the "next-generation" DVD, around 2003-2008, which was ultimately won by Blu-Ray. Ever since, Blu-Ray adoption was very slow. Computers were virtually cut-off from the DVD's successor (due to costs) while Sony focused more on the game platform and movie market. 

By the time Blu-Ray reader/writers for PCs and laptops became affordable (along with affordable blank discs), they were next to useless - despite the need to replace the obsolete DVD-R. 

Right now Blu-Ray size is small, cost/GB not that good, and their read/time writes suck. Hard disks, USB flash sticks and data cards (a 1TB SD card was just released by SanDisk) have an advantage in terms of throughput, size and rewritable status.

For the most part of the last 10 years, we are living in a distorted market in which PC users have been buying and stacking hard disks (!) to store their data, due to impractical alternatives (huh?) rather than actual technological constraints.

In the age of 20+ megapixel photos and 1080p30 or p60 videos, that even our phone can record, the rate at which we would fill 2TB disks is unsettling. On the other hand, what can we do? Delete our photos and videos? What good is a video if we can't store it, or if its storage costs too much?

The problem is only going to get even bigger with 4k60 video. Do you see any storage solutions in sight? We can continue to use hard disks, although since their $/GB ratio is not falling dramatically, and our video storage requirements are going up considerably, this means that the price that we'll be paying to store videos on our disks will be higher. And what about 8k video or even higher frame rates?

All these would be very different if we could use, say, 2TB to 4TB optical discs (in line with the size of our modern hard disks - which is something that optical technology was always good at matching or exceeding). It's a real shame that optical discs, which were always a tremendously cost effective storage technology, have been disallowed to scale by the market monopolies. And even when they are allowed to grow, their growth is "locked" to the movie market instead of the desktop. While a Blu Ray upgrade to >1TB would allow the movie studios to market higher quality movies to the consumers, this in no way solves our problem of where to store the data we create, in an affordable manner.

So what's going to happen next?

I think what we'll have, is a bit more of the 1080p case, with even bigger quality losses: We'll get a very toned-down 4K experience that is a shadow of what the technology can actually deliver. I'd go as far as to claim that the 4K that we'll get will be a marketing trick. And all this will be due to our lack of affordable storage capacity.

Truth is that a video can be 720p, 1080p or 4k, but this doesn't automatically translate to very high quality. Bitrate, on the other hand, is a great indicator of quality: A 4k stream with 1 mbps is worse compared to a 1080p with more mbits per second. And both of them would be way more detailed at 100 or 200 mbps instead of being in the 1-2-mbps range. 

What ultimately matters, for the quality of a video stream, is that there is enough information on the stream to adequately reproduce high levels of detail. As we increase the bitrate (mbits per second), the quality is increased while compression artifacts and noise are reduced. (As a note, and since I got a bit technical, mbits are not megabytes, so 8 Mbits per second = 1 MByte per second.) 

Sites like YouTube have every reason to compress their streams and reduce quality because Internet connections are still slow (average connection speed is currently at 6.3 Mbps). With most connections being bandwidth-limited, a tradeoff in quality is considered acceptable because the alternative scenario is a choppy video that stops every 5 seconds to download the next video segment. For most viewers this is unacceptable and therefore quality concessions have to be made. 

I'm now checking a 1080p60 video on youtube, and I'm actually being able to stream it at around 5Mbps. But when I use full screen, it's clear that a lot of the detail that a 1080 video should have, is not even there. Whether it is due to the camera recorder or youtube secondary compression, the result is that the video doesn't have the expected quality.

A crippled 4K?

If this is happening with 1080/60 video you can expect this phenomenon to increase in 4K video. Most cameras (whether it's a phone camera, digital cameras or video recorders) will not capture high bitrates that exploit the 4K-definition to the higher-end of its potential because then the videos would be too large and users would have to delete them all the time. Our local storage devices (hard disks) would be getting filled like crazy and there would be no way to store all this material without seriously compromising its quality. And therefore the compromise in quality will become a "necessity".

Again, it is important to understand that all this will happen because there is a market failure in the optical drive industry. If people could shoot 4K / 60fps with a streaming rate close to 250 Mbps (31.25MB/sec), it would require around 112GB per hour. With hard disks costing 100$ per 3TB (33$ / TB), it would mean that just an hour of video (112GB) would cost around 3$ to store it - which is absurd. But if we could store 30-40 hours in an optical disc that cost us less than a dollar, that would be a different issue altogether.

So now that we have the camera technology, the processors, the graphic cards, the dirt-cheap 4K TVs, etc etc, we will not be able to enjoy it because a few companies have decided to choke the storage market. 

We could, potentially, bypass this problem for streaming applications, but only if our Internet connections were in the 200MBps range, streaming very detailed 4K video through the net (or our Internet provider offering TV services over broadband). Yet we have to be realistic that this will not happen anytime soon for the average Internet user.

Conclusion

Optical storage has always been a technology that was not only "promising", but actually delivering -consistently- the best data storage that our money could buy. This has changed in the last 10 years. Quick progress was replaced by stagnation in developing new optical disc formats - despite the fact that technological innovations are actively pursued (even if they aren't marketed). 

In the last 10+ years there was no serious progress in upgrading capacity, cost efficiency (cost/GB stored), or speed (read/write throughput). Yet, high analysis photography and video, are two applications that require an ever-increasing amount of storage. Due to the lack of affordable storage, mainly in the form of writable optical discs for the desktop, we will be getting a very toned-down experience of what 4K/60fps videos are able to actually deliver, both in terms of movies, but also in terms of what we are able to record and reproduce - at least until optical discs are upgraded significantly.