Neural Decorators

This post was originally published on Medium years ago.

---

I was watching the new TV show Humans the other day and in it, an older android model starts to lose its memory — not completely missing, but deteriorating, like in people with Alzheimer’s.

In computers (and, therefore, androids), data either is or is not there. There is no “deterioration” — it’s 1 or 0, it’s present or absent. Sure, some datasets can be partially corrupted and you can then retrieve the remaining parts, if they’re still readable without the missing blocks, but there’s no “weakened” memory. It’s not like you can fire up your computer that had Bioshock installed and one day it’ll be running System Shock because they’re kind of similar.

Now, ignoring the show’s other inaccuracies, this one with the memory seemed interesting. What if, in the context of the TV show’s world, that’s how they solved the problem of artificial intelligence? With Neural Decorators.

Disclaimer: I have done literally zero research on this so please let me know if a concept such as this one exists (a version of neural networks perhaps), but here I’m calling it neural decorators because of a software development pattern — the decorator pattern. Let’s explain that first.

The Decorator Pattern

Let’s explain it in non-coder terms first.

Decorating in reality

Imagine a gift — unwrapped.

The purpose of the gift (not the iPhone’s specific purpose, but the purpose of the term “gift” itself) is to give the recipient satisfaction and surprise. We then put the gift into a pretty box.

The end-goal of this box is to delight you with its prettiness and, ultimately, to give you what’s inside so the original purpose can be fulfilled — giving of satisfaction and surprise. Now, around this box we put a pretty bow.

The purpose of this bow is the same as the purpose of the box before — delighting. Only this one needs to delight further, but in a slightly different way. Then, we put the entire wrapped thing into a pretty gift bag.

I enjoyed drawing this post’s images more than I thought I would.

The bag’s purpose is twofold: making it easier to transport the gift and, once again, delighting with appearance.

In the example above, every “layer” — the box, the bow and the bag have the same “action” which produces the same “output”. The action is “open” and the output is “next layer”, until the final layer (the gift) is reached. Every layer is called a “decorator”, and wrapping up final output like that is called “the decorator pattern”.

Every layer enhances the final outcome (the outcome being pleasure, not the iPhone itself), but every layer is optional or perhaps more accurately, interchangeable. You can easily switch out a box for another, a bag for another, or even add more boxes into the whole configuration, put a bag into a bag and so on — the only thing that matters is that the output and input stay of the same type: gift decorator goes in, gift decorator comes out. That way, we can keep stacking gift decorators one on top of the other, and the recipient of the gift will always get to the gift sooner or later, but the unwrapping experience may vary.

Decorating in Software

Ok, so how is this a software development thing?

Image credit: https://www.pinterest.com/pin/249527635575912009/

No, not like this.

Here’s a brief example (you don’t really need to be familiar with programming to understand this).

When you make an account on a forum (are those still a thing?), you are generally given the lowest possible role by default (guest). Only once you’re approved in one way or another do you get additional permissions (a moderator needs to click the “approve user” button next to your username, for example). So, on a forum, you are given basic “read” permissions with your new account. You can read all topics. The account is stored in the database with a flag that says “guest” under the “roles” field meaning you have the role of a guest.

This is a theoretical example. Please don’t design your databases like this.

To be able to post, a new role needs to be added into the mix. Let’s say we add “user”, so the roles field looks like this: “guest, user”. The “user” role is a decorator of the “guest” role. In other words, all accounts are based off of “guest” and have new abilities added to them by means of a decorator. Once “decorated” with a “user”, the “guest” can make topics. Going further, if we want to administrate a particular sub-forum on the forum (i.e. become able to delete/approve accounts and topics), yet another role (and thus decorator) would be added to the roles field: “guest, user, php-mod”, letting the user moderate PHP topics. The same “role” can be applied to several users — the role is, of course, reusable.

As above, the only thing that is important is that the decorator pattern used takes the same type of input as it gives output — a “user”, decorated or not. So when the forum system (as in, the software running the forum website) looks at someone to see if they’re allowed to do something (e.g. someone tried to post a topic by clicking the “New topic” link), it always gets a user back after going through all the decorators and piling abilities onto the account.

Depending on the decorators it encounters, the action will either be allowed or denied, but that’s outside the scope of the decorator pattern — the pattern is there only to let the system know which abilities an account has, it’s up to the system (and a different sub-system) to enforce those restrictions.

Neural Decorators

Ok. So how does all this fit into the idea of neural decorators?

First, we should talk about memory.

Memory

What is a memory? By that I mean one “instance” of a memory. Let’s go wild with an ad-hoc definition:

A memory is a recollection of an event or feeling from one’s past, in the degree that it can be retold in a manner confident enough to make it believable to the recipients.

Makes sense, right? Let’s see an example.

When I was 7, I almost drowned in a public pool in the center of our city park. Teo, a friend I was with, jumped in and bailed me out. (true story)

There was a slide, but I had never slid down a waterslide before and was reluctant to even think about it. Though I wasn’t yet the able swimmer I am today, after much convincing Teo talked me into sliding down, saying the end of the pool where the slide ends is kid-friendly and quite shallow.

I did, and immediately panicked as I sunk, not feeling the bottom of the obviously deep pool. I gasped and flailed for minutes, until Teo jumped in and started pulling me out — he had been taking swimming lessons for a while at that time, and had even been doing some semi competitive swimming. I remember him showing us the scratch marks of my fingernails (I had scratched him bloody on both his chest and back), for I flailed like a dead squid on soy sauce while he was pulling me out.

-

When, a couple of years back, this story randomly came up around my family, it turned out our versions were slightly different. My dad, who was there, issued the following corrections:

• the pool was in fact not one in which I could have drowned and was actually really shallow. If I just took the time to calm down, remember my training and put my feet down, I would have “survived”.

-

• it wasn’t really minutes of panic and drowning, it was more like six seconds.
• Teo hadn’t been a competitive swimmer until years later.
• There was a total of three to four scratch marks, each no more bloody than a scratched mosquito bite.

The more I thought about these “corrections”, the more I started to believe them and my memory was rewritten to match my father’s story. We then recounted several other childhood memories and “corrected” those. I was taken aback by the amount of misinformation sitting in my head.

So… my memory was false. Does the previous definition then no longer hold true? If I believe something firmly enough, I can definitely convey it to others in a believable manner. But then it’s a lie. A fabrication. Fiction, if you will. I am then no more historically accurate than Harry Potter, or the Bible. At which point do our memories become just… stories? How much alteration can we inflict upon them before turning them into fantasy? And why does it take me so damn long to get to the point of this post?

Calm down, reader. Here we go.

Neural Decorators

I believe our memories are a set of neural decorators, and that losing memory means removing some of those decorators, and the underlying memory submodule of the brain then makes up bogus ones to cover for those removed (purposely or by accident), if it can.

Sometimes, we even willingly remove some decorators if they fit the context better (maybe there was no drowning at all?) or replace them with better alternatives — maybe I’ll impress a girl more if I saved Teo rather than the other way around. It can also happen that we use this new decorator so much, the memory submodule sets it as a default and our memory is altered so much we need to rely on external corrector agents (like my father’s superior memory) to set things straight.

Let me recap all that:

A memory is a set of neural decorators, each decorating a common base memory. Decorators are interchangebale, and overuse can permanently install these overrides. When a memory is completely lost, it means the base memory was lost.

You know those transparent sheets (cel sheets) on which you can draw something, then use it to overlay over another, and another, and another, to get a fully composed image? It’s actually how they used to do animation back in the day. That’s how I see decorators.

This decorative approach of a single memory (in my case “drowning”) would work something like this:

You can see here that some decorators decorate decorators, while others decorate the memory directly. So the memory’s location is Cakovec, and a park. Neither influences the other, my memory might one day change and I might start thinking it happened in Rijeka, where I lived for 10 years, or that the pools weren’t in the park at all but at an entirely separate venue. The rest of the memory would stay intact.

On the other hand, I remember it happening in a pool, but there is a separate decorator for the pool’s depth, which got overridden by my father’s depth decorator. Same for scratching which, in turn, might have other decorators like “badly” or “barely”.

Though only a few were displayed here, there are millions of decorators for that one memory and all adjacent decorators — from the season and time of day, to feelings, to the number of pools in the pool complex, and so on.

Decorator Reuse

Which makes me think - what if the secret to our brain’s seemingly immense capacity is decorator reuse?

In the above story, I have a “Happened in a park” decorator. What if all my memories that happened in parks use this same decorator? What’s more, the concept of “park” in that decorator isn’t location-bound. It’s “a park”. The exact location of the park (the town of Cakovec) is yet another decorator, and that one is reused on all memories that happened in that town.

You can see here something of an extension of the previous image, where the Drowning memory is pushed off to the left, with Park becoming centerpiece and linking other memories together. As indicated by the different connector lines, we can see I had my first kiss in the park of the same town where I got drunk and had a picnic, and that I remember turtles in the park of the town where I went to university. Of course, additional links could be made — the park of the town where I went to university is a memory on its own, and it’s not the only park there, so additional space-time decorators would be needed to achieve precision. Basically, there is an infinite number of applicable decorators to each base memory, and each memory also can be a decorator to another decorator or base memory.

It is this seemingly infinite combination of atomic, separate decorators that makes up our various memories, and when a memory loses a decorator, that decorator isn’t lost — it is merely removed from the memory’s hierarchy, from its connection chain, but stays in our brain, stored and ready for reuse on other memories.

This would also be why people with memory problems can sometimes remember a vague concept of a park, or a pleasant park walk, but fail to pinpoint where it happened, or who they were with, or when. Maybe they encountered the decorator on its own and without a base memory to decorate, so they get the sensation of just the term of the decorator itself.

Mind. Blown.

So, in the above story of my father setting the record straight with his recollection, he actually copied over some of his decorators to me, and told me which ones of mine to replace. Or, in the context of the “gift box” story above — my gift box got switched out for another, one more fitting for the gift. We could say that the iPhone was given its original packaging, rather than being gifted in a Samsung box.

Conclusion

This is what I meant when I said "What if, in the context of the TV show’s world, that’s how they solved the problem of artificial intelligence?"— what if this is the solution to artificial memory? Not knowledge graphs or SQL tables, but decorator databases? What if the android in the show is losing its memory because the databanks that hold some of the decorators are getting corrupted? Thus, the android is losing some decorators, but the base remains and it tries to fill in the gaps of the missing decorators by taking suggestions from surroundings or just making things up? Which makes me wonder…

If we keep switching the decorators out unknowingly, how can we trust our memories? And, at which point does a memory stop being a memory and turns into a lie?

It’s the age-old question of — if we could switch out failing body parts for synthetic ones, at which point do we stop being “we”, and become a machine? Would my brain in a jar still be “me”? Is that enough? If my malfunctioning thyroid generates too many hormones (which then influence my thoughts and feelings) gets replaced or removed, am I still me despite the dramatic character change brought upon by the now normal hormonal levels?

Would a base memory, without decorators, still be my memory? Would one that is mis-decorated? I don’t know… right now, I’m thinking about the drowning story and I think I remember us catching some white mice under a plastic bucket, but I have no idea if I just imagined it and plugged in a new decorator or if it really happened… In any case, I think it all makes for an interesting discussion.

What do you think? Does this make sense? Is it a well established neural-network “well duh!” party that I’m just very late to? Let me know. I’m on Twitter.