All the haiku in the world

Why should he work on a trial-and-error basis when he could attack the problem scientifically? An old paper by Shannon (1950) enlightened him. He used Zipf’s law to estimate the entropy of the Japanese language (in bit per hiragana); he obtained a value of 3.14 ± 0.04. The characters in the traditional syllabary are 48, but this does not mean that the total number of possible haiku is 48¹⁷: most combinations are not even valid Japanese sentences, let alone haiku. Luckily so. 48¹⁷ is an unmanageable figure. The space he needed to examine was way smaller: approximately 2^3.14 × 17 sentences. How many of such sentences were actually haiku? One in ten thousands? Probably less than that. But it was irrelevant. He just needed to collect all of them to catch all the haiku as well. That was good enough for him. Any author knows he needs to fill dozens of uninspired pages to get to a single notable passage; he was bound to an extreme formulation of the same rule. At least he would be spared the grueling struggle of composition.

He repurposed an old warehouse of his in Yogasakihama (Miyagi prefecture). A new power line was installed together with better wiring and a dedicated refrigeration system. He used some personal acquaintances at Toshiba to gather up a hundred thousands 10-TB hard drives at a reasonable price, to be stacked and cabled in the warehouse. He contacted Moriyama, an old friend from high school, to design a neural network to go through the cacophony of random combination and identify only the admissible constructs, at a rate of one billion per minute. He trained the system on the classical corpus of the Bunruihaiku-zenshū and then tested it on a large inventory of sentences from thirty years of daily edition of the Asahi Shinbun and Yomiuri Shinbun. The results were excellent: the first ten thousand sentences were all grammatically correct; many of them were indistinguishable from actual headlines (“bald sixty-year old/falls on his knees while walking/in Harajuku”); others bordered on nonsense in a way that a surrealist may have liked (“a murderous pear/grins at a famed sugarcube/and goes on singing”). None was a haiku. But the system was certainly working. Before starting the execution, he adjusted the code to automatically reject any poem from the above mentioned Bunruihaiku-zenshū and from a number of more recent anthologies: he didn’t want to claim haiku that other people had already written. He was interest only on haikus that (due to laziness or lack of method) had not yet been discovered.

The processing was bound to last for centuries. New technologies along the way would probably speed it up, even dramatically. But he was a patient man anyway. He retired in a wooden cabin he had built in the park around the warehouse, not dissimilar to master Bashō’s own shelters on his way to Tōhoku. At night time, the whir of the machines blended with the sound of the wind blowing through the bay and lulled him to sleep. When he couldn’t sleep, he went out to smoke a cigarette and he surveyed the darkness to discern the invisible profile of Matsushima in front of him. He couldn’t help thinking that sooner or later the system was bound to find a haiku describing that very moment, that very feeling. His mind, once trained to count syllables, was finally empty and horizontal like the surface of the Ocean.

(Italian original here).