The beauty of text and words is often associated with the meaning behind them and the sound of their pronunciations. What if we blur the meaning, mute the pronunciation? What's left?
"Fairytale and Television" is a sonification and visualization of a Japanese fairytale audiobook, done in Unity. This piece explores the use of light and shadow, as well as word classes in written text and rhythm in spoken language.
Each word in the audiobook is mapped to a television. Depending on the class of word, for example verb/noun, televisions with different patterns will fall from the sky. Each type of television also has its own sound bank. Once a television falls into the space, its sounds will be played based on an aleatory statistical model. Whenever its sound plays, the specific television turns on and displays its visual pattern.
