A collaborative project with Xixi Zhan.
We habitually appreciate paintings by “looking at” them, instead of “listening from” them. We usually see the final results yet seldom see the process of drawing. Everytime I look at a painting I always wonder how it had been painted. Just as the idea about process art which emphasized the process of the formation of art. It must be very interesting when "investigating and documenting the artistic production process” [1]. I believe every painting has its unique journey behind.
As Chinese painting is a kind of expressionist brushwork, the artists not only emphasize the simplicity of the painted object, but also pay attention to the harmony between shapes and the composition in their works. However, we can hardly see the whole process of the painting and the painter's hand movements. What if I can record them and transform the whole process of painting into a sound piece, so that we can perceive the paintings by listening to a song generated by itself.
This project aims at exploring the possibility of combining digital sound manipulation together with painting artwork and creating a fresh experience of perceiving the Chinese traditional ink painting.
In this experimental project, the trail of painter's motion of the hand will be recorded as the painter executing the inked brush. Next, by using machine learning techniques, those movements will be digitized and used as parameters in order to control audio samples. Hence, each monochrome ink painting will generate a unique sound piece as the process of painting is kind of unique and can be different every time.
Behind-the-scenes is a project collaborate with my mom, a painter. We choose four paintings: plum, orchid, bamboo and chrysanthemum, which symbolize the four gentlemen in Confucianism. Plum blossoms serve as a metaphor for inner beauty and humble display under adverse conditions. Orchids are a symbol of divine purity and are the fragrance of the royalty. They are always associated with elegance. Bamboo is a metaphor of vitality, longevity, spiritual truth, and open-mindedness. Lastly, chrysanthemum is a symbol of high moral character and strong will. It is tranquil and harmonious with others, but also dignified and indomitable [2]. Hence, I select four traditional Chinese instruments to represent the four gentlemen. They are: Yangqin for plum, Guzheng for Orchids, Dizi for bamboo and finally, Erhu for chrysanthemum. I choose these four instruments because their different timbre that sort of correspond to the characteristics of the four gentlemen.
- Diagram of the workflow -
REFERENCES
1. Angie, K. (2016, August 17). Understanding the Origin and Legacy of Process Art. Retrieved from: https://www.widewalls.ch/process-art-artists-history/
2. The Four Gentlemen (四君子) Retrieved from: http://www.chinaonlinemuseum.com/painting-four-gentlemen.php
3. Scaletti, C. (1994). Sound synthesis algorithms for auditory data representations. In G. Kramer (Ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Addison-Wesley. 223--251.
5. Audio samples from http://www.freesound.org
Some code are adapted from:
“Pitch-shifting” patch by Nikolas K
“OSC Data Plotter” Processing sketch from Data and Machine Learning for Artistic Practice (2017-18)
“Simple sine control” patch from Data and Machine Learning for Artistic Practice (2017-18)