Main Page
From Synesketch Wiki
Welcome
Welcome to the official wiki of Synesketch, a textual emotion recognition and visualization software.
Synesketch is a free open-source engine and a library (Java API) based on the concept of synesthesia, or in other words: code that feels the words visually. It dynamically transfers the text into animated visual patterns.
Synesketch Video Demonstration
Introduction to Synesketch
The way we communicate and express ourselves has drastically changed in the last few decades. With computers we talk, draw, write, compose, read, listen, and collaborate; we create things and consume other things. Even fall in love. Information technologies became the main infrastructure of our modern culture. Thus, the artists of today have to be a little bit of engineers.
On the frontier of this human-oriented technological revolution – for computers are made for people, not vise versa – there are systems which can sense and cause emotions. Affective computer systems, using multimedia mining techniques, can sense some manifestations of people's feelings, while computer-based digital art and animation can really touch people's feelings.
Synesketch bridges those ideas together: code is serving as a medium between words, emotions, and images; between poetry and painting. Synesketch serves both as a practical sofware API and a media art installation. Also, it is the first of that kind that is also a free open-source project – not just a closed academic research experiment – so that the whole community can benefit from it. The name ‘Synesketch’ is a portmanteau of ‘synesthesia’ and ‘sketch’ – where ‘sketch’ does not only refer to drawings, but also to the Processing artworks called ‘sketches’.
Synesketch uses the WordNet-based lexicon of words and their emotional weights together with NLP heuristics to “feel” basic emotions in the text: happiness, sadness, fear, anger, disgust, and surprise (categories defined by Paul Ekman). Based on that, software generates appropriate visual patterns, written in Processing/Java. Colors and shapes of these patterns depend on the type and intensity of interpreted textual emotions. So far, there are two sample visualization systems, Hooloovoo and Synemania.
More about the way Synesketch works can be found in this paper, with in-depth description of the project (unfortunately, still in Serbian; soon to be published in English).
One important remark: our goal was not to make a truly empathetic software, but to use the software as a medium of enhancing human creativity. Read our statement for more information about our underlining ideas.
Synesketch is created by Uroš Krčadinac and friends, and is part of the GOOD OLD AI Research Network efforts at the University of Belgrade.
Using Synesketch
Our goal is to enhance creativity, learning, and research in several ways. Being an end-user, digital artist working with Processing, or a fully skilled Java developer, you can find Synesketch rewarding. To name a few of the use cases:
- Synesketch as an end-user application. If you want to see how our visualizations look like, check out the demostration section. We have also created a small demo application, Synesketch Empathybox, which illustrates the basic idea: automatic transformation between words and images. You can download it here. But, more much important then that, several real applications should be built on Synesketch. For example, a visual chat program – while we talk, colors and shapes are being generated. If you are interested in making such program, take a look at the examples.
- Synesketch as a tool for visual artists and designers. If you want to create your own visualizations based on textual emotions – Synesketch API will make that very easy for you! Practically, all you have to do is add one method into your Processing sketch. Visualizations we have created – Synemania and Hoovooloo – are just demonstrational ones. You can be more creative than us!
- Synesketch as a textual affect sensing and emotion recognition engine. If are interested only in the textual emotion sensing and recognition part of this project, Synesketch API will prove very useful. We have developed a Synesketch Lexicon – WordNet-based lexicon of words with their general and specific emotional weights (for Ekman’s emotion types: happiness, sadness, fear, anger, disgust, surprise). Our simple emotion model is easy to learn and use. With just one line of code, you can get estimated emotional content of one text fragment.
- Synesketch as an experimental polygon for synesthesia and real-time text visualization. If you are a real synesthete or just one more artist/scientist fascinated with synesthetia, Synesketch API will allow you to create your own synesthetia-like algorithms! Emotion recognition approach to word-to-image transformation is just one way to do it – it was just a starting point for us. You can visualize rhymes, different languages, text mistakes, anything. Thanks to our highly flexible architecture, mostly designed by design patterns, Synesketch API will make your extensions very easy to integrate with Synesketch.
So far, there were several great projects created with Synesketch, like a Karaoke lyrics visualizer or an emotional Twitter.
Getting started with Synesketch
If you wish to work/play with Synesketch, take a glance at the whole project:
- Gallery and Demo
- Statement
- Download
- Documentation
- Term Papers
- Examples
- Publications
- Web, Press & Festivals
- Synesketch-Based Projects
- Synesketch Logo
- mutuelle
And get involved:
- Register in this wiki and put some information about you.
- Subscribe to Synesketch Google Group.
- Create things and share it with us.
We wish you a happy and creative time with Synesketch!
