Main Page
From Synesketch Wiki
Welcome
Welcome to the official wiki of Synesketch, an artistic software library with synesthesia. It is a Web's first free open-source textual emotion recognition and visualization engine – code that feels the words visually!
Introduction to Synesketch
The way we communicate and express ourselves has drastically changed in the last few decades. With computers we talk, draw, write, compose, read, listen, and collaborate; we create things and consume other things. Even fall in love. Information technologies became the main infrastructure of our modern culture. Thus, the artists of today have to be a little bit of engineers.
On the frontier of this human-oriented technological revolution – for computers are made for people, not vise versa – there are systems which can sense and cause emotions. Affective computer systems, using multimedia mining techniques, can sense some manifestations of people's feelings, while computer-based digital art and animation can really touch people's feelings.
Synesketch follows the same train of thought. It is a software library (Java API) for sensing and creative visualization of textual emotions! Code is serving as a medium between words, emotions, and images; between poetry and painting. Gluing computers with our emotions and aesthetic experiences, it is both a software innovation and an interactive art experiment. Also, it is the first of that kind that is also a free open-source project – not just a closed academic research experiment – so that the whole community can benefit from it. The name ‘Synesketch’ is a portmanteau of ‘synesthesia’ and ‘sketch’ – where ‘sketch’ does not only refer to drawings, but also to the Processing artworks called ‘sketches’.
Synesketch is a result of a research that spreads out through several diverse fields – from natural language processing techniques based on WordNet, across Ekman's research of emotions, to color psychology, visual design, data visualizations, and affective computing. Graphics were done by Processing, a great tool for programming visual art. More about the way Synesketch works can be found in this paper with in-depth description of the project (unfortunately, still in Serbian).
One important remark: our goal was not to make a truly empathetic software, but to use the software as a medium of enhancing human creativity. Read our statement for more information about it's idea and goal.
Synesketch is created by Uroš Krčadinac and friends, and is part of the Good Old AI research group efforts at the University of Belgrade.
Using Synesketch
Our goal is to enhance creativity, learning, and research in several ways. Being an end-user, digital artist working with Processing, or a fully skilled Java developer, you can find Synesketch rewarding. To name a few of the use cases:
- Synesketch as an end-user application. If you want to see how our visualizations look like, check out the demostration section. We have also created a small demo application, Synesketch Empathybox, which illustrates the basic idea: automatic transformation between words and images. You can download it here. But, more much important then that, several real applications should be built on Synesketch. For example, a visual chat program – while we talk, colors and shapes are being generated. If you are interested in making such program, take a look at the examples.
- Synesketch as a tool for visual artists and designers. If you want to create your own visualizations based on textual emotions – Synesketch API will make that very easy for you! Practically, all you have to do is add one method into your Processing sketch. Visualizations we have created – Synemania and Hoovooloo – are just demonstrational ones. You can be more creative than us!
- Synesketch as a textual affect sensing and emotion recognition engine. If are interested only in the textual emotion sensing and recognition part of this project, Synesketch API will prove very useful. We have developed a Synesketch Lexicon – WordNet-based lexicon of words with their general and specific emotional weights (for Ekman’s emotion types: happiness, sadness, fear, anger, disgust, surprise). Our simple emotion model is easy to learn and use. With just one line of code, you can get estimated emotional content of one text fragment.
- Synesketch as an experimental polygon for synesthesia and real-time text visualization. If you are a real synesthete or just one more artist/scientist fascinated with synesthetia, Synesketch API will allow you to create your own synesthetia-like algorithms! Emotion recognition approach to word-to-image transformation is just one way to do it – it was just a starting point for us. You can visualize rhymes, different languages, text mistakes, anything. Thanks to our highly flexible architecture, mostly designed by design patterns, Synesketch API will make your extensions very easy to integrate with Synesketch.
Getting started with Synesketch
If you wish to work/play with Synesketch, take a glance at the whole project:
And get involved:
- Register in this wiki and put some information about you.
- Subscribe to Synesketch Google Group.
- Create things and share it with us.
We wish you a happy and creative time with Synesketch!
