Picking a Graphics Library

There's some Creative Pact thing that people do for the month of September. I've decided that I should work on my commission for the Vocal Constructivists. I'm going to generate some graphics notation for them. When I say 'generate,' that's pretty literal. My plan is to computer generate graphics notation in real time, in response to computer listening to what the choir is doing, so as to create a feedback loop of sorts.

Obviously, there are a lot of parts for this. I have some vague ideas of how to do the listening (SCMIR and some other feature extraction with SuperCollider). But as this is a feedback loop, there's also a question of how people will tend to respond to various notation, so I'm starting with the notation aspect first.

There have been some very interesting live notation projects I've seen. They often use LilyPond as the rendering engine. There are some SuperCollider libraries that interface with this system. However, the Constructivists are more about graphic notation, so unless lilypond lets you disassemble notation symbols like Cardew's Treatise, it's not quite what I need. I really need a vector graphics library that I can communicate with via OSC.

Practising and Performing

The LilyPond stuff works really well because the notation is already very widely understood. There are a lot of ways a performer can interpret a crotchet on B-flat, but the pitch and duration are already set, at least, Other dynamics markings, phrase markings, etc give a lot of data to a performer. In essence, the perofrmer has thousands of hours spent already learning the notational symbols. Many or most of these elements may be absent or modified in graphical notation. Therefore, a performer of a graphical score must learn the notations they learn the piece. If the notation is changing in real time, this sets up a serious issue as to how they should practice.

There's a chasing-one's-tail problem with feedback loops, so for now some set process will generate the notation. Performers need to be able to see the notation while they practice and perform. This implies that either I'll need to create some static printed sheets, which are in danger of beocming a fixed score, or else I need to be able to send notation to the laptops, tablets or other devices that they can then take home with them and look at.

I was considering using Open Frameworks or Cinder, but there are some issues that prevent this. These issues are political, not technical. And they have to do with Apple, who have deicded that free software is not welcome on their platforms. A tablet is exactly the form factor that's needed, but as the real owner of all iPads is Apple computer, who sees fit to dictate what consumers can do with something they spend a small fortune on, I'm out of luck unless I want to write multiple versions in multiple languages and pay a fee to Apple for the privilege of wasting my time. So I'm going to have to do it in javascript.

If anybody is wondering why javascript is so popular right now, the answer is because Apple are wankers.

Right, ok, I hate javascript and I hate browser plaforms but sometimes art forces one to gaze down into muck so as to better appreciate the stars overhead. Anyway. So there's a version of Processing for javascript, which I'm leaning towards because I've got some processing experience and there is a lot of documentation. Alas, somehow processing in the past has had the power to make even single rendered lines look ugly. But you know, sometimes you need to trade off things like aesthetics and performance and joy in life to make some Silicon Valley billionaires happy.

If I want to try something new, there are actually quite a few javascript vector graphics libraries on offer. One seemingly popular one is called Raphaë1 and then there is one with fun demos Bonsai and then one designed for 'creative coding' called Sketch.js, (which works with Coffee Script, which seems to be less painful than normal java script) and then there's the very old school looking JSGL. I have no idea which one to use.

I'm hoping somebody gives me some suggestions, but barring that, Processing has a lot of documentation and a large user base. Also, since this is a piece for performers, I'd like it to have a longer self-life than I would expect out of a purely digital piece. If people are going to go to the trouble of learning it, they shouldn't be forced to abandon it before they're ready.

So my next task is to dig out my art supplies and start sketching out ideas. What might a snapshot of the notation look like? What are elements on the page and how might they transform?