Skip to content
December 19, 2012 / raffazizzi

The Score as a Controller: The Orchestra for iPad

Since the debut of the Apple iPad, app-maker TouchPress has been leading the reinvention of digital publishing. Most of their titles start from texts such as a novel or a collection of poems and enhance them with plentiful multimedia content to produce elegantly crafted hypertextual publications.

TouchPress’ latest app The Orchestra is an interactive tour of the orchestra and orchestral music; an ambitious project that required TouchPress to engage with two worlds of the classical music industry. They partnered with the Philarmonia Orchestra for recording audio and video, and with The Music Sales Group to engrave the digital scores.

The main section of the app presents excerpts of eight orchestral pieces, with three HD videos coordinated with the score. One camera is always on Esa-Pekka Salonen, the conductor. His composed conducting style works well with this frontal camera, which gives the user/listener an unusual point of view and insight on the conductor’s role. Other videos are also edited to showcase the modus operandi of the orchestra by focusing on the dominant instruments, or by showing the whole group from the top.

Main view of The Orchestra app

The main view in the app

A special diagram show each instrument in the orchestra as a dot that pulses when the instrument plays. Each section of instruments (and of dots) can be made louder by holding a finger on the corresponding area of the diagram, effectively giving an idea of how the sound would change by physically moving through the orchestra (for most pieces this is only available after an in-app purchase, forced by the 2GB size limit that Apple sets for each app).

The score is displayed at the bottom and is coordinated with the audio. A vertical bar marks the notes being played and the score scrolls together with the performance with good precision (things get occasionally a bit less precise in Salonen’s own Violin Concerto, which combines frequent tempo changes with the relative autonomy of the solo violin). The score acts as a control system for the user/listener to hold, scroll back or forward the whole apparatus.

Salonen describing the orchestra diagram at the launch at the Apple store in London (14 December 2012)

Salonen describing the orchestra diagram at the product launch at the Apple store in London (14 December 2012)

The size of the iPad imposes an important limit to the amount of score and video that can be displayed at the same time. To address this, instead of the full score, one can view a “curated” score that only shows the important instruments at every given time (not necessarily all those playing). This is a clever mechanism that immediately visualizes changes in the orchestra’s texture, but instruments (or rather, their staves) jumping in and out of view make it uncomfortable to actually follow the score. A special sequencer-like view can replace the score. This, I believe, is a great aid for the non initiated: it immediately shows the complexity of the music.

Audio commentaries (or their transcriptions) can be switched on to accompany the user/listener through the piece. Salonen’s commentaries will occasionally refer to a piece’s history; performers’ comments tend to stick to the practicalities of performances, often quickly moving away from the context piece being heard.

Extra educational content is quickly reachable from the home screen or from the score. There is a profile for each instrument, with a photographic 3D-model (a signature feature of TouchPress apps), short video explanations by the performers and an interactive piano to sample the instrument range and timbre. There are also introductions to each piece, also targeted to the interested, but untrained reader (an amusing yet communicative example is in the commentary on Beethoven’s fifth, where the famous opening is referred to as “the ta-ta-ta-TAH“).

The relative precision of alignment between the different media reveals a closely curated publication, where human-produced metadata is key (for example by recording someone’s “tap” to get tempo information). The aim here is to publish fewer minutes of music, but at the highest possible quality; everything is edited so automatic cross-media alignment, in brief, is probably out of the picture (or only used as support).

The scores, however, are perhaps the least curated components of the app. They are supposed to be the same as those used by the orchestra in preparing the performance and it is even possible to spot the odd editorial intervention (use of square brackets and dotted lines, mostly), however there is no explanation of what the marks mean or why they are there. Clearly, the focus of this app is on the orchestra as a coherent music-making group, while it is not a means of discovering the musical works themselves.

This shows a departure from other TouchPress titles based on literary works, for which an exemplary publication is The Waste Land. The app publishes T. S. Elliot’s work together with annotations, images of the manuscripts, video interviews to experts, and a full performance by Fiona Shaw. While in The Orchestra the works and their texts (the scores) act as nothing more than starting point, in The Waste Land the focus is strongly on the text.

Those scholars in the Digital Humanities interested in digital scholarly editions were quick at noticing the example set by The Waste LandMark McDayter wrote an extensive review on his blog and Elena Pierazzo reviewed it in a few talks. Both praise the innovation and point out the same issue: it is not a scholarly edition of the work; should it be, and how?

The costs for producing these beautiful publications also seem relatively high for academic-led projects; The Waste Land costed $120,000 in the making (source: BBC). Does this mean that scholarly digital editions are to remain a below-par product visually and functionally? Can software like Scalar help?

In the context of digital editions of music, TouchPress shows what can be done with today’s digital publishing platforms. Nonetheless, the strong focus on the orchestra itself does not work well for scholarly editions, which are by definition text-centric. This does not mean that the two approaches cannot be united: one can imagine The Orchestra enriched with a more interactive score with connections to the explanation of editorial marks and extra content about the “making of” the text. What is hard to image, though, is making the scores performable. Issues with the size of the hardware are already evident for listener-oriented fruition, and TouchPress (or Music Sales for them) has not even began dealing with individual parts.

The Orchestra is a unique piece of work and an example for the Digital Humanities community with interests in music editing and publishing. Many a project have attempted to show the communicative power of multi-modal alignment, for which good examples are the Probado Music Document Interface (see also this paper) and the experiments with aligning score, performance and scholarly commentary at Purcell+ (alas, the demo is not publicly available). Nonetheless, The Orchestra‘s carefully designed user interface and curated content really shows the potential of this medium. A next essential step is achieving this quality in delivery for scholarly projects and performer-oriented editions (though the iPad might not be the best device, for more details have a look at one of my previous posts).

June 14, 2012 / raffazizzi

Open Golberg Variations – new directions in music publishing

The Open Goldberg Variations, a project crowdfunded on Kickstarter, has recently released a new score and recording of J.S. Bach’s Goldberg Variations BWV 988 directly into the public domain. This means that both the recording and the score can be accessed, played, remixed and used in any possible way. This commendable effort has had considerable resonance throughout social media, blogs and has been covered by Wired UK magazine.

The recordings can be downloaded from the project’s site in several formats, including a high quality FLAC version. The score is available as PDF at IMSLP or one can access a digital edition on the platform. They also prepared a free iPad app that marries the recording with the score.

The process of production of the edition and its digital publication are particularly interesting. They are certainly pioneering and they have succeeded in bringing issues of digital publication of music to the attention of a more general public. This said, in this post I will deconstruct some aspects of the project to see how much of this can be relevant to an academic context.

There are already editions and recordings of the Variations in the public domain, but the project is the first of its kind to produce a new rendition to go straight into the public domain. This opens up countless opportunities for anyone to create new content, new art, new editions and whatnot based on this work. This is great, though the “setting Bach free” rhetoric that has pervaded the project since its early stages needs to be taken carefully. Someone else already pointed out that it’s not the Variations that have been set free, but a specific performance of it; likewise, the score is one specific edition of the text.

The score is not meant to be a critical edition, so there isn’t any information about which sources have been used, what has been corrected, normalized or modernized, etc. The editorial work is attributed to Werner Schweer and the text went through two public “peer-review” phases. This was done through the platform, which allows users to leave annotations on published scores (sort of like a Flickr for sheet music). Comments were left on an already drafted edition of the text, so practically all of them were concerned with layout and legibility of the notation, not with the accuracy of the editorial work (the comments can be seen here). The issue of variance across sources has occasionally come up, but was dismissed with the principle that anyone is free to create different versions:

Conversation about sources on

Nonetheless, it is important to remember that the now finalized official Open Goldberg Variations version is more likely to be distributed that any derivative edition.

To typeset the score, the editor has used the open source desktop application MuseScore, of which he is the lead developer. This is a very good piece of software (I use it regularly), though the editor/developer (an interesting position to be in, more on this some other time) was required to greatly improve it to cope with the complex notation of the Variations that includes abundant cross-staff beaming and non-standard baroque ornaments. As a result, MuseScore is probably one of the few free score editors that is able to deal natively with schleiferdoppelt-cadence, etc.

This makes me think of an old topic in the debate around scholarly digital editions: where are the tools for producing them? Many have argued, and I second them, that each historical text will have specific requirements that are likely to make a generic tool feel like a constraint rather than a facilitation. The editor/developer of the Open Variations must have realized that each repertoire and even each single piece, will have specific requirements that reach the limits of the tool.

iPad app screenshot

In the iPad app, the score follows the recording almost note-by-note

The score has been produced with a digital-centric workflow, so that both printed and digital publications of the work are possible. The digital score has substantial knowledge built-in, which allowed developers to come up with a few interesting things for its publication on a digital medium (website, iPad app, etc.). Such a score is able to “play” itself, but more interestingly it allows a certain fluidity of layout; in the iPad app, for example, one can seamlessly switch from portrait to landscape orientation for better readability. The score also “knows” the pitch and duration of each note and the app has been programmed to highlight each note when it’s played in the recording (read more about the technology behind this here).

But what makes this stand out from a printed publication? The iPad app in particular is more usefully targeted to listeners than performers: the alignment with a pre-existing performance is probably of little relevance to a performer and the page-turning system is awkward.

Finally, who are the readers that need note highlighting when reading and listening? Those trained to read music notation will not need it. The introductory text to the app often mentions the “fans” of the Variations, which can easily be both musically trained or not, so the highlighting may have some pedagogical use. Mostly, though, the highlighting shows what a “smart” score can do (or an e-score, or something else, a name for this new thing hasn’t really been established yet!).

Could there be more, though? I would say yes; the Variations’ app still presents a very static, print-like text, with a wow-inducing feature. A publication targeted to listeners (such as this one) could be a lot more informative, give access the editorial work behind the text, historical notes, show suggested (even better contested) resolutions of ornaments, etc. If it were to be targeted also to performers, the situation would get more complicated.

The Open Goldberg Variation project shows that the technology for digital editions of scores exists. The interest that the project generated (be it for its Open Source ideas, the open peer-review process, or the quality of the recordings) shows that there is also an interested audience. Is there interest in academia to take a step into this innovative market?

What do you think? Is this the direction for digital music publishing both scholarly and not? Would you use this edition for your work or performance? 

March 28, 2012 / raffazizzi

Day of DH

Yesterday (27 March 2012) many scholars, students and professionals that define themselves as Digital Humanists have been blogging as part of the “Day of DH” initative.

Every participant blogged about their day, their activities and anything worth sharing. Many participants also contributed their own definition of the field, something for which there is still much disagreement.

I have also joined and tried to post a few reflections with each update. Here are all the posts that I wrote:

  1. Getting started; a short introduction and agenda for the day.
  2. Teaching Programming in DH; in-class reflections about teaching Python to MA Digital Humanities students.
  3. Jane Austen (a TEI project); after-meeting reflections.
  4. IOSPE, Solr and Kiln; description of project work.
  5. Enhanced Scholarly Publications… and Wrap-Up; short notes from an evening seminar.

Stopping and reflecting on my day activities was a good exercise and look forward to do it again next year. The Day of DH site will archive the posts and publish statistics about the blogs in a few weeks.

March 1, 2012 / raffazizzi

Ronald Broude’s new article on musical works, texts, editions

Ronald Broude’s article “Musical Works, Musical Texts, and Musical Editions” in Scholarly Editing (vol. 33, 2012) came to me as a surprise, almost a breath of fresh air. It entertains some key reflections on the role of music editions and their digital future. The entire issue of Scholarly Editing is, in fact, quite good and freely available on the Web under a Creative Commons A-NC-SA 3.0 License.

Broude makes an important distinction between “historicizing” editions and “enabling” editions. The first are musicological and critical; the latter “mediate between a piece of music and a specific community of performers” such as “amateurs”, “proficient recreational musicians” and “professionals” (pp. 4-5). Enabling editions are the most common because the largest share of the market is not the professional musicians and the main selling point are exactly the “performing suggestions provided by [the] editors” (p. 5).  In appendix, Broude provides a few “exhibits” of these different approaches to support his claims.

The tension between historicizing and enabling editions is ideological. Broude thinks that this tension can be reduced to how a creative work is conceptualized: is it a “product” or “process”? Seeing it as a product is typical of the work-concept that dominated musicology from the nineteenth century to modernism and beyond (for which see Lydia Goehr’s seminal book). Within this framework, even if the composer revises the work in response to specific circumstances, it is still the composer only that has the authority and right to create different versions. “[A]lterations effected by performers, for example―can claim neither authority nor the legitimacy that authority is understood to confer” (p. 7) (recent research in performance practice and advocacy is perhaps the currently strongest challenger of this idea). Editions produced within this framework typically cannot escape pursuing the presentation of a text that reflects the composers’ intentions.

By seeing the work as a process, the role of the composer becomes less central. The reception of the work becomes important too, and as performance practice and audiences’ taste evolve, so does the work. Taken to an extreme, “any number of texts is possible, since each state [of the work] can be reflected in a text”. Broude still warns that enabling editions do not necessary fall under this latter framework, mainly because their editors to do not choose to follow a “philosophical rationale for their activity” (p. 8).

"It makes a difference when musicians play from well-used material"

He’s not the first, but Broude reminds us that “[m]usical works are performing works, and each time a musical work is performed, it must be adapted to the circumstances of a specific performance” (p. 10). He seeks a middle ground, though, by claiming that a lot of these adaptations more often than not are notated as text or, in his words, “performed onto paper” (p. 11). These texts have been put together in the past, and still are today. They often are the only witnesses to performance practice before the invention of the phonograph, but today written notation is not the only form of “text” and Broude seems to voluntarily underplay the role of recorded music in the shaping of audiences’ expectations and performance practice. He chooses instead to present the example of Bernard Haitink  wanting to play from “well-used material” (p. 12).

Broude continues by clearly stating that the historicizing edition can “produce a misleading view of musical practice and musical text” by emphasizing the role of the composer’s text. These editions may even not reflect the work’s identity at all as they “ignore the interaction of text and performance”. Textual theory, then, ought to take performance into account or it “cannot accurately describe the dynamics of musical texts” (p. 13).

Broude eventually starts to envisage a type of edition that is able to account for more than one point of view and span the two traditions. He’s referring to digital editions, which will be able to “make the works they represent available in a variety of ways”. It is a bit disheartening to read: “[s]uch editions are not here yet―I write in 2011―but sooner or later they will be” (p. 14). Such words are remarkably similar to James Grier’s short chapter on electronic editions from his 1996 book The Critical Editing of Music: “programs with [such] capabilities are under development for literature and I am sure that the situation in music will soon be rectified” (p. 177). It really is true that digital scores have not improved much in the past decade and more, mainly because the score is conceived and perceived as destined to be printed (perhaps I’ll elaborate more on this in another post and make a distinction between digital distribution and digital consumption).

Broude imagines digital editions that benefit the text scholar by being capable of generating collations and looking for (dis)agreement amongst sources. He also envisages digital editions that performers can use to second guess editorial decisions” by being able to survey “several texts of a work they are preparing to perform”. Eventually, he concludes, “digital editions will lead to a new idea of musical works”, one that is multiform and fluid  (p. 14).

Although I can only be excited by reading such support for digital editions by an established scholar and editor, I can also see Broude portraying the digital editions as some sort of holy grail, not unlikely what the literary community was doing in the mid-to-late 90s.

The issues behind automatic collation, for example, are numerous and they are both technical and monetary; data entry is expensive in time and money. When it comes to second guessing editorial decisions, or choosing performing marks from one historical document or the other, it is hard to tell whether performers would really make use of such a kind of feature. Broude claims that performers already do this sort of work when preparing a piece (p. 15), but is that true across the spectrum of repertoires and skill sets?

At least tentative answers to these and other important questions must be produced before being able to make claims on changing the idea of musical works. A few more: how should a digital edition be modelled computationally? What changes in method(ology) would such model impose on editors? Should digital editions be distributed and sold like a printed score? Should they grow and improve when new sources are found or should the process start over with new editors? Etc. Assuming that at some point there will be “tools” is not enough.

Nonetheless, I can’t help but agreeing with Broude’s claim that “the impulse to apply [digital editing (and publishing) technology] will be irresistible“. The pervasiveness of technology (particularly mobile) makes these times much more fertile for digital editions of music than 1996. The road is still long and bumpy, though.

November 28, 2011 / raffazizzi

Chatting with pros at CLoSer and some thoughts on digital editions and performance

The City of London Sinfonia has come up with a clever way of reaching out to new audiences. The CLoSer concert series started last Tuesday (22 November 2011) at Village Underground, in the heart of Shoreditch. The setting and the atmosphere were intentionally informal, with a glass of wine included in the price of the ticket, no real stage, pillows on the floor, and a bar open through the concert.

We were welcomed by some Philip Glass in the background while we grabbed our drink and waited for the orchestra to appear, perhaps a good appetizer to the opening piece, Shaker Loops by John Adams. Between each piece, the conductor/clarinettist Michael Collins would pause to chat about the next piece and his experience as performer. For example, he mention the “Bartók pizzicato” and got the violas to show us how it’s done before the concluding piece: Bartók’s beautiful divertimento for string orchestra.

Village Underground: atypical, exciting venue (source:

The informal setting was great fun, there should be more initiatives like this one. Notwithstanding the wine and the pillows, the audience still behaved “as expected”, listening in silence, saving claps for the right moments and waiting for the after-concert to visit the bar again. Perhaps this is because of our well rooted social expectations when it comes to “art” music, which are hard to shake off.

What I enjoyed the most was the after-concert. The players stayed for a drink, mixed with the audience, answered questions and talked about their instruments. I took the opportunity to ask a few questions to Michael Collins about the use of critical editions in the preparation of performances.

Perhaps unsurprisingly, it is becoming more evident to me that when performers use critical editions, they look for a clear reading text. When asked if conductors question editorial decisions, Collins said yes, often according to a performer’s notion of style. Diversion from the text, then, would mainly affect how something would be played (in terms of technique, colour, phrasing…), rather then involve the historical variance found in the apparatus, which is more likely to stay in the apparatus.

Recently, I also spoke with a professional music engraver who has contributed to the production of several critical editions of classical and romantic music. He had a similar opinion and explained how the short time that professionals often have to prepare performances fights against a more central role for the information in the apparatus. Moreover, rental scores and parts from critical editions typically exclude appendices and favour usability instead.

I’d add the oft-mention fact that the limits of the printed page make it hard to bring extra information to the foreground. Digital editions of music might have a better chance at it through, for example, interactive and flexible scores. Like James Grier said in his 1996 book The Critical Editing of Music, digital editions may have a role in rehearsals and performances (pp. 177-9), but interestingly, he suggests that the software should not allow to delete or alter any portion of the original text. Paraphrasing, it seems that editors have an ethical responsibility in making an accurate choice amongst variants (and reconstruction, and so on depending on repertoire, contingencies, etc.).

Needless to say, editions like those imagined by Grier do not really exist yet. However, since we’re talking in hypothetical terms already, what if digital editions also made historical variants and commentary readily available at the touch of the performers’ fingers?  Would that make a difference during rehearsals? Or would other practical matters of getting things done still push this information in the background?

November 9, 2011 / raffazizzi

Google Ngrams, Opera and Paperbacks

The Google Ngram Viewer is old news by now, and several have blogged about it. A few blogs were quick to identify the main issue around its reliability: OCR errors. This famous post about the medial (or long) s is particular amusing. Nonetheless, I’d like to go back on the topic after a discussion I had with my supervisor some time ago.

In brief, the Google Ngram Viewer allows one to plot occurrences of a search query on the vast collection of books that Google scanned and OCR’d for Google Books. To have a better idea of what it does, have a look at this Tumblr that collects interesting Ngrams, or try it out for yourself.

While looking at the Viewer with my supervisor, we tried to look for a few composers, particularly opera composers. We started by confirming our most obvious assumptions, like Beethoven being utterly dominant (in most corpora, and surely for a lot more than just Fidelio), closely followed by Purcell (in the English corpus) and Mozart. What baffled us for a bit was the fact that most of the composers seemed to experience a big drop around the 1950s and 1960s.

Once at home, I looked up in the English corpus some notorious opera composers between the 17th and early 20th century and confirmed this trend (N.B. this selection is not at all representative of the full landscape of opera composers, just first names that came to mind).

Google Ngram for some notorious opera composers

Google Ngram for some notorious opera composers

Now, what made us perplexed was the fact that academic and non-academic literature about composers probably grew after the 1950s rather than diminish. But in fact, the Ngram Viewer normalizes the occurrences by the number of books published every year.  This is why on the Y axis we see percentages rather than number of occurrences. So why is the drop there?

After some head scratching, we got to the approximate conclusion that starting in the 1950s, book publishing begins targeting a larger and growing audience, resulting in a wider range of topics and genres. Notwithstanding the normalization, composers get submerged -in percentage of total published words per year- by many other printed words.

My knowledge of history of book publishing is very limited, but after doing some digging around I stumbled upon A history of British Publishing by John Feather that seems to confirm this idea. Specifically, the increase of adult literacy determined the success of paperback editions that “made [their] way into the British publishing scene in the 1950s…” and were ubiquitous by the 1970s. Assuming that a similar phenomenon took place in other English-speaking countries around the same years, we can tentatively take this as a plausible explanation for the drop.

It goes without saying that because of the noise caused by OCR, one should look with some suspicion to any information inferred from the Google Ngram Viewer. However, because of the corpus size, it is a unique tool and it shows the potential -and risks- of using numeric “evidence” to formulate and discuss theories about text-based culture.

I also find somewhat amusing that Google, by means of relevance rather than complexity, finally brought some computational linguistics on Science Magazine.

Trivia: apparently, the Ngram Viewer, or some components of it, is written in Python, as I could see from one error message that I got during one of my tests😉

Why do you think the drop may be there? Leave your thoughts in the comments!

November 4, 2011 / raffazizzi

It is not sound – post 0

Stockhausen - Helikopter-Streichquartett

Stockhausen - Helikopter-Streichquartett (1994)

Hello World. I’m Raffaele Viglianti, a PhD student in Digital Humanities at King’s College London. My blog will contain some more or less structured thoughts and experiences from my work at the Department of Digital Humanities and on my PhD project.

My research focuses mainly on digital methodologies for musicology, but my interests lie in the written expression of music; particularly the phenomenology of writing and reading/performing a music score. So, in my posts I’ll rarely refer to sound and more often to written notation.

As part of my PhD, I am currently working with the WeGA on a digital edition of Carl Maria von Weber’s Der Freischütz. I will occasionally be posting about oddities that I find in the sources, and how I’m dealing with them using the Music Encoding Initiative and the Text Encoding Initiative formats.

This blog will not just be about music, however. Working as a developer and teacher at the department means that I get involved in Digital Humanities projects that span across several disciplines (I particularly enjoyed working on Jane Austen’s Fiction Manuscripts and a few projects involving Epidoc), so I will also post about other DH topics, methods, technologies, teaching

Thanks for reading! And if you were wondering how the score in the image is played, here’s an example.