Innovation

IMUTUS aims at the development of a interactive open platform to assist practising on a typical traditional non-MIDI instrument as the recorder. It introduces innovative features both at the pedagogical and at the technological levels. Pedagogical and technological innovation are closely linked since new technological tools will allow for new teaching practices, while new pedagogical demands may call for new technological R&D.

The extension of interactive music tuition systems to the traditional instruments is a quite innovative element in the state of the art music tuition research area.

IMUTUS intends to extend the interface between musical instruments and computer software to traditional instruments with no MIDI output. This operation is not trivial for instruments such as the recorder since the single notes cannot be mechanically tracked like on a keyboard. An audio recognition module is required to convert the musical sound sequence into MIDI format. The recorder has been selected as a case study. The integration of audio-to-MIDI technology will form the basis for an interactive music tuition environment for the recorder, and, in the future, for other traditional instruments as well.

The student’s performance will be assessed objectively by a specially developed objective performance evaluation module, based on a comparison between the extracted MIDI information and a target performance compiled from the original score. The module will offer feedback suitable to the current skill level of the student, presenting it as easy-to-understand feedback messages referring to high-lighted notes or parts in the original score.

The possibilities of formulating an adequate list of basic skills (technical and musical) that are involved in music performance will be examined. Using such list of skills, both as a basis for the assessment of the student level, as well as for the characterization of the melodies and exercises, would allow the system to suggest follow-up exercises and provide more personalized learning paths.

Additionally, the student will have the capability to listen to changes in pitch or note values made in the original score by means of a graphical editor. This basic editor provides the means to edit the score of a piece, to convert it into MIDI format, and to listen to the transformed score. In this way, the student can obtain a immediate realization of variety of musical elements through direct interaction with the system.

The conversion from audio to MIDI is a music recognition problem, which, until now, has not had a satisfactory solution, in case of complex scores. Programs for speech recognition already achieve an accuracy of 80-90 %, whereas the existing systems of music recognition work with an accuracy of 70-80 % for single voice melodies. For polyphonic music the accuracy is even lower. The variety of musical timbres, harmonic combinations and transitions is so rich that music recognition of the musical score of a symphonic orchestra is a problem whose solution still seems distant. Thus, the project opens the road for a longer-term research with contributions to next generation e-learning systems and services.

Audio recognition and optical recognition algorithms will be improved resulting in higher recognition rates in more complex acoustic performances or printed scores. The incorporation of high quality recognition techniques leads to an innovative integrated system facing the total situation of interactive music tuition using either acoustic or written input, in case MIDI output is not available.

Another element that increases the up-to-date functionality of the system is its Internet-based communicative character. The musical courses database will be downloaded through the Internet and will be updated based on the comments and the results of the users, enhancing the dynamic and interactive nature of the system. Additionally, the incorporation of a distance-learning facility is harmonized with the pursued electronic orientation of the learning systems (e-learning).

Audio to MIDI conversion modules have already been developed internationally, but have not been combined with user performance evaluation tools, virtual reality modules, optical music recognition, MIDI editing, teacher-user transactions and organized music exercises. The combination of all these techniques adds an innovative character to the whole system.

The problem of optical music recognition has not been completely solved up to now. Commercial products are frequently non capable of recovering the complete music score and are very complex to be used. In the project, these two aspects will be considered to go over the state of the art and to integrate the OMR module into the tool for courseware fruition and production.

The fingering technique is one of the most important aspects in learning and practicing with a musical instrument. Thus, the educational value of appropriately representing the movement of the fingers and the positioning of the instrument between the lips is (embouchure) evident. The selection of virtual reality (in the form of a 3D representation technology such as VRML) presents a superior solution over simple video, since it will enable the pupil to freely navigate within the scenery, observing the movement of the virtual player from different viewpoints, zooming in and out, and emphasizing on aspects that they consider especially difficult (e.g. the position of two fingers to each other). Moreover, the use of a high level coding for the finger positions and movements and a music-notation-to-hand-position mapping, will make it possible to dynamically render in 3D the fingering of any score or passage, which would be impossible through the use of simple video. The introduction of virtual reality will also allow experimentation with new pedagogical mechanisms for teaching music and more specifically for showing correct hand and finger positioning.

It should be noted that presently there exist only few solutions for the distance teaching/learning of music. In most cases the music received from the pupils is not interactive. These courses are typically only built by using standard authoring system and the user interface does not present 3D virtual representation of hands. In this view a strong innovation will be produced.

Thus, providing such an efficient and innovative environment for music practice can significantly improve the quality of computer-based music tuition, balance the position of traditional instruments over electronic keyboards, and allow for the definition and experimentation with new music teaching practices enriching the traditional procedures and involving advanced technological tools and distance learning.