Selecting input melodies

Moving right along, the next block in the configuration file is:

tunes:

  - query:
      conditions:
        solo_info:
          performer: 'Miles%'
          title: '~Impressions%'
      display:
        transcription_info: filename_sv
      type: sv

This block is named tunes and is responsible for defining the set of solos you want melfeature to work with. The block is hierarchical and must contains a list of definition statements. Lists in YAML are indicated by hyphens - (followed by a blank!). This lists here contains only one element, a so-called query block. But in general, at this point an arbitrarily long list of item could follow. melfeature will work on them consecutively, retrieving all melodies that are defined by the items, adding them to a total input set.

Basically, there are two different modes for retrieving input set of melodies: file mode and database mode. This modes are exclusive, you have to decide for one mode or the other. In the current example, melfeature is deemed to operate in database mode. Oddly, there is only one single parameter which decides which mode is used, and it is somewhere else in the YAML file. Let’s have a look at the very last lines:

database:
  type: sqlite3
  path: wjazzd.db
  password: None
  use: True
  content-type: sv
  version: 2

This is the database section, which defines important parameter values for database usage. The type field tells melfeature that the database is in SQLITE3 format. As it happens, it is the only type that is currently supported, but others are to follow soon. The path field is the name of the database, which is for SQLITE3 databases just a single file. For other database formats this will be most likely an URL, but I am digressing. Moreover, SQLITE3 does not support password protection of databases so the password field is set to None, which means “not used”, and actually the line could be missing, it is just there to show off. Finally, the use field is set to True (remember the boolean values of Python). This and only this determines that melfeature is operating in database mode. The only other valid value is False, which would indicate file mode. Since version 1.4. of the MeloSpySuite (and v1.2 of the MeloSpySuite), we use a refined data model into out data base, hence we have to tell melfeature that it will read from a database in the new data format, hence version: 2. Since this only applies to the Weimar Jazz Database but not the EsAC database, we also have to provide a content-type for the new database, which is sv. For using the EsAC database, the two fields content-tye and version are not needed and can be missing.

So, back to our tunes block from above. The list in the block consists of only a single element, which happens to be query block, which is the only reasonable choice in database mode. In file mode blocks with the keyword file instead of query are used. The query block itself contains three sub-fields: conditions, display and type. Let’s start with the last one. Since MeloSpyLib is designed not to work exclusively with jazz solos but with all kinds of monophonic melodies, we support different types of melodies. Though these have all the same core component, i.e., a list of tone events, but they can and will differ in the availabe metadata and the corresponding formats. There are currently two possible types: sv (from Sonic Visualiser) and esac. The jazz solos in the Weimar Jazz Database are in sv format, since they are originally produced using Soniv Visualiser, whereas as the folk songs from the Essen Folk Song Database, which is included in the MeloSpySuite as well, are in esac format – as you might have guessed. Depending on the type of melodies, the values in the two other sub-fields will differ.

The most important sub-field is conditions. The idea is to give conditions on the maximal set of all available melodies working as a filter. The conditions-block itself consists of a single block here, but could also be a list of conditions (which have to be all fulfilled at the same time). The names of the sub-field in the list of conditions must correspond to table names in the database. “How do I know these table names?”, you might ask know, and “what the heck are tables and databases anyway?”. These questions can be partly answered by having a look at the database documentation. But the ones used here are probably the most important ones for many cases of application.

SQL and relational databases

SQL databases are one the most common type of databases around. SQL means “Structured Query Language” and refers on one hand to a special kind of programming language and the other hand to this specific kind of databases, basically also known as relational databases. Both terms are not completely synomymic but go very often hand in hand. There are many different SQL databases out there in the wild, i.e., MySQL, PostgreSQL, Oracle, SQLITE3 and many others. The main idea behind relational databases is to store all data in tables, where data bits belonging together are connected by unique identifiers, the keys, between different tables. For instance, in the Weimar Jazz Database there is a table called melody, where all the tone events of all melodies are stored. Each event occupies one row with several columns in this table, whereby each event is tagged with a certain key, the melody ID. This melody ID is also used in various other tables in the database, e.g., solo_info, where under the same melody ID the title and the performer of the solo can be found. There is of course a lot more to it. You might click here if you really want to know more about relational databases and SQL.

So what does this query conditions define? The key is solo_info, the table of the same name in the database contains for each solo the fields performer and title amongst others, which refer to the soloist and the name of the tune the solo is contained in. (See Table: solo_info (WJD) for the other fields.) Using this key, tells melfeature to interpret the subfields to be part of the solo_info table. Let’s have a look at the two fields:

solo_info:
  performer: 'Miles%'
  title: '~Impressions%'

The value of the performer field is 'Miles%'. The single quotes are not necessary, but they do not disturb either. However, the percent sign could be replaced by an asterisk * giving the same results. But the asterisk does have a special meaning in the YAML syntax, which would make the quoting necessary. The percentage sign is a wildcard. It is a stand-in for any sequence of characters. Hence, the conditions here translates to: “Select every solo where the performer can be any string that starts with ‘Miles’ followed by arbitrary characters ” – which is true for only for “Miles Davis”. If you would use only a single “%”, it would match all performers, since this selects arbitrary strings, a condition which is not really a hard-boiled condition, you might call it an empty condition. So far so good, but the next line actually is part of the condition too and defines a certain exception to the first line. The field title refers also the database table solo_info and has the value '~Impressions%'. The first character ~ (tilde) has the special meaning of negation in this context. The rest translates to “Select all strings that start with ‘Impression’ followed by an arbitrary set of one or more characters”. The negation reverses this meaning to something like “Select all solos with titles not starting with ‘Impressions’”. As it happens, there is currently only one solo in the WJazzD with the title “Impressions”, a very long, 14 min solo by John Coltrane. Since we already selected for solos from Miles Davis, this conditions is actually doing nothing to your result set. If you would have used the single percent sign for the first “performer” condition, it would rule out Coltrane’s solo on “Impressions”, which might be a reasonable thing to do sometimes.

Note

Quoting can be done with single or double quotes and essentially signifies the YAML reader to ignore the very special meaning of certain characters, such as the colon :, which normally is the block indicator. I.e., if the value of a field is a string, let’s say, a text reading “I said: Hello, baby”, the string has to be quoted. Otherwise the YAML parser would think there is a field named “I said” with the value “Hello, baby”, which is nice in its own right, but not exactly what was intended.

The next field:

display:
  transcription_info: filename_sv

defines, which ID melfeature shall use in its result file. This field must contain a single mini-block, where the key must be a valid database table name, and the value must reference a field in that table. Here the table transcription_info is selected, which contains the field filename_sv, which is the name of the original Sonic Visualiser project file, in which the solo was encoded in. It is reasonable to identify each solo with an unique ID. Alternative solutions such as:

display:
  solo_info: performer

or something like:

display:
  solo_info: title

might seem reasonable at first sight, but actually, there might be more than one solo from a single performer in the database and more than one solo with the same title as well, so this display name would probably not be unique. Another possible solution:

display:
  transcription_info: melid

would be unique by using the unique numerical melody ID from the database, but this would not be very informative for later uses. So the solution presented is a good solution, since the names of the Sonic Visualiser files happen to have performer and title information included, possibly along with some extra specifiers to make identification unique (e.g, if a performer has more than one solo on the same tune, but from different recordings). For all other options, please refer to Database format.

Next part: Feature selection.