Skip to main content ↑ ↓
Topic: Envelope.hmm (1.0) - supporting MIDI envelopes in NWC (Read 10817 times) previous topic - next topic

Envelope.hmm (1.0) - supporting MIDI envelopes in NWC

Introduction

I have a dream: That I can work with NWC as easily as with a DAW to create music that is fun to listen to.

In these times, this means - I believe - that one can control one or more virtual instruments from NWC, like simple sfz players, a Kontakt container with embedded Kontakt instruments or some standalone instruments. Besides the notes, such instruments need additional control to produce realistic sounds, which is accomplished by the following means:

a) velocity of note-on and note-off;
b) a few predefined commands like "pitch bend";
c) bank selection - I will not talk about this here.
d) 128 MIDI controllers, which can be used to send some arbitrary data to the MIDI channel (and hence the instrument). Some of these have a predefined semantics, like "volume" or "damper pedal", but many others are simply labelled "general purpose" or even "undefined" in the MIDI standard; and especially these are used more and more by virtual instruments to control "breath", "pan", "blend" and whatever terms the instrument designers use for various properties and effects. These are essential to capitalize on the more and more realistic sound production capabilities of such instruments.

Controllers can be used to control effects on a very detailed level, e.g. increasing and decreasing the volume of an instrument in a continuous fashion. This finegrained control requires an intuitive user interface, which is typically provided in the form of "envelopes".

Here is a very simple musical piece as shown in the Reaper DAW:

<Image Link>

It consists of a single note, which is presented as a horizontal line; the green rollercoaster below it is the "volume envelope": It controls how the volume changes over time. The important thing here is that 
  • it is very easy to understand the envelope: It is immediately clear where the volume goes up or down, or where it remains constant.
  • Moreover, it is also very easy to change the envelope: In Reaper, as an example, you just shift-click on it somewhere and pull the line segments up and down with the mouse - "direct manipulation".

This is all fine and well, but for those of us who like to arrange and compose via traditional scores, using a DAW for envelopes does not really work. The problem is that arranging and composing are looping processes: There will always be "some more changes" to the score, and all the location-dependent post-processing in the DAW would be lost if a new version comes down from the score software, i.e., NWC. Thus, it is necessary to integrate these additional instructions into the NWC score. Now, NWC does have some support in the form of "Multi-point controllers" (MPCs), but these objects are, in my opinion, too limited: Not all 128 potential MIDI controllers are supported, and usability of MPCs is quite a lot worse than DAW envelopes.

To overcome these shortcomings, I started to write a new object plugin that would provide as much as possible of the envelope functionality.

NWC envelopes

The concept is as follows:
  • At the beginning of the score, an "EnvelopeMaestro" object defines up to 10 value ranges with their own names (this is useful for showing "ppp"..."fff" for a volume controller - see examples) and some presentation parameters for a controller.
  • The value of each controller is set by "Envelope" objects along the score that define the current value and possibly a value step.
  • The value line is visibly shown, like with DAW envelopes.
  • The value and the step are manipulated directly: The value is derived from the staff position (which can be changed conveniently with ctrl-shift-up and ctrl-shift-down), and the span is controlled by the built-in spinner, i.e., the + and - keys.
  • As a special feature, a step can be "inclined" a little so that the value changes not to abruptly; this so-called "attack" can be changed directly
    with the , and . keys.

Here is a screenshot from NWC of a "single note" piece with varying volume:

<Image Link>

The important feature here is that it is very easy to see and modify the envelope - I repeat this because it is the whole point of the plugin: The following change was accomplished by just selecting the envelope object, pressing ctrl-shift-up about 6 times; and + and then . a few times:

<Image Link>

NWC's GUI immediately shows these changes, and they can be heard when playing the score next time.

Here are two pictures that show the properties of the EnvelopeMaestro and Envelope objects:

<Image Link>

<Image Link>

Here are some suggestions for using the Envelope object:
  • Put each controller on a separate staff with a single line. This allows muting of each staff, which disables the controller; and it allows for more graphical control. On the other hand, the "Staff range", "Line Type" and "Line Offset" options allow placing two or more controllers on a single staff to save "GUI real estate". Last, it is also possible to put the envelope objects into a note staff, e.g. by setting the "Staff range" high above the notes.
  • Use half and eighth rests on the controller staffs to allow for coarser and finer spans. Quarter rests take up a lot of space and are visually more disturbing. Hidden rests do not really work, as they huddle together at the left side of the bars and therefore distort the span slopes.
  • When copying envelopes to another staff, make sure that their Maestros have similar range definitions so that the effect rates do not change.
  • Don't use a volume controller with piano, harpsichord, guitar and other instruments where the volume cannot be arbitrarily changed after note started.

A user tool 'Shift' allows shifting a group of selected envelopes of the same controller. The first Envelope or EnvelopeMaestro in the selection defines the controller to be shifted.

H.M.

Re: Envelope.test (under development) - supporting envelopes in NWC

Reply #1 –
Greetings,

I have only looked over your code and comments briefly so far, but I have one suggestion that might help, regarding your use of {ntnidx}:objProp() to retrieve the properties of your Maestro object. You might try using {ntnidx}:userProp() instead, as this will be "filtered through" the spec table, and should thus have default values (and types) applied.

I will give it a closer look-at this weekend, but this is an interesting plugin you are undertaking - thanks for your efforts!

Mike

Re: Envelope.test (under development) - supporting envelopes in NWC

Reply #2 –
Thanks a lot for the pointer to :userProp() - I have replaced :objProp() with :userProp() everywhere and removed the clumsy ... or 0) + 0 code, and everything seems to work fine!
I look forward to any comments you have!

H.M.

Re: Envelope.test (0.3) - supporting envelopes in NWC

Reply #3 –
I hope this is not too shameless a plug if I get a little bit excited about this plugin: I started to try it out on a choral composition I wrote (and am still writing) - here is an experiment I did this morning with the first few bars, for the first soprano.
First, here is a score image of this voice  - I have added two volume envelopes, for experimentation (of course, when playing, only one of them is unmuted - the other is muted and does not send any MIDI commands):

<Image Link>

And here are three renditions as MP3s:


My setup, by the way, is:
  • (free) loopMIDI as virtual MIDI cable
  • Reaper 5.80 as VST container
  • (free) Kontakt Player 5.0 as instrument container for the choir; (free) sfz player for SFZ soundfonts
  • Soundiron Olympus Micro Choir for choir voices
  • (free) Sonatina Symphony Orchestra Keyboard sound for piano

And here are two renditions of all four voices (the score is in the attachment). The Olympus choir voices are not at all bad out of the box, but ...:


... don't you agree that that soprano crescendo on that last "stärkt" but one is much more "elating" (or "creepy"?) in the latter MP3?!

Of course, the same effects could also have done with a standard volume MPC - but I have to tell you I would not have had much fun playing around with that many controllers for an hour; whereas moving up and down those envelope objects is so quick ...

H.M.

Re: Envelope.test (0.3) - supporting envelopes in NWC

Reply #4 –
Quote from: hmmueller –
(5) [see code/please help] I would like to set the color of all envelope objects for one controller to the corresponding maestro's object color - is this possible?
Alas, it appears not. Currently, a user object can only draw in a single color. Attempting to set t.Color in an object's Create or Audit function gives an error about trying to change a read-only property. Perhaps in a future version of NWC, this parameter will be allowed to be writable in these functions, as is the Pos property.

One thing you might try is to create an embedded user tool for your Envelope object, and have it set the color based on the color of the corresponding EnvelopeMaestro object. This should work in theory because a user tool can modify any object property.

Re: Envelope.test (0.3) - supporting envelopes in NWC

Reply #5 –
Ok - then no implicit color transfer from maestro to players - so be it. Thanks!

I found that I will need some user tool(s) anyway - at least one for the purpose of shifting a whole range of controllers up or down: This does not work in NWC, as only the rests selected in between are shifted - and if there are different controllers on one staff; I would like to shift only one "selected" anyway ...

H.M.

Re: Envelope.test (0.4) - supporting envelopes in NWC

Reply #6 –
I have uploaded a version 0.4, with some small presentation improvements: There is a short line at the end of a slope which helps if the envelopes are shown in a smal scale; the text shortens itself somewhat if there is not enough space; and the text shows < or >, depending on whether the value increases or decreases.

However, I have spent most of my time on putting volumes into my composition. The user tool for shifting selected envelope ranges up and down would have come in handy, but my mind was not on programming ...
If someone is interested: I have attached the piece's NWC score here, and here are two links to a version with, and one without volume control (just muting the "Controller" staffs produces the latter):


There is one more reason why I think this plugin is useful: The various effects on virtual instruments require lots of experimentation until one understands what can (and cannot) be done with which knob/controller. Typically, one does first experiments via some GUI knobs - but eventually, one has to play around with the controls via some sort of "script", or "score". I believe (but have yet to learn and do that myself) that doing it directly in an NWC score is more intuitive than via some MIDI tools ...

H.M.

Re: Envelope.test (0.5) - supporting envelopes in NWC

Reply #7 –
Version 0.5 has a few improvements:
a) It is now possible to show the text below the envelope;
b) The locations where the lines begin/end; and where MIDI emitting begins/ends are now exactly at the start of the next note or rest - this is what one expects.

H.M.

Re: Envelope.test (0.4) - supporting envelopes in NWC

Reply #8 –
Here is something you couldn't do with NWC up to now (I think ...):  Changing the value of "Sound Controller 2 (Harmonic Content)" (CC 71).

Why would you (or I) want to do that?

Well, the "Soundiron Olympus Micro Choir" Kontakt virtual instrument allows the selection of two different "styles" for a voice, which can then be "blended together". Two generic styles are "oo" and "ah", and blending them with more oo or more ah yields a more or less "sharp" voice; on the whole, I find the "ah" quite ugly, but the "oo" is too soft, and mixing in some "ah" gives it more clearness. Here is a link to a picture of the GUI of this virtual instrument, with highlights around the style selections and the blend control. And according to the documentation, the blend control can be "automated" (i.e., controlled from a MIDI command stream) via CC71!

Now, I started to compose a gospel (based on Psalm 114), which begins with a long crescendo chord. After that, the basses sing a melody over S+A+T chords. The lyrics interlace so that there are more text lines ("stay in the faith ...", "stay with me", "pray with me"); and therefore the voices need to say their words with some emphasis. I wanted to create at least a little bit of that impression with the simple micro choir, and I found that for the long chord, I wanted more "oo" in the basses, but more "ah" later when they take the melody.

- I have attached the score of the first 7 measures below. If you want to see the envelopes, you have to enable the "Controllers" staffs in "Page Setup -> Contents".
- Here is a link to the resulting MP3 (yes, it ends quite abruptly).
- In this (silent) GIF you can see how the volume (near the upper right) and the blend controls move while playing.

This is also an example of two controller envelopes on a single staff - I could have placed them on two, but I wanted to try this out.
I hope this gives you an additional understanding of why I develop this plugin ...

H.M.
 

Re: Envelope.hmm (0.6) - supporting envelopes in NWC

Reply #9 –
Version 0.6 has now lost the ".test" status and is fully ".hmm". I have modified all the scores in the previous postings - I hope they do their work. Changes in 0.6 are:
  • Controller selection is now via a drop down list for predefined controllers; and, additionally, by number if one selects 'Select by number' in the dropdown. I think this is a good compromise between pure "magic numbers" and the necessary flexibility for accessing arbitrary CCs (like my 71).*
  • Text fonts are still fixed, but a scale can be used to make texts larger or smaller.
  • Both plugins got useful header comments.**

H.M.

* The source code, however, is even worse now, as I had to copy-paste the enum list into both plugins, plus two additional functions. Is there any way that two plugins can share some code (an immutable list; and two functions) ?? ???
And there is still the copy-paste problem in Envelope.hmm, that essentially the same functions are duplicated for the "draw..." and the "play..." objects - any idea how to improve this?

** Question: Where does one see the comments after @... properties? I knew that once, I believe - but have not found that explanation ...

Re: Envelope.hmm (0.6) - supporting envelopes in NWC

Reply #10 –
Quote from: hmmueller –
** Question: Where does one see the comments after @... properties? I knew that once, I believe - but have not found that explanation ...

When you have the object properties window open, you would click the "?" in the title bar, and then click on a particular field. If your second click is the object name (or any property which doesn't have an @ entry) it will display the information that precedes the first @ entry.

However, one note: you have named some of your @ labels incorrectly. You should use the "id" value, not the "label" value. For some of your Envelope.hmm entries, they are the same, so they work.

Mike

Re: Envelope.hmm (0.7) - supporting envelopes in NWC

Reply #11 –
Thank you! -> I have uploaded version 0.7 with corrected @ labels. Also, I have removed all the commented nwc.debug calls.

H.M.

Re: Envelope.hmm (0.7) - supporting envelopes in NWC

Reply #12 –
Quote from: hmmueller –
Thank you! -> I have uploaded version 0.7 with corrected @ labels. Also, I have removed all the commented nwc.debug calls.

H.M.
You're very welcome.

BTW, this is not a big deal, but I noticed that when you bumped the version to 0.7, it still says 0.6 in the help comment section.  You can fix that when you go to 0.8 :)

Also, I saw this usage comment:

Quote
--!! (17) Why are these plugins (both Envelope and its Maestro) shown as tiny triangles?

It's more of a "tear drop" but that is the standard icon for a user plugin. You've probably noticed that a number of other plugins show as sideways text with an oval around them.  There's a "toolbox" for drawing that which you can use if you like; you should look at the source code for one of those objects to see how it's done (try Cueheads.ms as an example).

Basically you would add the following code:
Code: (lua) [Select · Download]
-- Following line is outside any functions
local userObjSigName = nwc.toolbox.genSigName(userObjTypeName)
-- Following three lines are added to the top of the _draw hook
local isStaffSig = (t.Class == 'StaffSig')
local w = isStaffSig and nwc.toolbox.drawStaffSigLabel(userObjSigName) or 0
if not nwcdraw.isDrawing() then return w end

You could include additional information inside the oval, such as the name of the controller.
You would also need to add "width = _draw" to your return hooks.

Re: Envelope.hmm (0.8) - supporting envelopes in NWC

Reply #13 –
Version 0.8 now has an added user tool for shifting envelopes points of a single controller.

I tried the oval presentation for Maestros (thanks Mike!), but I do not use them as I need a clear readability for the Maestro objects (their controller number is important); and as they should typically be on separate envelope staffs anyway, there is no need to distinguish them visually from texts.

And the version should now be 0.8 everywhere :)

H.M.

Re: Envelope.hmm (0.9) - supporting MIDI envelopes in NWC

Reply #14 –
0.9 is a pure maintenance release: I have understood how to remove quite a bit of duplicated Lua code; and I moved the "problems" section here so that it does not pollute the main posting.

H.M.

Current problems

(1) [see code/please help] copy-pasted code for some methods because they work on different objects. Repaired inside Envelope.hmm; but duplication over EnvelopeMaestro.hmm and Envelope.hmm cannot be avoided, it seems.

(2) [see code/I'll investigate - by reading & understanding this] :find('prior',...) does not search to start of score. - solved

(3) [see code/please help] Defaults for properties have to be repeated when accessing them via :objProp(). - solved thanks to Mike.

(4) [see code/please help] It's necessary to add "+ 0" to objProp results, even though they are escaped with an "or constant". - solved thanks to Mike.

(5) [see code/please help] I would like to set the color of all envelope objects for one controller to the corresponding maestro's object color - is this possible? - not possible, says Mike - I will have to write a (probably embedded) user tool for this.

(6) [see code/please comment] The supported values for controllers are 0...127. The same range is also used for the pitch bend "pseudo controller", even though there, MIDI nominally supports fine-grained values from 0 to 16383, with "no bend" being 8192. In the Envelope object, "no bend" is indicated by 64, which might be confusing for users. However, I wouldn't like to change the 0...127 standard only for the pitch bend pseudo controller. "as designed

(7) [see code/please comment] 128 is used to select the "pitch bend pseudo controller". Controllers are selected from a drop-down list so that the "pitch bend controller" can be easily found.

(8) [see code/please comment] It would be nice to present the controllers also by name (where a standard name exists). On the other hand, controllers for advanced virtual instruments are often in the "general purpose" or even "undefined" segments, so this is maybe not that helpful anyway (envelopes can get a "Show as" string for presentation). Done in 0.6

(9) [see code/please help, or comment] The sloped lines do not really end where one would expect it. For example, a slope of span 1 over a whole rest ends whortly after the rest symbol, not at the place where the rest "logically ends". Should I extend the slope to the start of the next object? Fixed (I think) in 0.5

(10) [see code/please help] Is it helpful if the maestro object is of type "StaffSig"? What for?

(11) [see code/please comment, or help?] Printing of envelopes does not work - they get cut at line breaks. However, one must say that envelopes are not there for printing, but only for playing - so this is a "luxury problem". On the other hand, NWC objects typically do print themselves gracefully ...

(12) [please comment, or help?] Start and reset: When one starts playing a score in the middle, previous changes are not applied to the output, and I do not see how this is possible as there is, in general, no object at the start point that could do this. Moreover, such a start will use the values from the previous play, which might be unhelpful - e.g., if the previous play ended with a zero volume. I would assume that MPCs also have this problem ... but I have not yet looked into this. This is not a problem: I have now added volumes to a complete choral score with more than 100 measures, and I could start at any place and always got useful volumes. I'm not sure why it works so well, but it does!

(13) [please comment] Should the font of the "Show as" text be selectable (this might be helpful if there is more than one envelope on a staff).Fonts are not selectable, but in 0.6 I added a scale if someone is in great need.

Re: Envelope.hmm (1.0) - supporting MIDI envelopes in NWC

Reply #15 –
A few weeks ago, I completely rewrote the Envelope plugin: In a self-test, I'm (slowly) trying to create a better sound rendition of Tchaikovsky's String Serenade op.48, and I found that I - the inventor himself - was unable to efficiently use the Envelope object. The new version 1.0 is very different from the old one, but - I hope - much more usable. The initial posting of this thread has been updated accordingly.

The old version (0.9) of the plugin (Envelope and EnvelopMaestro) have been saved in this posting.

At some point, I'll also publish the Tchaikovsky - but there are quite a few other problems that I want to tackle or at least understand (e.g. a simpler way for soundfont selection).

Re: Envelope.hmm (1.0) - supporting MIDI envelopes in NWC

Reply #16 –
For some fun, here is a special use of an envelope: The "special controller 133" emits note values (instead of MIDI CC values). Thus, one can actually hear the changing controller values, which creates some special effects. Try and listen ... and (maybe after a small shock) have fun!

H.M.

Re: Envelope.hmm (1.0) - supporting MIDI envelopes in NWC

Reply #17 –
This looks Good". Thank you for your remittance.