ModularFX is a GUI for Gensound

[ali1234]

https://github.com/ali1234/modularfx

This is highly experimental code using https://gitlab.com/pavel.krupala/pyqt-node-editor

After starting the program, you must click file->new to make a new graph. Then drag nodes from the right hand side and drop them in the graph. Connect them up by clicking the orange dots. The right side is the output. The left side has two sockets: the top one applies a transform, the bottom one concatenates to whatever is connected. After you made a graph, click play on any node to play the sound up to that point.

The list of available nodes is (mostly) automatically generated by introspecting gensound.

It is currently not possible to connect curves to signal generators.

Warning: saving is not implemented yet, so don't make anything too complex.

[ali1234]

• Made the nodes and sockets automatically resize to fit their contents.
• Added parameter sockets to match the parameter fields on each node. The fields now disappear if the sockets are connected, and the value from the linked node is used instead.
• Grouping of nodes that have the same parameters eg oscillators and noise.
• Automatic mixing (+) on signal inputs.

[ali1234]

• Made the nodes prettier.
• Implemented serialization.
• Serialization allows load/save, copy/paste, and undo to function.

Note that the file format is still very likely to change.

I think this is basically feature complete now. You should be able to use almost all node types. Some special nodes may be needed for slicing, gain etc.

At this point it would be very helpful if all the user-facing objects had default arguments and type annotations.

[Quefumas]

This is really cool! Thanks for sharing!

I haven't had time to play too much with it, but it's working great.

What are your plans for the future?

[Quefumas]

As for default arguments and type annotations: I'd like to have type annotations, but I feel the code is not yet ripe enough. For example, the frequency for Sine may be either a numeric type, a Curve or a string which is then parsed (using melodic shorthand notation as in the wiki), and I feel that some more thought is needed to find the 'correct' type annotation for this type of parameter. Namely, feeding a numeric or Curve will yield a single note, while a string can yield a melody (Sequence object). I'm very open to discussing this, but I'd like to be more certain before I introduce this to the code.

Default arguments: I believe Signal objects typically have those, while many Transforms don't. However, in many cases it is not clear what the default arguments should be, and even how many of them should there be (e.g. Repan). Also, I'd like to consider other use cases of this code, for which it is sometimes preferable to force the user to make a choice for the arguments. Are there any arguments in particular that you think should have a specific default value?

[ali1234]

I'm thinking about vendoring the nodeeditor library, because I want the program to be able to run headless without installing Qt. I want to use it in a build pipeline to generate the sound effects for my game. That needs a fairly simple but also fairly large refactoring of the core library.

I think some operations are completely missing because I don't know about them.

Default arguments don't necessarily have to be "correct", they just have to ensure that the node makes some kind of sound by default. This is really most useful for me to test that things work if I don't understand what they are supposed to do. :)

Type annotations would allow the sockets to be coloured to indicate what they can connect to. For example if frequency has Curve as an annotation, it could be coloured green, like a Curve output. If it doesn't have the Curve annotation, then connections from a Curve output can be automatically rejected.

I also want to make generic "value" nodes so that you don't have to keep typing the same number in multiple places, perhaps with sliders and knobs instead of a text box.

I think there is also a problem with some nodes that accept an object as a parameter. ast.literal_eval can't handle that so they will need to be converted to combo boxes.

[ali1234]

Also, this is all very rough proof-of-concept code, so it needs a big rewrite to make it good.

[ali1234]

Another thing i will need to add is some kind of "output" node. I'm not sure whether to allow multiple outputs in one graph. They would then need to be named in order to differentiate them on the command line. On the other hand, only allowing one output would be an artificial limitation, and either way I have to handle the case of zero outputs.

On a related note, it would be nice if gensound supported signed and unsigned for every byte_width. I need signed 8 bit raw samples.

[ali1234]

Something I'm working on...

[ali1234]

Today I implemented value nodes and a polyphonic sink using pygame:

To use this on Linux:

• Run ModularFX and click Help -> Examples -> Midi
• Run aconnect -l in terminal and look for the ModularFX MIDI port. It will look like this:

 client 128: 'RtMidiIn Client' [type=user,pid=1150781]
    0 'ModularFX       '
	Connected From: 14:0

• Run vkeybd --addr 128:0 replacing 128 with whatever number you saw next to client in aconnect -l.
• The virtual keyboard can be used with mouse or keyboard (so long as the window is focussed.

[ali1234]

Today I implemented an Inspector node, which attempts to turn nodes back into the equivalent python code:

Clicking "Inspect" in the above causes the following output:

(
    gensound.signals.Sine(
        frequency=gensound.curve.Line(begin=220, end=880, duration=300.0),
        duration=5000.0,
        phase=None,
    )
) * gensound.transforms.ADSR(
    attack=2000, decay=1000, sustain=0.75, release=2000, hold=0
)

You need black or python 3.9 for code formatting, otherwise it will all be on one line.