Advanced effects demos

[swingingtom]

Hey ThreeMeshUI community !

For the current project I work on (on hold during holidays) after requiring letterSpacing and kerning, it also requires (lets called these) "advanced effects".

Those requirements weren't there at project kick-off, but they've been added during production.
I probably wouldn't have choose three-mesh-ui if I would have known the whole bundle of current requirements.

But now, this project is mainly built on three-mesh-ui, and I have to implements more advanced effects :

Second Set of UV, textures, FragmentShaders

Animating characters

As we can see, those effects are not that hard to make with Three-Mesh-UI. And I would like to produce some codesandbox demos. But It might be great to first ask how thoses effects should be handle in your opinions.

Core vs Fork ( Main question )

I currently forked the project to make some changes, but thoses changes are not mandatory, and they could be rolled back and placed in onAfterUpdate instead.

The real question here is, are those kind of features/effects rising some interest in the users/maintainers/contributors of three-mesh--ui ? Because I would rather bent my project code to use the official three-mesh-ui than easing my project code and use a forked/umaintened version of three-mesh-ui.

Optional core change to ease these kind of effects

• Add boolean to merge/not-merge geometries for per character animation ( unable to animate chars without this change or another )
• Use more #include <chunks>; in shaders. Place three-mesh-ui shaders chunks in Three.shaderChunks easing onBeforeCompile and reducing code duplication.
• Synch cloned fontMaterial. Font texture is not yet loaded when fontMaterial is cloned, leading to checks and changes onAfterUpdate.
• Access of Shaders constructors ( new ThreeMeshUi.FontShader({...x})
• Allow segments counts in Glyph ( could smooth some vertex shaders effects )

Fun fact
While working on those demos, my SO walked in and thought I was working for a food chain called "3 méchouis" Méchoui on wikipedia

[felixmariotto]

Hi @swingingtom !

The real question here is, are those kind of features/effects rising some interest in the users/maintainers/contributors of three-mesh--ui ?

Personally I think a demo (or several) like this would be a very nice addition ! Those GIFs above look really good.

Add boolean to merge/not-merge geometries for per character animation ( unable to animate chars without this change or another )

Why not, although for some use-cases such as this one, the impact on performance would be important.

Use more #include ; in shaders. Place three-mesh-ui shaders chunks in Three.shaderChunks easing onBeforeCompile and reducing code duplication.

Do you mean this ? : THREE.ShaderChunk.tmu_msdf_frag = 'fragment code..'
Not sure it's good practice to meddle with three.js objects... why would it ease onBeforeCompile and reduce code duplication ?

Allow segments counts in Glyph ( could smooth some vertex shaders effects )

I don't get what you mean, what do you call segments ? All glyph are quads currently..

I wonder If we should make a textFragment attribute, so users can throw their bit of fragment shader that we would parse if necessary and put here ?

[swingingtom]

Why not, although for some use-cases such as this one, the impact on performance would be important.

Exactly. And "effect text" use cases would surely be less required than "content text" use cases. So ideally "content text" should suffer from no performance loss in order to allow "effect text".

For the moment, and without further thinking it could lead to :

• A new property ( mergedGeometries? ) with a default value of true, documented only via Effect Samples in order to avoid it as much as possible in all standard cases.
• A new type new ThreeMeshUI.AdvancedText() , new ThreeMeshUI.UnmergedText() ?

They both have their pros and cons that could be listed.
One major pro for "a new type", but it would also be its main cons : This could initiate a core refactor smoothly : Composition over inheritance, FontClass, etc...
During the transition, Text and 'AdvancedText' would have different source code and perhaps different ways to be used, which won't be good, especially in a long run.

Do you mean this ? : THREE.ShaderChunk.tmu_msdf_frag = 'fragment code..'
Not sure it's good practice to meddle with three.js objects... why would it ease onBeforeCompile and reduce code duplication ?

That was exactly what I was saying, and as you are, Im not sure too this will be a good practice. It just offers some nice abstraction, and easy replacement. What I was trying to avoid is 'outdated replacement' :

const text = new ThreeMeshUI.Text({ content: "HelloWorld"});
text.onAfterUpdate = function () {
     
        const standardMaterial = text.fontMaterial;
        const customMaterial = standardMaterial.clone();

        customMaterial.onBeforeCompile = function (shader) {
            shader.fragmentShader =
                 shader.fragmentShader.replace( 
                 'if( alpha < 0.02) discard;',
                 "updatedAlphaTest_Chunk"
            )
        }
    }

I choose the worst example, but as soon as TMU would like to operate some changes ( ie: affinate 0.02 to 0.018, reduce 0.02 to .02, etc... ) this would make the previous replace outdated and inoperant.

Going full chunks include as follow, won't suffer from that

const text = new ThreeMeshUI.Text({ content: "HelloWorld"});
text.onAfterUpdate = function () {
     
        const standardMaterial = text.fontMaterial;
        const customMaterial = standardMaterial.clone();

        customMaterial.onBeforeCompile = function (shader) {
            shader.fragmentShader =
                 shader.fragmentShader.replace( 
                 '#include <tmu_alpha_test>;',
                 "updatedAlphaTest_Chunk"
            )
        }
    }

Also, being able to create / extend a TMU material would be nice addition

text.fontMaterial = new ExtendedFontMaterial()// class ExtendedFontMaterial extends ThreeMeshUi.FontMaterial;
// or
const customMaterial = new ThreeMeshUI.FontMaterial();
customMaterial.onBeforeCompile = (shader) => { /*[...]*/ };
text.fontMaterial = customMaterial;

// vs
text.onAfterUpdate = function(){
  const customMaterial = text.fontMaterial.clone();
  // ...
}

I don't get what you mean, what do you call segments ? All glyph are quads currently..

export default class MSDFGlyph extends PlaneBufferGeometry {
    constructor( inline, font ) {
        const char = inline.glyph;
        const fontSize = inline.fontSize;

        super( fontSize, fontSize );

with

export default class MSDFGlyph extends PlaneBufferGeometry {
    constructor( inline, font ) {
        const char = inline.glyph;
        const fontSize = inline.fontSize;

        super( fontSize, fontSize, 5 , 5 ); // width and height segment counts

But I need to produce a vertexShader demo to be sure it really affects something.

Thanks for your feedback. I suggest that I start a demo without any core changes, and we could affine the produced code to make it more intuitive later. Also, sorry for always making big subjects on anything.

[felixmariotto]

Also, sorry for always making big subjects on anything

No worries, I'm grateful that you put so much effort into contributing ! I am sorry if I seem offhanded, my personal life is a mess lately, so even if I will always find a bit of time to maintain three-mesh-ui I can't currently throw days into it...

I got what you mean with the segment count, you want to make the letters wriggle ! Sounds like a valid use-case, so we could have new widthSegments and heightSegments props.

About "effect text" new prop vs new type: I think a new type may lead to some code duplication with Text, but I agree it would be cleaner. Maybe Text and AdvancedText should delegate to a common new type.

Anyway you can indeed file a PR with the demo, so we can refine it on the go !

[swingingtom]

Anyway you can indeed file a PR with the demo, so we can refine it on the go !

PR or codesandbox ?

[felixmariotto]

I prefer a PR

[swingingtom]

No worries, I'm grateful that you put so much effort into contributing ! I am sorry if I seem offhanded, my personal life is a mess lately, so even if I will always find a bit of time to maintain three-mesh-ui I can't currently throw days into it...

Even if you didn't threw days lastly, you were always prompt to respond and take time. Hopefully this will get better for you, but aside opensource libs, programming, contributions, etc... We all have lifes, and they should always comes first. No matter what.

[swingingtom]

I wonder If we should make a textFragment attribute, so users can throw their bit of fragment shader that we would parse if necessary and put here ?

Of course we could start with it, but Im not sure this will allow enough flexibilty in the long term, especially if Uniforms|Attribute|Varying are involved in the required effect.

[swingingtom]

(SegmentCounts) But I need to produce a vertexShader demo to be sure it really affects something.

I didn't produce the best example, but we can notice differences according the number of segments.

5 segments w&h

only one segment

MSDFGlyph.mapUv() wasn't ready to handle multiple segments, but it seems to simplify its logic

mapUVs( font, char ) {

        const charOBJ = font.chars.find( charOBJ => charOBJ.char === char );
        const common = font.common;
        const xMin = charOBJ.x / common.scaleW;
        const xMax = (charOBJ.x + charOBJ.width ) / common.scaleW;
        const yMin =  1 -((charOBJ.y + charOBJ.height ) / common.scaleH);
        const yMax = 1 - (charOBJ.y / common.scaleH);

        // added new vars
        const width = xMax - xMin;
        const height = yMax - yMin;

        const uvAttribute = this.attributes.uv;
        for ( let i = 0; i < uvAttribute.count; i ++ ) {

            let u = uvAttribute.getX( i );
            let v = uvAttribute.getY( i );

            [ u, v ] = (()=> {

                // same logic for any uv (seems ok)
                return [xMin + u*width, yMin+v*height];

                // Per uv logic
                // switch ( i ) {
                // case 0 : return [ xMin, yMax ]
                // case 1 : return [ xMax, yMax ]
                // case 2 : return [ xMin, yMin ]
                // case 3 : return [ xMax, yMin ]
                // }
            })();

            uvAttribute.setXY( i, u, v );
        }
    }

For segment counts, instead of static values, something "per size" might be better to offer a uniformity.
If we defined 5 vertical segments, space between vertical segment of a "M" glyph would be more spaced than vertical segments of a "I" glyph.

[felixmariotto]

For segment counts, instead of static values, something "per size" might be better to offer a uniformity.

This makes the issue much more complicated.. I'm not even sure what the API should look like if we do this.

Let's say the user has a line of numbers, which are higher on average. Then when setting heightSegments = 5 and getting 8 segments instead they will be upset. At the same time an API like heightSegments = 'medium' is a bit too inaccurate.

[felixmariotto]

Or else we add a lineHeightSegments prop, so it's clear the segmentation is per-line. InlineBlock Frames should also be segmented if we follow this logic.

[swingingtom]

Something like lineHeightSegments prop is indeed much useable and will offer more consistency amongs glyphs variations than simple "counts".

At first I thought about an area or distance between segments rather than segment counts. When computing glyph dimensions, we could compute "glyph width / hSegLength" equals x horizontal segments for that specific glyph.
At the end I guess they will both tend to the same results.

Let's not forget those use cases won't be that frequent