I'm trying to make a simple 3D editor and I'm not sure how I should go about it.
I came up to two option:
1. In first I store non transformed model, cached model that store transformation on face and edges and display model that store already full transfomed model, If I scaling model I take vertecis info from cache and model will not destroed if I scale it to straight plane, that all sound good but with this solution I faced with this problem that I just have no idea how to solve (If you have any ideas please let me know). Basically wanted to do this only for optimization of full model scaling operation.
2. My second option (and plan B.) it's storing applied transformation for each vertices, so I will have origin model and display model so I don't need do some inverting or mapping of tools axis to unscaled space. Cache model also should have been implemented in that way so it's seem not bad solution also memory cost will be quatertion + translation = 16 byte, ~ 32KiB on 1000 vertecis, that more than enought for my need by I just intristing to now is there exis better solution?

To be honest I don't understand what are you doing in this or reddit's post/images.

What does "scale it to straight plane" means?

What is the issue with axes when you scale? Axis direction does not change during scale, why would they be any problem at all for scaling forward or inverse?

What does "scale it to straight plane" means?

I meen this, it's a cube scaled by X axis to plane (sorry I can't register on imgur from my country)

What is the issue with axes when you scale? Axis direction does not change during scale, why would they be any problem at all for scaling forward or inverse?

I'm understand this can be not obvious, It is not been my main question soso I explained it badly. So as I wrote in reddit post in example I interact with model face, all interaction happan on display model, it is model that been affected by some global model transformation (it's not shader global transform it's happen on CPU side) in that example it's scale.
Then, when the user applies scaling to the model's face, I need to scale in the direction of the axis the user is interacting with, because that is of course the expected behavior. For this I need apply scaling on face that not yet transformed by global model scale (as display model does), if I just take axis that I produce for interaction with display model scaling scaling will be happaning in wrong direction.

So if i get rid of transformation that must be affected globally for model, interaction that happening with display model will be happening in same space so I don't will have that problem.

Sorry If it maybe still be not clear what problem is.

That why I ask if somebody know better solution than storing applied transformation for each vertices for preventing destroying vertices information.

I think by "scale" here you mean "translate", no? It seems to me that you are picking faces of 3d model and moving them in direction of specified direction. So translation, not scaling. Or "transform" in general - which can be translate, scale or rotate combined.

Usually editors do that by keeping inverse object-to-world transformation matrix - so world-to-object transformation. And every transformation in world space is multiplied by this matrix to get to object-space. Then you can store it and/or continue operate with in object space.

Calculating inverse matrix may seem expensive, but if that is only for UI based interactions, then it's not a big deal. Another cheap trick is too keep inverse matrix calculated all the time, so whenever you're applying some transformation to object-to-world matrix, you do inverse transformation to world-to-object matrix. And this "inverse" does not need to be actual "inverse" - like if you're applying translation matrix for vector P, you simply apply translation matrix for vector -P to the inverse matrix. This way your world-to-object matrix will be always up to date.

As you said it will require extra memory, but unless you're talking about 100 million vertices, it should not be big deal.

If your face/vertex transformations need to be applied only to few places, you can keep transformations indirectly - assign only one extra 32-bit int "index" to each vertex or face. If index=0 that means no transformation needed, otherwise it references matrix array element. Basically keep all matrices in separate array and "allocate" them only for vertices you need. This way you'll pay memory only for things that need it.

I think what OP means is that he wants to scale a 3D object in local space instead of world space.

From the reddit post/images, I gather that you're using the model space axis in world space which, instead of scaling, it's applying a shear to the model.

The axis in the following equation is probably the local (or model) space axis, e.g. (1, 0, 0) for the X axis.

m4x4 Transform = Model->ModelScale * Model->Axis * Model->GlobalScal;

I don't know how you're applying the transform, but from what I can see, you're applying a "local" X scale on the world (or global) space. To fix this, what you need to do is transform first the local X axis to world space, and then scale that axis.

Having said that, I'm not sure why you'd want to do that, though. This is the same operation as scaling the object in local space and transforming the result to global space.

Actually I meen "scale", "translate" it's different tool and is work well with just applying inverse matrix as you said to direction in wich I calculate in global space. It working because for translation I need only one axis (or direction in which I need move vertices) so I don't have some "non-uniformity".

Things bellow I tested without dividing scaling on gobal and model as been on this firt reddit post, so model transform looks like


Also applying scaling to face that scale only on basis vector (1, 0, 0) (0, 1, 0) (0, 0, 1) also work well (example). Here I already need apply scale matrix but in this case I just need move global basis to model space and I don't need to know current state of model scale because it already in "model space" for this case.

(I use row-mojor matrix so it's from left to right)


ScaleV is just represent magnitude of scaling for some axis, for example scale by X axis wiil look like (1.1f, 1.0f, 1.0f) for example.

But when I need perform scale in arbitrary direction (as I said in post doing this on face of model just for example) I need perform full inverse transform so that after applying Model->ScaleMat vertices will be place in expeted position. In solution that you suggest it should looks like this?(maybe I wrong)

In this case vertices completely "go crazy"


Result that looks more like closely for what I need but is also produces incorrect result (example)

If not looking that scale happen in not adequate speed, rotation happaning from my understanding because after apply InvScale transform, vectors in scale matrix not anymore orhogonal as I demonstrate on screen shot on reddit post.

Sorry for long replying I have a little time in the evening for test and to think carefully about this

I still don't fully understand what you're trying to do, but one thing caught my attention:

ScaleV is just represent magnitude of scaling for some axis, for example scale by X axis wiil look like (1.1f, 1.0f, 1.0f) for example.

You have an axis you want to scale, i.e. Model->Axis so, to represent a scaling factor along that axis you only need a scalar value. E.g. if you want to double the size along the X axis you do: 2.f * (1.f, 0.f, 0.f), which results in a scaled vector: (2.f, 0.f, 0.f).

Your ScaleV factor is a scaled vector already so, when you multiply a "scaling vector" with a "scaling factor" that's a vector as well, you're scaling along the transformed "factor" axis, not your intended axis.

You can try doing the matrix multiplication by hand to see how the transformation works.

You have an axis you want to scale, i.e. Model->Axis so

Sorry I don't yet reply to you previous comment so you may misunderstand some things. Problem not in scaling on model axis, in reddit post example cube scaled by X axis is right, it is just been rotate by 90deg and then scaled by global X. This case work well as I said here.

Also applying scaling to face that scale only on basis vector (1, 0, 0) (0, 1, 0) (0, 0, 1) also work well (example). Here I already need apply scale matrix but in this case I just need move global basis to model space and I don't need to know current state of model scale because it already in "model space" for this case.

Problem arise when I try scale on axis that not align with the basis vector like I build for face scaling

But when I need perform scale in arbitrary direction (as I said in post doing this on face of model just for example) I need perform full inverse transform so that after applying Model->ScaleMat vertices will be place in expected position. In solution that you suggest it should looks like this?(maybe I wrong)

In this case I need more data for right scaling. At first I used only 3 value as you said but this been not worked, that why I use Tool->Axis. At first I tested on uncalled model faces and when do test on scaled then that problem been arise.

In this case I need more data for right scaling. At first I used only 3 value as you said but this been not worked, that why I use Tool->Axis. At first I tested on uncalled model faces and when do test on scaled then that problem been arise.

That's exactly what I explained in my last reply. You're confusing the scale factor with the scale axis. A scale of (2, 2, 0) is not a scale of 2x in X and a scale of 2x in Y, but a scale of 2X in a vector exactly between the X & Y axes (45 degrees up from the X axis).

When you combine a scale of (2, 2, 0) with either the X or Y axes (1, 0, 0) or (0, 1, 0) you effectively cancel the other coordinate, so it ends up working by chance. However, when you rotate the axis first and it's no longer aligned to a basis vector, you end up scaling along the (2, 2, 0) vector, which is not what you want.

What you want is a scalar factor and a direction vector:

Hmm, yeah it seems like I misunderstood you.
I will think about it more carefully and try to test it in the coming days.

You been technically correct, I don't get rotation or some other artifact. Thet code that I use when I experimented:


But this is still give to me incorrect result. Maybe I still don't something understand (please don't get angry:) )
Problem arise when model rotated around more than one axis (before scale and after scale), to me this feel like vertices don't have enough freedom for move. This why I been start used axis for scaling because they "correcting" vertex position on scaling and it solved this problem for non-scale model.

As I said maybe I don't get something but after all trying I just don't see how this can work with just one scale axis and I need full scale matrix.

You can't just pick one axis and invert it here:


You need to invert the entire space (whole matrix) and, *then* pick the axis you want to scale.

In this way result the same

This is not how scaling a matrix works:


As I said in my first reply, I recommend you work through the matrix operations by hand. That way, you'll see how scaling affects the matrix and why your method doesn't work.