List
using getPoseMatrix() MAXST
Posted Date: 2018-03-19 21:07     Edited Date: 2018-06-07 1:21     Writer: inactive

Hello!

I`m trying to use Instant Tracking example (android SDK) from MAXST with libGDX 3d render.When my application draws TexturedCube, it is located correctly. But when I place my model, using getPoseMatrix() ,it is located in the wrong coordinates without reference to the plane.  

private void setModelPos( TrackingResult trackables) {
    if (trackables == null || trackables.getCount() == 0)
        return;
    ModelInstance model = instances.first();
    Trackable trackable = trackables.getTrackable(0);
    final float [] maxSTcamM = CameraDevice.getInstance().getProjectionMatrix();
    final float [] trackPosM = trackable.getPoseMatrix();
    final float [] GDXcamM   = cam.projection.getValues();
   
    float[] modelLocalMatrix = new float[16];
    float[] modelViewMatrix  = new float[16];
   
    Matrix.setIdentityM(modelLocalMatrix,0);//object at zero-cord
    Matrix.invertM(modelViewMatrix,0,trackPosM,0);
    Matrix.multiplyMM(modelViewMatrix,0,modelViewMatrix,0,modelLocalMatrix,0);
   
    model.transform.set(modelViewMatrix);
    cam.update();
}

2nd option - rotate camera around model in coordinates center:

 

private void setCameraPos( TrackingResult trackables) {
    if (trackables == null || trackables.getCount() == 0)
        return;
    ModelInstance model = instances.first();
    Trackable trackable = trackables.getTrackable(0);
    final float [] maxSTcamM = CameraDevice.getInstance().getProjectionMatrix();
    final float [] trackPosM     = trackable.getPoseMatrix();
    float[] maxSTMVP = new float[16];
    //create MVP matrix
    Matrix.multiplyMM(maxSTMVP,0,maxSTcamM,0,trackPosM,0);
    //invert matrix to obtain trnasitions value
    Matrix.invertM(maxSTMVP,0,maxSTMVP,0);
    cam.position.set(maxSTMVP[12], maxSTMVP[13], maxSTMVP[14]);
    cam.up.set(maxSTMVP[4], maxSTMVP[5], maxSTMVP[6]);
    cam.lookAt(maxSTMVP[8], maxSTMVP[9], maxSTMVP[10]);
    model.transform.set(new Matrix4());
    model.transform.rotate(1.0F, 0.0F, 0.0F, -90.0F);
    cam.update();
}

What am i doing wrong? Thank

 

Posted Date: 2018-03-20 15:43     Edited Date: 2018-03-20 15:43     Writer: inactive

Hi,

It's very nice to know that there is a developer like you who try to attach our sdk to other rendering engine!

Did you succeed already to place cube on correct position?

And that was the combination of libGDX and opengl code?

If our quetions are right then we have two more questions about your rendering engine.

1. Can your engine set 4x4 matrix to model matrix?

2. Can your engine set 4x4 projection matrix to camera?

If two questions above are right also then our recommendations are like below

- Set trackable's pose matrix to model without inversion or transpose.

- Set projection matrix (maxSTcamM) to camera.

Please don't hesitate to post in this forum if you still have problem or another questions.

Thanks

- Maxst support team 

 

 

Posted Date: 2018-03-20 17:13     Edited Date: 2018-03-20 17:14     Writer: inactive

Thank for your reply!

I use sample project (MaxstARSDK_Android_Sample) to place cube on the plane. Then I want to draw my model according to the cube place.

this is my current result: https://prnt.sc/itv8jc

1. Yes, I can set 4x4 matrix, as model matrix.

2. libGDX works with 4x4 matrices, but I cant set it as projection matrix directly (it calculates on update). Instead I can set position, direction and up Vectors to my scene camera like this:

    cam.position.set(maxSTMVP[12], maxSTMVP[13], maxSTMVP[14]);
    cam.up.set(maxSTMVP[4], maxSTMVP[5], maxSTMVP[6]);
    cam.lookAt(maxSTMVP[8], maxSTMVP[9], maxSTMVP[10]);

 

Posted Date: 2018-03-21 4:54     Edited Date: 2018-03-21 4:54     Writer: inactive

Thanks for your posting.

I've found that libGDX game engine have two types of camera that are OrthographicCamera and PerspectiveCamera.

And any of the cameras don't provide api for setting projection matrix directly.

It's very weired the engine doesn't support the api because most of game engines provide api for setting projection matrix directly.

Anyway, we need time for finding the solution for you but I'm afraid that our team won't be able to do it in near future.

- Maxst support team

Posted Date: 2018-03-22 10:10     Edited Date: 2018-03-22 10:10     Writer: inactive

This is bad news since our team planned to purchase a "Pro-One time fee" license for use in our project. But this does not make sense if MAXST can only works with Unity.

Can you give access to the source code of your example with a penguin?
Maybe with your help we could get the right camera settings for our project based on the project matrix, or MAXST initial settings? Did you use gluLookAt(), if so, with what parameters?

Also we noticed an interesting fact about project matrices, when we compared them in different engines.

According to OpenGL documentation complete projection matrix is:

such values we got during debugging:

maybe you apply additional transformations?

Thanks for your reply.

Posted Date: 2018-03-22 15:20     Edited Date: 2018-03-22 15:20     Writer: inactive

Hi,

I don't have any experience of GDX game engine and it's very hard to support you directly.

So..I have made GDX game project at github. 

Here is the link. 

https://github.com/maxstdev/MaxstAR_GDX

Please let me know your github id and name.

Can you help me to add some codes in ImageTrackerRenderer.java ?

I can't even add the most simple 3d model for gdx engine in my projtect and I'm afraid that my project has right setting of gdx ( I made this project by using wizard )

If you can allow your time for adding PerspectiveCamera and simple 3d model then our team can start this projtect with you.

Please don't try to calculate projection matrix because it's very hard thing to understand if you don't have lots of computer vision and graphic skills.

Thanks 

- Maxst support team

Posted Date: 2018-03-22 19:27     Edited Date: 2018-03-22 19:27     Writer: inactive

Thank you so much for agreeing to help! this is very important for our team.

my github id: nikbazarov

I have already prepared a pull request with some code changes to your project. I hope that with your help, we will solve this problem )

Posted Date: 2018-03-22 21:50     Edited Date: 2018-03-22 21:50     Writer: inactive

It's such glad to know that my gdx project is acceptable to you.

I Added your id to collaborators list.

I home to see your changes soon.

- Maxst support team

 

Posted Date: 2018-03-23 10:30     Edited Date: 2018-03-23 10:30     Writer: inactive

I pushed some changes to "dev" branch. you can take a look when you have time

Posted Date: 2018-03-24 2:07     Edited Date: 2018-03-24 2:08     Writer: inactive

Hi,

I pushed some codes to feature/ApplyProjectionMatrix branch and you should find what they are.

The key point is that GDX camera doesn't have function for projection matrix setting but provides the 'combined' variable with public identifier instead.

It was very fortunate to find the combined variable.

In AR field the projection matrix usually means the combination of projection and view matrix.

And combined variable in GDX camera seems to mean same.

Please adjust our sample project as what you want.

Thanks

- Maxst support team

 

Posted Date: 2018-03-25 10:10     Edited Date: 2018-03-25 10:10     Writer: inactive

Hello!
Thank you so much for helping to solve this problem! You saved me a lot of time. The solution turned out to be simple, but I missed it because I was updating the camera, which led to the recalculation of vectors - my mistake.

Now I can safely finish our project )

Posted Date: 2018-03-26 0:08     Edited Date: 2018-03-26 0:21     Writer: inactive

Hi,

Honestly, AR project needs lots of skills for starting. In example they includes rendering engine, graphics, etc.

We want that your team will get good result in your project and expand our relationship more.

And we have a question that there is any problem if we provide your sample code and content in our github sample in public.

Thanks

- Maxst support team.

 

Posted Date: 2018-06-06 11:00     Edited Date: 2018-06-06 11:00     Writer: inactive

hey, can you guys give me the source code?

Posted Date: 2018-06-06 22:43     Edited Date: 2018-06-06 22:43     Writer: inactive

Hi,

GDX engine sample is on github and please refer the link below.

https://github.com/maxstdev/MaxstAR_GDX

Thank you.

Jack

SDK Team, Maxst

Posted Date: 2018-06-07 1:06     Edited Date: 2018-06-07 1:06     Writer: inactive

hey,

thanks for your reply, but i can't find the use of libgdx in your source code, is it to show the aircraft model or just to show the cube? 

thanks

Posted Date: 2018-06-07 1:21     Edited Date: 2018-06-07 1:21     Writer: inactive

The green box is a gdx game object and you should find the way how to create it from 'core' module.

And the textured cube is our sdk sample.

Sample code on github is to show the combination of our sdk sample and gdx game object.

Our team has little knowledge about gdx game engine so we can't provide more about the usage of it.