#1 2018-10-12 17:32:49

windwakr

Member

Registered: 2010-06-10

Posts: 22

freelook Lua script

Putting this here to get people to stop replying in the issue page on github.

DeSmuME now has the ability to implement freelook through Lua scripting. Read zeromus' explanation on that here.

I've created this crappy script to look around with a controller. Honestly have no clue what I'm doing, but it somehow works well enough with most games I've tried. It's not a universal solution, though. There's a bunch of games that it doesn't work that great or at all with.

Some webms showing it off:
https://files.catbox.moe/zfni2x.webm
https://files.catbox.moe/0fn3dt.webm
https://files.catbox.moe/s8hev7.webm

Use with something like an xbox 360 controller, or modify the controls to suit your needs.

Left stick moves left/right and forward/back
Right stick rotates
Triggers move up down(according to where you're looking)

A button resets position, rotation, and pitch offset
B button resets rotation and pitch offset
X sets pitch offset
Y resets the movement and rotation scale

Dpad Up/Down sets the movement scale
Dpad Left/Right sets the rotation scale
Hold the left or right bumper to make smaller or bigger changes to the scales

Setting the pitch offset is important. Without doing so, your view will roll around as you look left and right in most games.
Angle your pitch towards the "horizon" and press X. Then try and rotate the camera 90 degrees to the left or right and adjust up/down a little while pressing X until everything's level.

--Assumes you have an Xbox style controller

joyID = -1; --Joystick ID to use. The script will attempt to find a suitable joystick, but you can manually set this if you wish
yaw = 0.0;
pitch = 0.0;
pitchoffs = 0.0; --used to correct for the initial pitch of the camera
xpos = 0.0; --coordinates in a virtual 3D space
ypos = 0.0; --^^^
zpos = 0.0; --^^^
matAdjustment = {1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1};
movscale = 1.0;
rotscale = 1.0;
cooldown = 0;

function MatrixMult4x4(m1, m2) --sorta copied/inspired from some lua matrix library
    local mtx = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}
    for i=1,4 do
        for j=1,4 do
            local num = 0
            for n=1,4 do
                num = num + m1[(i-1)*4+n] * m2[(n-1)*4+j]
            mtx[(i-1)*4+j] = num
            end
        end
    end
    return mtx;
end

function Rotate(angle,x,y,z)
    local s = math.sin(math.rad(angle));
    local c = math.cos(math.rad(angle));
    return {(1-c)*x*x+c, (1-c)*y*x-s*z, (1-c)*z*x+s*y, 0,
            (1-c)*x*y+s*z, (1-c)*y*y+c, (1-c)*z*y-s*x, 0,
            (1-c)*x*z-s*y, (1-c)*y*z+s*x, (1-c)*z*z+c, 0,
            0, 0, 0, 1};
end

function vecrotx(angle, vx, vy, vz) --used for pitch correction in the functions below
    return vx,
            vy * math.cos(math.rad(angle)) - vz * math.sin(math.rad(angle)),
            vy * math.sin(math.rad(angle)) + vz * math.cos(math.rad(angle));
end

function MoveForward(l) --sorta based off https://www.gamedev.net/forums/topic/415144-get-forward-right-up-vectors-from-pitch-yaw-roll/
    local fX = math.sin(math.rad(yaw)) * math.cos(math.rad(pitch+pitchoffs));
    local fY = -math.sin(math.rad(pitch+pitchoffs));
    local fZ = -math.cos(math.rad(yaw)) * math.cos(math.rad(pitch+pitchoffs));
    
    fX, fY, fZ = vecrotx(pitchoffs, fX, fY, fZ);
    
    local mag = math.sqrt(fX*fX + fY*fY + fZ*fZ);
    fX = fX / mag; fY = fY / mag; fZ = fZ / mag;
    fX = fX * l; fY = fY * l; fZ = fZ * l;
    xpos = xpos + fX; ypos = ypos + fY; zpos = zpos + fZ;
end

function MoveUp(l)
    local fX = math.sin(math.rad(yaw)) * math.cos(math.rad(pitch+pitchoffs+90));
    local fY = -math.sin(math.rad(pitch+pitchoffs+90));
    local fZ = -math.cos(math.rad(yaw)) * math.cos(math.rad(pitch+pitchoffs+90));
    
    fX, fY, fZ = vecrotx(pitchoffs, fX, fY, fZ);
    
    local mag = math.sqrt(fX*fX + fY*fY + fZ*fZ);
    fX = fX / mag; fY = fY / mag; fZ = fZ / mag;
    fX = fX * l; fY = fY * l; fZ = fZ * l;
    xpos = xpos + fX; ypos = ypos + fY; zpos = zpos + fZ;
end

function MoveRight(l)
    local fX = math.sin(math.rad(yaw+90));
    local fY = 0;
    local fZ = -math.cos(math.rad(yaw+90));
    
    fX, fY, fZ = vecrotx(pitchoffs, fX, fY, fZ);
    
    local mag = math.sqrt(fX*fX + fY*fY + fZ*fZ);
    fX = fX / mag; fY = fY / mag; fZ = fZ / mag;
    fX = fX * l; fY = fY * l; fZ = fZ * l;
    xpos = xpos + fX; ypos = ypos + fY; zpos = zpos + fZ;
end

function on3d()
    emu.set3dtransform(2,matAdjustment);
end
emu.register3devent(on3d);
emu.set3dtransform(1,nil);

if joyID == -1 then
    for i = 0, 15 do
        c = controller.get(i);
        if c.x and c.y and c.z and c.u and c.r then --simple check that only looks for all the needed axes
            joyID = i;
            print("Found suitable joystick, id " .. i);
            break;
        end
    end
else
    print("Using manually set joystick, id " .. joyID);
end
if joyID == -1 then
    print("No suitable joystick found!");
    while true do
        emu.frameadvance();
    end
end

while true do
    local key = controller.get(joyID)
    
    scalemult = 1;
    if key["5"] then --Left bumper
        scalemult = 0.1;
    elseif key["6"] then --Right bumper
        scalemult = 10;
    end
    if key.up and cooldown==0 then --Dpad
        cooldown = 10;
        movscale = movscale + (0.1 * scalemult);
        print("Movement scale: " .. movscale);
    end
    if key.down and cooldown==0 then --Dpad
        cooldown = 10;
        movscale = movscale - (0.1 * scalemult);
        print("Movement scale: " .. movscale);
    end
    if key.right and cooldown==0 then --Dpad
        cooldown = 10;
        rotscale = rotscale + (0.1 * scalemult);
        print("Rotation scale: " .. rotscale);
    end
    if key.left and cooldown==0 then --Dpad
        cooldown = 10;
        rotscale = rotscale - (0.1 * scalemult);
        print("Rotation scale: " .. rotscale);
    end
    
    if(math.abs(key.x)>0.25) then --Left Stick
        MoveRight(-key.x * movscale);
    end
    if(math.abs(key.y)>0.25) then --Left Stick
        MoveForward(key.y * movscale);
    end
    if(math.abs(key.z)>0.25) then --Triggers
        MoveUp(-key.z * movscale);
    end
    if(math.abs(key.u)>0.25) then --Right Stick
        yaw = (yaw + (key.u * rotscale)) % 360;
    end
    if(math.abs(key.r)>0.25) then --Right Stick
        pitch = (pitch + (key.r * rotscale)) % 360;
    end
    
    if key["1"] and cooldown==0 then --A button
        cooldown = 20;
        yaw = 0.0;
        pitch = 0.0;
        pitchoffs = 0.0;
        xpos = 0.0;
        ypos = 0.0;
        zpos = 0.0;
        movscale = 1.0;
        rotscale = 1.0;
        print("Reset everything");
    end
    if key["2"] and cooldown==0 then --B button
        cooldown = 20;
        yaw = 0.0;
        pitch = 0.0;
        pitchoffs = 0.0;
        print("Reset rotations");
    end
    if key["3"] then --X button
        pitchoffs = -pitch;
    end
    if key["4"] and cooldown==0 then --Y button
        cooldown = 20;
        movscale = 1.0;
        rotscale = 1.0;
        print("Movement/rotation scales reset");
    end
    
    local tmp = Rotate(-yaw, 0.0, math.cos(math.rad(pitchoffs)), math.sin(math.rad(pitchoffs)));
    tmp = MatrixMult4x4(tmp, Rotate(-pitch, 1.0, 0.0, 0.0));
    local translation = {1,0,0,0,0,1,0,0,0,0,1,0,xpos,ypos,zpos,1};
    matAdjustment = MatrixMult4x4(translation, tmp);
    
    if cooldown>0 then cooldown=cooldown-1 end
    emu.frameadvance();
end

Last edited by windwakr (2019-03-23 00:22:39)

zeromus

Radical Ninja

Registered: 2009-01-05

Posts: 6,053

Re: freelook Lua script

It looks like we need your fork. Which is now AWOL.

Even if it's sloppy, paste your code onto our master and send a PR and I'll make sure it's OK
also I too was missing math.abs.. I have no idea why (don't want to know; I hate lua) but anyway here is a math.abs:

math.abs = function(x)
  if x == nil then return 0 end
  if x<0 then return -x end
  return x;
end

windwakr

Member

Registered: 2010-06-10

Posts: 22

Re: freelook Lua script

It looks like we need your fork. Which is now AWOL.

Even if it's sloppy, paste your code onto our master and send a PR and I'll make sure it's OK
also I too was missing math.abs.. I have no idea why (don't want to know; I hate lua) but anyway here is a math.abs:

math.abs = function(x)
  if x == nil then return 0 end
  if x<0 then return -x end
  return x;
end

My fork was just what the queefersutherland guy PR'd, I don't think he made any changes to it. I didn't want to make a PR myself. The script just assumes you have a controller plugged in that's joystick id 0 and has the 3 axes it checks for, and that causes an error when trying to use them if they don't exist. It really should check for the axes before trying to use them.

Last edited by windwakr (2018-11-01 02:00:36)

windwakr

Member

Registered: 2010-06-10

Posts: 22

Re: freelook Lua script

Got a new controller recently and was messing around with this a bit. Updated the first post but here's the changelog.
*Now attempts to scan for a suitable controller instead of just using the first one
*Only updates the matrix on 3D events(still don't think I'm doing it properly, but this fixes most of the issues I had)
*Left and right bumpers now act as multipliers for the dpad scale settings
*Y button resets movement and rotation scale to 1.0

Last edited by windwakr (2019-03-08 22:19:22)

Flyhngon

Member

Registered: 2019-04-12

Posts: 1

Re: freelook Lua script

Total coding noob here... I'm trying wrap my head around how to use lua code, and I can't wrap my head around how to get the freelook script to work. I get that I actually have to adjust some things in the code myself, but I'm unclear about what values need changing, and no matter what I do the code stops running because register3devent is a nil value..? Even when I think I'm defining it correctly?

Freecam is a feature I'd really like, but I'm too tech-illiterate to wrap my head around how the lua script works, even with the instructions given on the github and in the script itself.. help?

Bnthomason

Member

Registered: 2019-04-16

Posts: 9

Re: freelook Lua script

Hey, what emulator build are you using, and how would I get this to work with a 9.12 emulator with decent speed and graphics?

windwakr

Member

Registered: 2010-06-10

Posts: 22

Re: freelook Lua script

Total coding noob here... I'm trying wrap my head around how to use lua code, and I can't wrap my head around how to get the freelook script to work. I get that I actually have to adjust some things in the code myself, but I'm unclear about what values need changing, and no matter what I do the code stops running because register3devent is a nil value..? Even when I think I'm defining it correctly?

Freecam is a feature I'd really like, but I'm too tech-illiterate to wrap my head around how the lua script works, even with the instructions given on the github and in the script itself.. help?

You need a recent-ish build of the emulator.

Hey, what emulator build are you using, and how would I get this to work with a 9.12 emulator with decent speed and graphics?

https://desmume.org/download/
Scroll down to "Official Nightly Builds"

Last edited by windwakr (2019-04-19 23:05:45)

zeromus

Radical Ninja

Registered: 2009-01-05

Posts: 6,053

Re: freelook Lua script

Note that lua was kind of broken until a commit I did today, so you may have to get the build from appveyor instead

Bnthomason

Member

Registered: 2019-04-16

Posts: 9

Re: freelook Lua script

Total coding noob here... I'm trying wrap my head around how to use lua code, and I can't wrap my head around how to get the freelook script to work. I get that I actually have to adjust some things in the code myself, but I'm unclear about what values need changing, and no matter what I do the code stops running because register3devent is a nil value..? Even when I think I'm defining it correctly?

Freecam is a feature I'd really like, but I'm too tech-illiterate to wrap my head around how the lua script works, even with the instructions given on the github and in the script itself.. help?

You need a recent-ish build of the emulator.

Hey, what emulator build are you using, and how would I get this to work with a 9.12 emulator with decent speed and graphics?

https://desmume.org/download/
Scroll down to "Official Nightly Builds"

So, how would I reverse or change the controller assignments?

For example, in some games, when you push the L or R stick right, the camera pans in the opposite direction.

windwakr

Member

Registered: 2010-06-10

Posts: 22

Re: freelook Lua script

So, how would I reverse or change the controller assignments?

For example, in some games, when you push the L or R stick right, the camera pans in the opposite direction.

If both the up/down and left/right of the stick are inverted you can try using the dpad to set a negative scale. If you only need one axis inverted then you're on your own. The code's quite ugly but there are comments by the bits of code handling each button/stick. You should be able to slap(or remove) a '-' in there somewhere to invert the axis you need.

edit:
For example, to invert just the left/right of the left stick you'd change "MoveRight(-key.x * movscale);" to "MoveRight(key.x * movscale);". For up/down on the right stick you'd change "pitch = (pitch + (key.r * rotscale)) % 360;" to "pitch = (pitch + (-key.r * rotscale)) % 360;"

Last edited by windwakr (2019-04-21 18:37:52)

Bnthomason

Member

Registered: 2019-04-16

Posts: 9

Re: freelook Lua script

So, how would I reverse or change the controller assignments?

For example, in some games, when you push the L or R stick right, the camera pans in the opposite direction.

If both the up/down and left/right of the stick are inverted you can try using the dpad to set a negative scale. If you only need one axis inverted then you're on your own. The code's quite ugly but there are comments by the bits of code handling each button/stick. You should be able to slap(or remove) a '-' in there somewhere to invert the axis you need.

edit:
For example, to invert just the left/right of the left stick you'd change "MoveRight(-key.x * movscale);" to "MoveRight(key.x * movscale);". For up/down on the right stick you'd change "pitch = (pitch + (key.r * rotscale)) % 360;" to "pitch = (pitch + (-key.r * rotscale)) % 360;"

I figured that part out to invert just the x axis movement

xpos = xpos - fX

Now, the camera's also inverted (only the X axis (Left and Right))

I'm having an issue where when I simply input this:

function Rotate(angle,x,y,z)
    local s = math.sin(math.rad(angle));
    local c = math.cos(math.rad(angle));
    return {(1-c)*x*x+c

instead of

function Rotate(angle,x,y,z)
    local s = math.sin(math.rad(angle));
    local c = math.cos(math.rad(angle));
    return {(1-c)*x*x-c

The camera would flip and invert everything when looking up or down. Left and right works fine with this code.

I tried combinations of changing the other ones from - to + or + to -, but nothing seemed to work.

Then it clicked that if I invert every single one, it should possibly invert the camera look. So I tried this:

    return {(1-c)*x*x-c, (1-c)*y*x+s*z, (1-c)*z*x-s*y, 0,
            (1-c)*x*y-s*z, (1-c)*y*y-c, (1-c)*z*y+s*x, 0,
            (1-c)*x*z+s*y, (1-c)*y*z+s*x, (1-c)*z*z-c, 0,
            0, 0, 0, 1};

And all that did was make the camera collapse, expand, invert, etc whenever I turn the camera (Even the Z axis, but only slightly)

Last edited by Bnthomason (2019-04-21 18:54:53)

Bnthomason

Member

Registered: 2019-04-16

Posts: 9

Re: freelook Lua script

So, how would I reverse or change the controller assignments?

For example, in some games, when you push the L or R stick right, the camera pans in the opposite direction.

If both the up/down and left/right of the stick are inverted you can try using the dpad to set a negative scale. If you only need one axis inverted then you're on your own. The code's quite ugly but there are comments by the bits of code handling each button/stick. You should be able to slap(or remove) a '-' in there somewhere to invert the axis you need.

edit:
For example, to invert just the left/right of the left stick you'd change "MoveRight(-key.x * movscale);" to "MoveRight(key.x * movscale);". For up/down on the right stick you'd change "pitch = (pitch + (key.r * rotscale)) % 360;" to "pitch = (pitch + (-key.r * rotscale)) % 360;"

Hey, so... how would I input keyboard values in addition to the Joystick?

For example, I would like the NUMKEYs to have certain positions.

I believe what I need to do first is add below joyID something like Keyboard?

Here's what I have so far:

KeyboardID = -1; --Keyboard ID to use. The script will attempt to find a suitable Keyboard, but you can manually set this if you wish

...

while true do
    local key = input.get(KeyboardID)

Haven't had a chance to test this yet.

What are your thoughts or suggestions to make this work?

EDIT: I'm so close to figuring it out.

local table = {-1};
if table[1] == -1 then
   for k = 0, 15 do
       k = input.get(k);
         KeyID = k;
         print("Found suitable keyboard, id ", k);
         break;
    end
else
    print ("Using manually set keyboard, id ", KeyID);
end
if KeyID == -1 then
    print("No suitable keyboard found!");
    while true do
        emu.frameadvance();
    end
end

Last edited by Bnthomason (2019-05-02 07:01:47)

Bnthomason

Member

Registered: 2019-04-16

Posts: 9

Re: freelook Lua script

local table = {-1};
cooldown = 0;
if table[1] == -1 then
   for k = 0, 15 do
       k = input.get(k);
         KeyID = k;
         print("Found suitable keyboard, id ", k);
         break;
    end
else
    print ("Using manually set keyboard, id ", KeyID);
end
if KeyID == -1 then
    print("No suitable keyboard found!");
    while true do
        emu.frameadvance();
    end
end
while true do
    local key = input.get(KeyID)
    if key["0"] and cooldown==0 then --0 Key
        cooldown = 20;
        print("0 key");
    end
    if key["numpad0"] and cooldown==0 then --numpad 0 Key
        cooldown = 20;
        print("numpad 0 key");
    end
    if key["numpad+"] and cooldown==0 then --numpad + Key
        cooldown = 20;
        print("numpad+ key");
    end
    if key["A"] and cooldown==0 then --A Key
        cooldown = 20;
        print("A key");
    end
    if key["backspace"] and cooldown==0 then --backspace Key
        cooldown = 20;
        print("backspace key");
    end
    if key["space"] and cooldown==0 then --spacebar Key
        cooldown = 20;
        print("spacebar key");
    end
    if cooldown>0 then cooldown=cooldown-1 end
    emu.frameadvance();
end

-- A full list of keys can be found here: http://tasvideos.org/LuaScripting/TableKeys.html

Case Solved

Sokaren

Member

Registered: 2019-07-02

Posts: 1

Re: freelook Lua script

Can some one show me how this works I'm lost and have bounced between the issue page and here to no avail of it working on any game.

zeromus

Radical Ninja

Registered: 2009-01-05

Posts: 6,053

Re: freelook Lua script

Right now it's just a toolkit for knowledgeable people. So, no.

RingoArgonaut

Member

Registered: 2020-03-16

Posts: 2

Re: freelook Lua script

how could I control the camera on a different layer?

zeromus

Radical Ninja

Registered: 2009-01-05

Posts: 6,053

Re: freelook Lua script

3d games typically do not have multiple "layers", so your question makes no sense to me

RingoArgonaut

Member

Registered: 2020-03-16

Posts: 2

Re: freelook Lua script

Like in Desmume under tools and view layers, in some games I can only control the camera on one layer. Sorry if this is a stupid question.

zeromus

Radical Ninja

Registered: 2009-01-05

Posts: 6,053

Re: freelook Lua script

those are 2d layers. there's no way to do that, and those layers are smaller than most game worlds, so most games are continually updating the smaller layer to make it seem infinite, so that there is no point to moving the camera over them since you won't see anything but the edge of the updated area

ARMarioBroz7

Member

Registered: 2023-03-28

Posts: 1

Re: freelook Lua script

local table = {-1};
cooldown = 0;
if table[1] == -1 then
   for k = 0, 15 do
       k = input.get(k);
         KeyID = k;
         print("Found suitable keyboard, id ", k);
         break;
    end
else
    print ("Using manually set keyboard, id ", KeyID);
end
if KeyID == -1 then
    print("No suitable keyboard found!");
    while true do
        emu.frameadvance();
    end
end
while true do
    local key = input.get(KeyID)
    if key["0"] and cooldown==0 then --0 Key
        cooldown = 20;
        print("0 key");
    end
    if key["numpad0"] and cooldown==0 then --numpad 0 Key
        cooldown = 20;
        print("numpad 0 key");
    end
    if key["numpad+"] and cooldown==0 then --numpad + Key
        cooldown = 20;
        print("numpad+ key");
    end
    if key["A"] and cooldown==0 then --A Key
        cooldown = 20;
        print("A key");
    end
    if key["backspace"] and cooldown==0 then --backspace Key
        cooldown = 20;
        print("backspace key");
    end
    if key["space"] and cooldown==0 then --spacebar Key
        cooldown = 20;
        print("spacebar key");
    end
    if cooldown>0 then cooldown=cooldown-1 end
    emu.frameadvance();
end

-- A full list of keys can be found here: http://tasvideos.org/LuaScripting/TableKeys.html

Case Solved

ik this was made for a keyboard how can you make the controller lua script work with this one? I want to be able to controll the camera with using a keyboard.