You are not logged in.

Read the FAQ and Knowledge Base before posting.
We won't make a 3DS/2DS emulator.



#1 2014-02-18 16:25:00

hcf
Member
Registered: 2014-02-18
Posts: 23

Xbox port

Hi! First of all, congratulations for the amazing work made in Desmume!

I only wanted to let you know that we are creating a port of Desmume for Xbox (not Xbox 360 or Xbox one, but the original Xbox), so I hope that soon your awesome emulator will be enjoyed in another plattform smile

In fact, in December we released the first version of Desmume for Xbox, nicknamed "DesmumeX v0.2". It is still not very polished, but some games are already playable at full speed, like "Yoshi Island", "Zoo Keeper", "Chicken Shoot", and some more. There are some videos in Youtube where people can see the emulator working, if someone is interested.

Of course we know that some 3D games will never run at full speed, due to the limitations of the Xbox hardware (its power is, more or less, similar to the Wii), but we are still working hard and we have achieved important improvements since that v0.2 version, so we hope to make a new release soon.

Now, we are including the dynamic compiler (JIT), and we hope to have a nice boost when this work is done. Also, we have updated a lot of code to the last core of Desmume, to increase the compatibility.

At the moment we are having a small problem with the JIT: our emulator seems to be in an infinite loop, repeating forever the same string of compiled functions. The original Desmume seems to repeat those instructions for a while, but then it exits from the loop, so we are now trying to discover what we are doing bad. If this is fixed, we hope to do a new release soon.

As we are gathering more experience in DesmumeX, we would be glad if we could help also in the Desmume core too, if our help could be usefull. As Xbox hardware is limited, we are allways trying to discover optimization tricks, and maybe some of these tricks could be also usefull in Desmume for PC. Do you use the IRC channel to discuss about possible improvements?

Finally, I want to say (in my name and the other people of the xbox scene related to this project): thank you very much for this awesome emulator! We hope to bring soon an Xbox port able to run much more games.

Cheers

HCF

Offline

#2 2014-02-18 18:43:08

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Xbox port

#desmume on freenode. yes, we use that, primarily, depending on what's going on.

I look forward to seeing your changes. Keep in mind we have somewhat different goals than you, however. You're interested in doing whatever it takes to get fun, and we're interested in maintaining a reference emulator. We don't like obfuscating clean code (not saying the code is clean to begin with really) with speedhacks.

Regarding the bug you're dealing with, check which registers it's polling, or which variable's its polling, and then debug a trace of the game to see how that variable gets set. If it's a register in question, you can skip that step and go straight to finding out why that hardware isn't running from within the jit loop.

Offline

#3 2014-02-19 12:54:32

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

Thank you very much for your kind answer! smile

You are right, at the moment we have different goals, because we are now mainly focusing in optimizing the speed, while you are trying to make the most reliable emulator. Anyways, if we find any bug or any chance of improving the core, we would be very glad if we could chat about it and help in your project. It's obvious that our knowledge of Desmume is still very far from yours, but well... last year we reported a small bug that we found in Virtual Jaguar (another emulator that we have ported to Xbox). If we find any improvement that is not going to obfuscate the code, we will tell you wink

Of course, if you want we can inform about our releases. Is this forum a good site to do it, or is it better to leave this forum for other things?

As far as the bug is related, thank you very much for your advice. We will try to do something similar to what you say, to try to see what is happening. We cannot do it exactly as you said, because the compiled functions are hard to debug. I mean, in the function "armcpu_exec" there are these lines:

ArmOpCompiled f = (ArmOpCompiled)JIT_COMPILED_FUNC(ARMPROC.instruct_adr, PROCNUM);
return f ? f() : arm_jit_compile<PROCNUM>();

We can debug easily when the program goes into the "arm_jit_compile" method, we can run step-by-step and so on... but unfortunately, when the program  goes into the compiled "f()" function, the debugger does not go into it.

Our problem is that, after running a lot of instructions well (both in the "arm_jit_compile" and the "f()" branches), our emulator begins to repeat the same 4 or 6 "f()" functions in a loop. The official Desmume repeats the same "f()" functions for a while, but then suddenly it goes into an "arm_jit_compile" and breaks the loop.

Your idea is very usefull, thank you very much: it is probably a loop polling for a value (variable or registry), so we must investigate which value is it and why we are not updating it. The difficult thing is that we cannot enter into the "f()" functions while debugging, so it's hard to know what are those functions polling smile

Thanks for all! And keep on with your awesome work!

Offline

#4 2014-02-19 15:07:19

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Xbox port

sure, you can announce it anywhere on this forum when you make your release.

you should be able to know when it is about to get hung up by checking the instruct_adr. you could switch to the interpreter at that point and study what the game is doing (there are debug printf statements you can enable to get disassembly) and get a clue about why it might be looping through that code

Offline

#5 2014-02-19 15:30:16

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

Thank you very much, we will try to use that trick, and I hope that soon we would be able to announce a new release here! smile

Offline

#6 2014-02-19 15:53:47

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Xbox port

where did you post v0.1, or, where are you hosting the sources?

Offline

#7 2014-02-19 16:39:47

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

Maybe I explained it bad: the first version ever released was "DesmumeX v0.2" (at the moment, we have not released any other one). The "v0.1" was not released, it was only the first beta version that the beta testers used, to help us to find bugs and playable games.

When we make a release, we include the source code in the same ZIP file. In this way we can be sure that we are GPL complaint, as the sources and the binaries are allways together. The emulator has been released in some pages related to the Xbox scene like www.emuxtras.net and after that, I see that some people has uploaded it to many other sites (searching "DesmumeX" in Google, we can see several sites where DesmumeX has been uploaded by other people). As they upload the same file, the source code should be allways inside of the same ZIP.

Anyways, if you want, we can do it in a different way in later releases. For example, if you want that we upload our sources to some github or sourceforge page, or even give you the sources so that you can publish them on the Desmume official page, let me know and of course we will do it.

Besides, let me tell you that the first release was based on an old core (Desmume 0.9) as it was only a quick port made as a proof of concept to see the performance of Desmume in Xbox. So, if you look at the source code of the v0.2, don't be upset by the fact that it is old code... Now, as we have seen that Desmume can work at decent speed in the Xbox, we are updating a lot of code to the last Desmume core (v0.9.10), so next releases  will have a source code more "similar" to the main Desmume project wink

Offline

#8 2014-02-19 17:42:51

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Xbox port

we wont be publishing it here. i was just curious where you were working on it.

Offline

#9 2014-03-02 17:25:32

OldButGold
Member
Registered: 2014-03-02
Posts: 3

Re: Xbox port

Nice news

i may try to port it on xbox 360
but i need source codes, i could not find

Offline

#10 2014-03-03 11:01:23

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

OldButGold wrote:

Nice news

i may try to port it on xbox 360
but i need source codes, i could not find

Searching "desmumex v0.2" in Google, you should find several sites were you can download the emulator, and the source code is inside of the same ZIP. If you still have troubles to find the sources, let me know!

This early release of the Xbox port (v0.2) is based entirely on SDL, so it should be easy to port to Xbox 360 or any other system.
smile

Offline

#11 2014-03-03 13:32:56

OldButGold
Member
Registered: 2014-03-02
Posts: 3

Re: Xbox port

Thanks

Yoshihiro (unknown) already made DSon360 v0.2
also cant find any source code of that AND DeSmuME 0.9.10

Offline

#12 2014-03-03 13:40:21

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Xbox port

if you cant even find the desmume 0.9.10 source code then you are going to have serious trouble doing anything with it

Offline

#13 2014-03-03 21:50:59

OldButGold
Member
Registered: 2014-03-02
Posts: 3

Re: Xbox port

i expect githubs wink
didn't download anything yet

and im not the one who will port it really, just give it to me

Offline

#14 2014-03-04 03:12:59

ff11
Member
Registered: 2014-01-04
Posts: 20

Re: Xbox port

OldButGold wrote:

i expect githubs wink
didn't download anything yet

and im not the one who will port it really, just give it to me

If you want the github, have a mirror here: https://github.com/mirror/desmume
But on the sourceforge, you can get the SVN here: http://sourceforge.net/p/desmume/code/HEAD/tree/

It's a public thing that all dev should know, unless he is living inside some cave by at least 14 years.

Offline

#15 2014-08-20 11:12:19

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

After some troubles, I'm glad to tell that we have recently released another version of the Xbox port of desmume: DesmumeX v0.5. The main change in this new version is that we have included the main part of the dynamic recompiler (around 75%), so now emulation is a lot faster, as you know. Later in this post I will explain why I say that we have ported 75% of it smile

In our first release, we ported an old version similar to Desmume v0.9 (our work started from the PSP port, and we add a lot of things to it, to make it similar to the v0.9 version). In this new release, at first we were trying to upgrade the core to the last one, plus inserting the dynamic recompiler. Unfortunately, we faced huge troubles when trying to do everything at the same time (there are huge changes between our old core and the newest one...), so now we are doing this step by step: in this release, we kept the old core and we have simply inserted the dynamic recompiler, to understand how it works. And in next releases we will go for the core upgrades.

Maybe you are wondering how is possible that we have ported "the main part of the dynamic recompiler (around 75%)", because the normal thing is porting it all wink I will try to explain it.

Of course we have included in our project all the AsmJit code (modifying all the things not supported in Xbox), but after that, the arm_jit.cpp and arm_jit.h files were the following step. But now we faced a problem: we are using an old core with and old MMU and an old set of instructions (arm_instructions.cpp) but the arm_jit.cpp has the equivalent assembler code to the instructions of the NEW cores. So, when we tried to insert the new arm_jit.cpp in the old core, games didn't work. Of course, we could re-do all the assembler code to be similar to the old instructions, but we decided that this was a huge non-sense effort (sooner or later, we must upgrade the core!) so we did the following:

We saw that the main part of the arm_jit instructions DO WORK with our old core, and only around 25% of the instructions make things crash. So,we filled a boolean array telling if each instruction is valid to be executed with the JIT code, or not. If it is valid, it is executed with the arm_jit code, and if it is not valid, we break the dynamically recompiled block (the same thing that we do with branch instructions) and run it from the arm_instructions.cpp file.

Of course this is not optimal, because we are not running JIT in around 25% of the instructions, and also we are everytime questioning "is this instruction valid?" and that make things slow in the compile_basicblock function. But once we have done the initial compilation, the block is executed quickly all the times that it is called, so we have got a huge boost! And this boost will be much better whenever we managed to make run the rest of the JIT instructions.

To complete this explanation, I will say that the instructions that we are NOT executing in JIT (because they make our games crash) are basically the OP_STM... and OP_LDM... , and also the OP_B, OP_BL, OP_BX, OP_BLX_REG... Now we are investigating if the error is in the assembler code used in arm_jit.cpp (maybe not compatible with our old core) or maybe we need to add some codeline like this in other places:

JIT_COMPILED_FUNC_KNOWNBANK(adr, MAIN_MEM, ARMCPU_ARM9, 0) = 0;

We think that we have copied that codeline in all the places where memory is written, but maybe we have overlooked some think. We are considering this due to the fact that the failing instructions are mainly memory accesses (OP_LDM, OP_STM...).

As far as other changes in this version, we have also fixed an annoying bug inherited from the PSP port, that made every game crash after 10 or 15 minutes of playing. And also, we have converted the basic blittering code from SDL to native Xbox Direct3D code, and that has given us another nice boost.

Speaking about Direct3D (it is the system that Xbox natively supports) we have discovered that some time ago, a Japanese coder started a Direct3D version of Desmume, converting some of the funcionality that we are doing now by software (in gfx3d.cpp) to Direct3D. We are thinking about studying what he did, because maybe we can re-use some of his work to boost things in Xbox (although it seems that his work was not completed).

I know that you are more interested in accuracy and compatibility, while we are now targeting to performance (we need to do it, because the original Xbox is not a very powerfull system) so we have slightly different goals than the original Desmume project, but if we find some interesting optimization in the future, we will post it in these forums. Helping this marvelous project to keep growing will be amazing.

For example, a friend told me that maybe the dynamic recompiler could be optimized a bit, because as is now, we are breaking the dynamically recompiled block every time that we arrive to a branch instruction, and he told me that, if the jump is made to an address that has another dynamically compiled block of code, maybe we could join both code blocks. Honestly, I only understood this slightly, but if we finally make this and it works (and it is noticeably faster) I will tell you.

Finally, as far as the general status of the emulator, I can say that now many 2D games are playable (games like Yoshi Island work even with frameskip=0), you know: New Zealand Revolution, DK-Jungle Climber, Zoo Keeper, and so on... and we hope that every 2D game will be playable when the rest of the dynamic recompiler is working. Unfortunately, 3D games work painfully slow (with only a few exceptions) and that is why we thought about ways of boosting it, like using that Direct3D hardware accelerated code that I said, or maybe removing some functionalities from the 3D (fogs...).

Please, keep up with your awesome work. We will keep you updated about every new release that we make of the original Xbox port smile

Offline

#16 2015-08-05 09:01:39

hcf
Member
Registered: 2014-02-18
Posts: 23

Re: Xbox port

A new version of DesmumeX (the Xbox port of Desmume) has been released: DesmumeX v0.8.

The most important change is that we have finally upgraded de core of the emulator to the last Desmume release (0.9.11). This has increased the compatibility a lot, and also the speed has improved (the dynamic recompiler is running very well).

In the future, we hope to keep the Xbox port always updated to the last Desmume version if possible.

Once again, I want to say huge thanks to the people involved in Desmume, and people who helped in the Xbox port development. Together, we have achieved something that seemed like a dream a couple of years ago: emulating the DS in the old Xbox big_smile

A lot of 2D games run very well (games like Henry Hatsworth or Yoshi Island run at full speed!) with some exceptions like the Castlevania ones (unfortunately, they still run slow). 3D games are still slow, but some of them like New Super Mario Bros give us around 45 FPS during the gameplay, so I still hope to play them in the future smile

Offline

Board footer

Powered by FluxBB