You are not logged in.

Read the FAQ and Knowledge Base before posting.
We won't make a 3DS/2DS emulator.



#1 2017-07-13 16:10:48

bingbingBOPOMOFO
Member
Registered: 2017-07-13
Posts: 4

Asian Languages in filenames break DeSuME

I am currently working on a project with roms that have been modified to be localized into Chinese. I found out that the most popular DS emulators on PC: DeSmuME, RetroArch and no$GBA all have the same issue. I am on 0.9.11 and when attempting to load the ROM it is unable to be loaded if the filename has ANY Chinese Characters in it, regardless of the Chinese character(s) position. I can also confirm if there is any Hangul (Korean) within the file name it has the same issue. Hopefully next patch this can be resolved.

If you want to recreate this scenario, simply copy-paste this "你好“ to the file name of any .nds rom and it won't load at all.

bingbingBOPOMOFO

Last edited by bingbingBOPOMOFO (2017-07-13 16:18:54)

Offline

#2 2017-07-13 17:21:08

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Asian Languages in filenames break DeSuME

this isn't likely to be resolved due to a combination of multiple layers of crusty old filesystem code made by multiple programmers who arent around anymore, paths getting copied about for save files and savestates, who knows what going on with the dearchiving layer, and all of everything related to non-ascii paths being a colossal cock-up on windows. Don't count on it. Better rename your roms.

Offline

#3 2017-07-13 22:32:18

bingbingBOPOMOFO
Member
Registered: 2017-07-13
Posts: 4

Re: Asian Languages in filenames break DeSuME

zeromus wrote:

this isn't likely to be resolved due to a combination of multiple layers of crusty old filesystem code made by multiple programmers who arent around anymore, paths getting copied about for save files and savestates, who knows what going on with the dearchiving layer, and all of everything related to non-ascii paths being a colossal cock-up on windows. Don't count on it. Better rename your roms.

That's understandable. I thought it was still important to take note of this issue. Thanks for the help.

A Reddit user and apparent Citra dev, yuriks, helped explain the issue more on a technical level
"These kinds of issues are usually unique to Windows. Unix systems usually just use UTF-8 encoded strings as filenames in all APIs. Since UTF-8 is compatible with ASCII strings there doesn't need to be any special support as long as you're not mangling the string somehow. (Usually you'll just be passing them through from a Open dialog or the commandline to your open file function.)
Windows however has made several (bad, but historically understandable, imo) decisions which mean that there are two separate APIs for any operation involving strings or filenames: One operates only on the local code page (Usually ISO 8859-1/latin-1 for most people in western countries.) and doesn't support any foreign characters not in the codepage. The other one operates using UTF-16, which is not compatible with regular ASCII strings. This means that unless you have taken special precautions (such as transcoding your internal UTF-8 strings to UTF-16 and then using the appropriate wide API, or using a library that will do that for you) Windows programs will not work with international characters by default."

https://www.reddit.com/r/emulation/comm … l_popular/

Offline

#4 2017-07-14 00:28:08

zeromus
Radical Ninja
Registered: 2009-01-05
Posts: 6,169

Re: Asian Languages in filenames break DeSuME

More to the point, windows programs will not work with international characters unless you make a dedicated effort, and maintain that effort indefinitely by having a developer who uses international characters consistently during coding and testing to ensure that there are no regressions, since the support is always frail--instead of a developer who considers use of international characters to be user error in every case.

Offline

#5 2017-07-14 11:51:41

bingbingBOPOMOFO
Member
Registered: 2017-07-13
Posts: 4

Re: Asian Languages in filenames break DeSuME

zeromus wrote:

More to the point, windows programs will not work with international characters unless you make a dedicated effort, and maintain that effort indefinitely by having a developer who uses international characters consistently during coding and testing to ensure that there are no regressions, since the support is always frail--instead of a developer who considers use of international characters to be user error in every case.

Completely understandable. I have found an easy fix however if someone wants to use Chinese in their file names on Windows that have to change their system local. This doesn't actually change the language of the operating system either.

Go to the start menu, type region, and click region.
7118e4e0f9d37145c3c746b78349bf0f.png

Click on administrative tab then change system locale button
d073d07ba662e5163e90eb230f70192d.png

Then from the drop-down menu, select the desired language. **In this case, we want to choose Chinese (Simplified, China) as it will allow us to read both simplified and traditional characters.**
bdfce62695d962844e39d9f3fbf15ef6.png

Press ok
50d7c4f138880ec26e818aad0be67854.png

Restart your computer
f0c0614429727c965bdbedfbef4fccd1.png

Offline

Board footer

Powered by FluxBB