Lua Scripting (Mesen)

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Lua Scripting (Mesen)

Post by Sour »

I'm planning on adding LUA scripting into Mesen's debugger over the next few days and before I started I wanted to get some input about what people would need out of it. I thought about reimplementing FCEUX's API, but at the moment I think that I might be best off creating a new API in whatever way makes the most sense from Mesen's point of view. I asked rainwarrior for his opinion on this and here's what he replied:
rainwarrior wrote:I actually don't think FCEUX's lua API is very good. It's very useful, but only because it is barely good enough. It's really very hodge podge, various people have added some minimal things they needed over the years, and at this point it's a fairly useful base set.

Here's features I think are useful. FCEUX has many of these, but not all.

callback in response to read or write (ranged), should contain the address and value to be read/written as parameters.
callback in response to Power, Reset, NMI, and IRQ.
peek at all memory (CPU, PPU, OAM) without triggering any read side effect.
peek at PPU/APU state (latch, flags, pixel, scanline, frequency, etc.), peek_state(enum) would allow you to add to the interface just by adding more enum values, rather than a new function for every piece of state?
write to all memory with side effects (e.g. bank switch register writes).
write to all memory without side effects.
printf style text overlay (option for either an X,Y position, or a maintained cursor position)
line and box drawing overlay
overlay should have a parameter to last 1 frame, a specified number of frames, or until "cls"
Other than reading registers (e.g PPU/Cartridge registers) with no side-effects, all of this should be relatively easy to implement.

Does anyone have other ideas that they'd like to be included in the API?
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Re: Lua Scripting (Mesen)

Post by thefox »

Did you give any consideration to other scripting languages, like Python?
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Lua Scripting (Mesen)

Post by tepples »

Python might help people who feel turned off by some of Lua's quirks, such as 1-based indexing. If you choose to embed Python in your app, Python C Foreign Function Interface (CFFI) covers the "extending" (main function in Python) and "embedding" (main function in C++) cases.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Lua Scripting (Mesen)

Post by rainwarrior »

Sour wrote:Other than reading registers (e.g PPU/Cartridge registers) with no side-effects, all of this should be relatively easy to implement.
For the most part this kind of thing can be worked around by duplicating the registers in Lua and snooping the relevant writes. (FCEUX scripts do this frequently.) It's cumbersome and inefficient, though.

It's kind of weird that FCEUX has "sound.get()" or "zapper.read()" but no access to the PPU internal state. There's a lot of stuff it's missing. I suggested a generic "peek_state" interface that takes an enum just because that's probably really easy to add things to gradually as you need them or have time to implement, rather than implementing a PPU reading API, a controller reading API, an APU reading API, etc... but however you want to organize it, access to the internal state of things is often very useful.

Ideally, every mapper implementation should have its own couple of enums for peek/peek_state too. If you have savestates, though, all the data you'd need to peek at should be present in that savestate already, so maybe that would make it slightly easier to organize the implementation?
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Lua Scripting (Mesen)

Post by Sour »

I think the biggest problems with Python are its size & dependencies. The source code is ~18mb, which is a number of times larger than Mesen itself. In comparison, LUA is pretty small (~600kb), which is a lot more reasonable. The external dependencies (which LUA has very little of (none?), as far as I can tell) also make Python harder to integrate.
rainwarrior wrote:I suggested a generic "peek_state" interface that takes an enum just because that's probably really easy to add things to gradually as you need them or have time to implement, rather than implementing a PPU reading API, a controller reading API, an APU reading API, etc...
At the moment, internally, the debugger gets the emulator's state via a GetState function which returns a struct of structs, one for each component (CPU, PPU, Cartridge - no APU/Controller as of right now, but easy to add). These contain the majority of registers/flags/etc but more often than not as an object rather than the actual register's value (e.g the PPU returns flags as actual booleans instead of a single byte encoding multiple flags like the registers). I figure a scripting function that returns the same structure would be good enough?
zzo38
Posts: 1096
Joined: Mon Feb 07, 2011 12:46 pm

Re: Lua Scripting (Mesen)

Post by zzo38 »

Does Lua have typed arrays like JavaScript now has? Because, it seem that would help if you want to access raw (rather than mapped) RAM and ROM data (this also avoids side effects, because it isn't mapped).
(Free Hero Mesh - FOSS puzzle game engine)
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Lua Scripting (Mesen)

Post by Sour »

I'm not overly familiar of what LUA offers specifically (I've only used it briefly a long time ago).
Either way, the API would not allow raw access to the ram/rom memory - you'd need to use read/write functions (otherwise it becomes too easy for a badly written script to crash the entire emulator)

That being said, you made me realize that JS would also be a potential candidate here. The lightweight JS engines I found only implemented EMCAScript 5.1 (which is 5+ years old by now), though. One of them was implemented as just 2 files though (.c+.h), which is nice.
I'd imagine more people are familiar with JS than LUA (I know I am) - but I'm unsure which one would be easier to work with from the point of view of someone who isn't too familiar with coding in general.
Pokun
Posts: 2681
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Lua Scripting (Mesen)

Post by Pokun »

Yeah isn't JS like one of the most used languages in the world due to it's use in web programming? And since JS is so easy to setup (you only need a standard web browser) it's very useful in general. I thought about using it to prototype my NES code.

I've only seen Lua as a scripting language in emulators and such, and have never come across it anywhere else.
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Re: Lua Scripting (Mesen)

Post by thefox »

Sour wrote:I think the biggest problems with Python are its size & dependencies. The source code is ~18mb, which is a number of times larger than Mesen itself. In comparison, LUA is pretty small (~600kb), which is a lot more reasonable. The external dependencies (which LUA has very little of (none?), as far as I can tell) also make Python harder to integrate.
Yeah, I agree. I kind of want to experiment with Python integration to see what are its runtime costs, because I like it as a language quite a bit more than Lua.

JavaScript is not a bad choice, either. Lua (as a language) is very similar. Could even support both, but fragmenting the (small) user base might be undesirable.
Pokun wrote:I've only seen Lua as a scripting language in emulators and such, and have never come across it anywhere else.
It's somewhat popular in gamedev circles. E.g., World of Warcraft and Crysis (CryEngine) use it as a scripting language. (Looks like there's a list at Wikipedia).
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Lua Scripting (Mesen)

Post by rainwarrior »

Lua is by far the most popular scripting langauge for games. In stuff I've worked on or used, I've seen either Lua or a custom language specific to that game most often. Python I've seen several times as well. Once or twice I've seen a version of Lisp.

The most useful thing is just having a scripting language. The advantages of this one or that one are overall kind of minor in utility. Even differences in efficiency, where they exist, tend to be minimized by the bulk of the work being done in native code anyway; generally the scripting language is not expected to be doing heavy computation.

Lua is fairly convenient to integrate into other things, since it was designed for just that purpose. There are other scripting languages designed for this purpose, too. Python was not designed with that in mind, but it's proven pretty usable as a scripting language anyway.
Sour
Posts: 891
Joined: Sun Feb 07, 2016 6:16 pm

Re: Lua Scripting (Mesen)

Post by Sour »

I've mostly settled on LUA for now - after spending a few hours messing around with its API yesterday, I finally know how to do just about everything I need to.
Once I've got the LUA API worked out, I'll probably take a look at how hard it would be to create (& maintain) an identical JS API - allowing users to pick whichever they are the most familiar with.
thefox wrote:because I like it as a language quite a bit more than Lua.
I'm not too much of a LUA fan myself (though I've only used it for a short period ~9 years or so ago in a WoW addon). I haven't really used Python much either though (pretty much only the handful of times I had the need to edit existing scripts) - C# has enough built-in functionality for pretty much any script/utility I ever need to make for myself, so I haven't really had any incentive to look into anything else.
Pokun
Posts: 2681
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Lua Scripting (Mesen)

Post by Pokun »

Scripting in games? I guess that would be mostly computer games. No wonder I haven't run into it. I could probably count the number of modern computer games I've played recently on both my hands.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Lua Scripting (Mesen)

Post by rainwarrior »

Pokun wrote:Scripting in games? I guess that would be mostly computer games. No wonder I haven't run into it. I could probably count the number of modern computer games I've played recently on both my hands.
What does "computer" mean here? PC?

Scripting is used pretty widely in modern games on console as well, has been a part of the game ecosystem for a long while, really. I think Quake's "QuakeC" popularized the idea, but it goes back earlier than that. I'd say it was common as far back as PS1, but there's examples like the bytecode used in Koei's NES games that look an awful lot like a scripting language too.
Pokun
Posts: 2681
Joined: Tue May 28, 2013 5:49 am
Location: Hokkaido, Japan

Re: Lua Scripting (Mesen)

Post by Pokun »

Yeah by computer I mean any video game capable machine that isn't considered a console or arcade machine, so a PC I guess. And by scripting I thought it was more about user editor that allows advanced scripting, which would normally require a typical PC environment with keyboard and therefore be scarce on consoles, though not non-existent.

I don't mean I haven't run into scripting in games, but haven't run into Lua a lot.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Lua Scripting (Mesen)

Post by tokumaru »

Don't forget SCUMM, which was used in many of LucasArts point-and-click adventure games. The NES version of Maniac Mansion apparently runs a version of it, but Nintendo didn't like to see it listed as "NES SCUMM" in the credits.
Post Reply