Snes emulation image processing
Moderator: Moderators
Forum rules
- For making cartridges of your Super NES games, see Reproduction.
Re: Snes emulation image processing
I feel like everyone is forgetting that the SNES has a 15-bit RGB (as in non-indexed) colorspace and that this whole colorspace is used for color math. So you can't really apply color indexing optimizations here unless you want really big lookup tables (For any single CGRAM state, theoretically up to 65536 entries, with many of them regenerated every time there's a palette change, and even then you'd basically just be caching addition which will probably give you very limited gains if any).
SNES NTSC 2/1/3 1CHIP | serial number UN318588627
Re: Snes emulation image processing
ie: twice the size of the master palette...HihiDanni wrote:theoretically up to 65536 entries
Good point. SNES hardware color math doesn't involve multiplication; it's just addition or subtraction with an optional right shift (basically - I'm not sure what the exact hardware implementation of the division by two is). It's done once per pixel, as the PPU is generating the signal to send to the TV, and the result is not stored.addition
Software color math on SNES is (I assume) much rarer and mostly done with precomputed palettes stored in ROM, like the day/night idea mentioned earlier.
The screen brightness register is a whole other matter; I believe the function is actually analog, operating on the post-DAC video signal. Of course it's only ever applied once per pixel, and if you're willing to use sixteen 96 KB palette tables you could skip it...
Re: Snes emulation image processing
I do not understand why pre-computing a 256x256 palette for colour math would be an optimisation over computing 256x240 screen pixels and applying colour math to them. This would be an optimisation in hi-res mode but I'm fairly sure the SNES can't do colour math (at least not as intended) when it is in hi-res mode.HihiDanni wrote:I feel like everyone is forgetting that the SNES has a 15-bit RGB (as in non-indexed) colorspace and that this whole colorspace is used for color math. So you can't really apply color indexing optimizations here unless you want really big lookup tables (For any single CGRAM state, theoretically up to 65536 entries, with many of them regenerated every time there's a palette change, and even then you'd basically just be caching addition which will probably give you very limited gains if any).
The fact colour math is potentially CPU intensive might be why SNES9x can disable them optionally, in older computer it might have been necessary to do that in order to run games at full speed.
-
- Posts: 611
- Joined: Mon Jan 23, 2006 7:47 am
- Location: Germany
- Contact:
Re: Snes emulation image processing
They just emulate the graphics pipeline of the emulated system.OneQuestionPlease wrote:how do emulators process images to end up being that fast?
The day/night cycle in SD3 is simply done via palette changes. Take a look at the palettes in these pictures: day, night
All that happened is that the colors used by certain backgrounds and sprites are darkened.
SNES screens can use up to 4 background layers, plus one sprite layer (whose pixels are individually combined with the other layers). Here are the ones used by SD3: BG1, BG2, BG3, OBJ
For each pixel on the screen, the hardware (or the emulator) determines which tile is used from each background and which pixels in these tiles are used. SD3 uses graphics mode 1, in which BG1 and BG2 have 16 colors per tile and BG3 has 4 colors per tile. If the tile data for a tile's pixel is zero, it is transparent and another layer's pixel is used.
This process is done twice, resulting in two screens. Only now is the tile data converted to actual colors (using it as indices into the global 256-color palette, in which every entry is a 15-bit color). The generated screens can be added (or subtracted) if necessary. This is how layers that are enabled only in one screen can appear transparent. (But SD3 doesn't use color math for this effect in the pictures above.)
So, in summary: the hardware/emulator does very simple operations on a few bytes. The real magic is how the game programmers used these facilities to create impressive effects.
Until very recently, working with floating-point numbers was much slower than with integer numbers.OneQuestionPlease wrote:Well, that makes me respect emu-devs more. I need to have a look at how they accomplish that.tepples wrote:Emulators don't use floating-point math, for one thing.
Unfortunately it has only 16 steps. (In the scene above, SD3 always keeps this register at the maximum brightness.)adam_smasher wrote:The SNES has a screen brightness register; that's what most games would probably used to darken the screen.
Emulators do not go through the picture looking for colors to change - the games simply generate slightly different source data (different palette entries) and the hardware/emulator generates a new screen from that initial data.OneQuestionPlease wrote:i count the number of colors on the picture above and is roughly 100, very low and common in pixel art. Having to process just 100 is a lot faster than computing each pixel. But thinking about it, might be nightmarish to apply transparency, isn't it? I imagine the workflow to be:[...] Now, if you have to take into account the new colors produced by alpha blending, having to locate every new color (probably it didn't exist before) can be a slow operation.Code: Select all
for each color in palette2: if it exists in palette1: update index in image 2 else: add new color and update index in image 2 for each pixel in image2: copy index to image1
Also, does the snes employ a different palette for every sprite/tile?
You can read more about how the SNES renders the screen in this document (among others).
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
- OneQuestionPlease
- Posts: 6
- Joined: Mon Aug 21, 2017 4:47 am
Re: Snes emulation image processing
I'm glad everybody is taking the time to give thoroughly answers, including links and pictures. I'm learning immensenly.
By the way, this thread improved my code, mostly thanks to avoiding floats at some places and processing unique colors and then replacing rather than processing the whole image. Also gave me a closer insight on how the snes works., Sometimes technical documents are too technical. But i'll have a glance at these links, the blending formulas are simple yet effective judging by the results.
tepples wrote:The S-PPU has 8 palettes for backgrounds and 8 palettes for sprites. A palette can be assigned to each 8x8 pixel tile or each sprite. Blending on the S-PPU uses the formula (r1+r2), (r1-r2), or (r1+r2)/2
But i was under the impression the snes actually supported different grades of transparency, for example https://youtu.be/XNQm9idpB1M?t=5m48s (clouds) or https://youtu.be/986vcULjMq8?t=25s (Boss).93143 wrote:SNES hardware color math doesn't involve multiplication; it's just addition or subtraction with an optional right shift (basically - I'm not sure what the exact hardware implementation of the division by two is). It's done once per pixel, as the PPU is generating the signal to send to the TV, and the result is not stored.
I was under that impression, the day-night cycle is handled more gracefully in DQIII, probably with "handmade" palettes.creaothceann wrote:All that happened is that the colors used by certain backgrounds and sprites are darkened.
But i suppose at some point it must transform this array of indexes into an actual rgba array. In that case, it must compare if the pixel is 0 or not.Emulators do not go through the picture looking for colors to change - the games simply generate slightly different source data (different palette entries) and the hardware/emulator generates a new screen from that initial data.
By the way, this thread improved my code, mostly thanks to avoiding floats at some places and processing unique colors and then replacing rather than processing the whole image. Also gave me a closer insight on how the snes works., Sometimes technical documents are too technical. But i'll have a glance at these links, the blending formulas are simple yet effective judging by the results.
Re: Snes emulation image processing
No, transparency is just half transparency. Multiple grades of transparency can be simulated by using additive color math and fading the palette to black. Example: the Great Fairy in LTTP (you'll notice there's kind of a "jump" as it switches to additive blending but the rest of the fade is smooth).OneQuestionPlease wrote:But i was under the impression the snes actually supported different grades of transparency
SNES NTSC 2/1/3 1CHIP | serial number UN318588627
-
- Posts: 611
- Joined: Mon Jan 23, 2006 7:47 am
- Location: Germany
- Contact:
Re: Snes emulation image processing
No alpha needed.OneQuestionPlease wrote:But i suppose at some point it must transform this array of indexes into an actual rgba array.
That's done before converting the tile data to actual colors.OneQuestionPlease wrote:In that case, it must compare if the pixel is 0 or not.
If you look at anomie's document that I linked above, you'll see this:
Code: Select all
Color Math
----------
Each main-screen BG (and the color-0 backdrop, and the sprites (although
sprites with palettes 0-3 never participate)) may be marked in register $2131
to participate in color math. If the visible pixel is from a layer/OBJ
participating in color math, we perform one of 8 operations on the pixel,
depending on $2130 bit 1 and $2131 bits 6-7.
0 00: Add the fixed color. R, G, and B are added separately, and clipped to
the max.
0 01: Add the fixed color, and divide the result by 2 before clipping (unless
the Color Window is clipping colors here).
0 10: Subtract the fixed color from the pixel. For example, if the pixel is
(31,31,0) and the fixed color is (0,16,16), the result is (31,15,0).
0 11: Subtract the fixed color, and divide the result by 2 (unless CW etc).
1 00: Add the corresopnding subscreen pixel, or the fixed color if it's the
subscreen backdrop.
1 01: Add the subscreen pixel and divide by 2 (unless CW etc), or add the
fixed color with no division.
1 10: Subtract the subscreen pixel/fixed color.
1 11: Subtract the subscreen pixel and divide by 2 (unless CW etc), or sub
the fixed color with no division.
https://www.youtube.com/watch?v=57ibhDU ... _lvGwfn6GV
https://www.youtube.com/watch?v=Tfh0ytz ... KdXygMO71Z
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
-
- Posts: 3140
- Joined: Wed May 19, 2010 6:12 pm
Re: Snes emulation image processing
In short, transparency can either be on or off, and any other level of transparency is just transparency + changing the palette.
If you ever see a smooth complete transition between invisible to non-transparent, it's probably done using a duplicate BG with a darkened palette on the subscreen with color math applied to sprites.
If you ever see a smooth complete transition between invisible to non-transparent, it's probably done using a duplicate BG with a darkened palette on the subscreen with color math applied to sprites.
- OneQuestionPlease
- Posts: 6
- Joined: Mon Aug 21, 2017 4:47 am
Re: Snes emulation image processing
I get it. I must say in favor of the snes that blending formulas are good enough to fool me. Is not a big deal of a limitation if it works as intended. The docs and videos were pretty interesting, i will spend some time searching for more on youtube.
I saw that rotation and scaling are performed in mode 7, which was a question that i had too, but i have seen a 'wave' effect in many games which makes me suspect it is also a snes function. Am i wrong or are these things performed 'by hand'? https://www.youtube.com/watch?v=3TdziNzMtaM
I saw that rotation and scaling are performed in mode 7, which was a question that i had too, but i have seen a 'wave' effect in many games which makes me suspect it is also a snes function. Am i wrong or are these things performed 'by hand'? https://www.youtube.com/watch?v=3TdziNzMtaM
Re: Snes emulation image processing
Ripple waving like the one in the video is done easily enough on systems that can do it (Sega Master System is one that can't): the code manually changes the vertical scroll on each scanline, usually through an interrupt, but can be done with timed code on other systems.
Changing the horizontal scroll on each scanline gives the familiar wavy/fire effect seen in Thunderforce III, etc.
Changing the horizontal scroll on each scanline gives the familiar wavy/fire effect seen in Thunderforce III, etc.
Re: Snes emulation image processing
In addition, the SNES and Mega Drive can both do offset-per-tile, or column scrolling. This can result in a wavy effect like this, or a fake rotation effect when combined with line scroll, like this, or this (note that only the backdrop is using this trick; the polygons are rendered at the proper angle). This scene combines horizontal and vertical line scrolling in the backdrop layer with column scrolling in the foreground, plus what looks like addition with the constant colour, and mosaic when you touch a Fuzzy.
The Mega Drive can use scroll offsets for every 16-pixel-wide column. The SNES can do it in 8-pixel columns, but you have to sacrifice a background layer - in Mode 1 you have two 4bpp layers and one 2bpp layer, but you can't do column scrolling; in Mode 2 you can do column scrolling but the 2bpp layer is gone. (Similar sacrifices are made going from Mode 3 to 4 and from 5 to 6, for the same reason, that being VRAM bandwidth.) On the Mega Drive there is no third layer, just the two 4bpp layers, meaning Mode 2 is a reasonably close match to the Mega Drive's feature set.
The Mega Drive can use scroll offsets for every 16-pixel-wide column. The SNES can do it in 8-pixel columns, but you have to sacrifice a background layer - in Mode 1 you have two 4bpp layers and one 2bpp layer, but you can't do column scrolling; in Mode 2 you can do column scrolling but the 2bpp layer is gone. (Similar sacrifices are made going from Mode 3 to 4 and from 5 to 6, for the same reason, that being VRAM bandwidth.) On the Mega Drive there is no third layer, just the two 4bpp layers, meaning Mode 2 is a reasonably close match to the Mega Drive's feature set.
-
- Posts: 271
- Joined: Sun Mar 27, 2011 10:49 am
- Location: Victoria, BC
Re: Snes emulation image processing
Another real neat trick you can do by carefully adjusting the scroll during hblank interrupts is vertical scaling.
-
- Posts: 611
- Joined: Mon Jan 23, 2006 7:47 am
- Location: Germany
- Contact:
Re: Snes emulation image processing
HDMAOneQuestionPlease wrote:i have seen a 'wave' effect in many games which makes me suspect it is also a snes function
Prime example: Axelayadam_smasher wrote:Another real neat trick you can do by carefully adjusting the scroll during hblank interrupts is vertical scaling.
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10
- OneQuestionPlease
- Posts: 6
- Joined: Mon Aug 21, 2017 4:47 am
Re: Snes emulation image processing
Thanks, the snes pipeline seems to be complex, given all the stages you have at hand to alter the output.
Well, i'm glad i had this thread. Got a lot from it, and i really like to continue but i don't know what more to ask. My experiment was fulfilled successfully. I did not know exactly if emulators pull other tricks to relieve the lack of custom hardware, but just by knowing what is going on behind the scenes on the snes gave me many ideas to optimize the engine.
The difference of knowledge between the average user and me is huge but i'd like to lurk from the darkness to read more interesting threads. I thank everyone who answered and provided me with examples to ease my understanding. And i'd say this thread will come handy for many people in my situation, maybe this place is out of the radar for the ones in my shoes, but regardless of that, the thread is here waitting for whoever arrives at it.
Well, i'm glad i had this thread. Got a lot from it, and i really like to continue but i don't know what more to ask. My experiment was fulfilled successfully. I did not know exactly if emulators pull other tricks to relieve the lack of custom hardware, but just by knowing what is going on behind the scenes on the snes gave me many ideas to optimize the engine.
The difference of knowledge between the average user and me is huge but i'd like to lurk from the darkness to read more interesting threads. I thank everyone who answered and provided me with examples to ease my understanding. And i'd say this thread will come handy for many people in my situation, maybe this place is out of the radar for the ones in my shoes, but regardless of that, the thread is here waitting for whoever arrives at it.