Basic fade in or out of a palette

Discuss technical or other issues relating to programming the Nintendo Entertainment System, Famicom, or compatible systems.

Moderator: Moderators

User avatar
Banshaku
Posts: 2328
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Basic fade in or out of a palette

Post by Banshaku » Thu Aug 30, 2018 7:30 pm

I just realized that up to now I never did any fade in or out on the nes :shock:

I'm sure there must be many topics on the subject on nesdev and first tried the phpbb search (bad idea :lol:) and got not much related to my question. Google was more useful and I ended up on this topic first:

https://forums.nesdev.com/viewtopic.php?t=8927

but... let just say that I'm not very knowledgable in color theory and reading this topic confuse me more than anything ^^;;; I will search again since I'm sure there must be more topic on the subject but some of the basic questions I have are:

- I guess with the limited colors it must be hard to make some smooth transition compared to PC VGA 320x200x256
- I would like to find some simple example until I understand more on the subject

My use case for now is mostly
- a screen that either fade in or out
- white text that fade in/out to allow to update with the next text

In the mean time I will continue to search nesdev since there must be other topics.

edit:

I just found this answer from Kasumi on the subject:

https://forums.nesdev.com/viewtopic.php ... in#p217143

Seems to give a simple example for fade out but mentioning that fade in is more complicated.

User avatar
tokumaru
Posts: 11466
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Basic fade in or out of a palette

Post by tokumaru » Thu Aug 30, 2018 7:56 pm

Basic fades are easy, just modify the brightness bits of every color each step until you reach the target... A lot of games do that, even in machines that use RGB, where each component (red, green and blue) is animated toward the target. The problem is that in machines that have few steps of brightness or color components the effect doesn't look smooth at all.

In RGB, there's a quick hack (often used in Master System and Genesis games) to improve the smoothness of fade sequences that works really well: for each color, animate one component at a time - first get rid of the red, then the green, and finally blue. When fading in, do the opposite. Ninja Gaiden on the SMS is an example of a game using this technique. The Sonic games are examples on the Genesis, but I think most Genesis games did this.

On the NES it's a bit tougher to make things smoother. I tried alternating between changing brightness and hue each step, sliding colors toward blue when fading out, and toward yellow when fading in. It worked reasonably well, but was too complicated/slow to do and wasn't as smooth as I'd like.

Another thing I tried was changing only colors of a specific brightness each step. For example, in the first frame of fading out, I'd make all $0x colors black. In the next frame, I'd darken all $1x colors to $0x. In the next, all $2x colors to $1x. Finally, all $3x colors would turn into $2x. Then I'd do the whole thing again and again until all colors were black, except that each new round has less steps because there are no colors of the highest brightness anymore (in the 2nd round, there are no $3x colors anymore, in the 3rd round there are no $3x or $2x colors anymore, and so on). The results for this technique were fairly good, as long as you had palettes with good distribution of brightness on screen, otherwise it looked as stiff as the classic 4-level fade.

User avatar
rainwarrior
Posts: 7677
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Basic fade in or out of a palette

Post by rainwarrior » Thu Aug 30, 2018 8:18 pm

The simple version:

1. Copy your palette to a temporary buffer.
2. Copy that temporary buffer back into the normal palette location, but subtract $10 from all values. If carry, replace the result with $0F.
3. Wait for an NMI to update the palette.
4. Repeat 2-3 but with $20. (I generally make this a function that takes the subtract value as a parameter.)
5. Repeat 2-3 but with $30.
6. Repeat 2-3 but with $40.
7. Turn rendering off so you can do your updates.

If you want to see what code for this looks like, here's a version of it I wrote a few days ago:
https://github.com/bbbradsmith/zensf/bl ... ase.s#L792

Fading back in is the reverse. I wouldn't say it's more complicated at all. Basically the same stuff just a different order of subtraction parameter.

Handling colours in the $0D column is a problem with this (either don't use them, or put in some special case to avoid $0D). Edit: it was just 2 more lines, so I put it in.


Trying to do "smoother" fades, or like bisqwit's example trying to do arbitrary fades to other colours, that's a big topic with a bunch of things people have tried. I don't think there's any one solution that seemed like the killer way, they all have their tradeoffs. Maybe it'd be a fun topic to play around with, though.
Last edited by rainwarrior on Tue Sep 04, 2018 11:49 am, edited 1 time in total.

User avatar
koitsu
Posts: 4216
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Basic fade in or out of a palette

Post by koitsu » Thu Aug 30, 2018 8:34 pm

Your understanding of the difficulty (re: fading in/out) on the NES is correct -- it has to do with the extremely limited (effectively) pre-defined palette and choices.

The biggest complexity, IMO, lies in deciding *visually* what looks good fade-wise. Let's talk about that. It's a highly subjective topic.

For "simple/dumb" fades, the NES palette colours are positioned in such a way where you can add/subtract $10 from the palette value to get a darker or lower shade... with two caveats: i) avoiding $0D (do not use this value!), and i) transitioning from the "darkest colour value" to $0F (total black). It also operates mostly on the concept that you will always be fading from brightest to darkest/black, or from darkest/black to brightest. Obviously you can tweak the code to limit the range, but... well... just keep reading.

This method is easy/dumb because you can do it in software. It's identical to what Kasumi demonstrates in that post. It works great for most things, but that's again subjective.

The more complex fading method doesn't involve simple +/- $10 math, it involves lookup tables for each "step" or "stage" of a fade. This is what a lot of games do; it's a pre-calculated table of each "phase" going from colour X to colour Y. Sometimes it's 4-phase, other times 5 or 6. To understand the model, you need to look at the palette closely and notice several things:

1. There are LOTS of choices when going from white to black or black to white. For example, white to black could be: $30 -> $20 -> $3D (or $10?) -> $00 -> $2D -> $0F. Do you need all those "phases"?

2. There are substantially less choices when going from a colour to black, or from a colour to the brightest colour. For example, let's take blue. It could be $01 -> $11 -> $21 -> $31. But maybe visually that doesn't look good, it's only a few "phases". What might look better is (I'm just making this up BTW) $01 -> $0C -> $11 -> $21 -> $2C -> $31.

The tricky part about #2 is that you have to remember 2 additional things:

i) Not everyone's TVs are going to have the same tint and hue. So a blue transition that uses some cyan as "intermediates" to help potentially make it look better might look like crap to some,

ii) The complication of how pixels of different colours look on actual TVs when placed next to one another. Ex. that red pixel next to a brown pixel visually looks very different/unique (almost its own shade). Really good NES art tends to cater to this; you load the game up on an emulator and you're like "wow, it looks so... sterile. Clinical. Plain. Literal." Load it up on a NES hooked up via A/V or RF and it's like "ah yeah the colours sort of bleed together". So you get to think about how that affects fades as well. IIRC, the games Magician and Faxanadu use this a lot.

If you want examples of what good fades look like, and this is again subjective, load up Ninja Gaiden. Entering the first bosses room there's a fade, after killing him another, and when loading into the next level another. Load this up on a NES and look at it. Then load it up on an emulator like Nestopia using the stock palette and look at it -- on some transitions a dark colour goes "too bright" for a phase, followed by it "darkening" for the next phase. Notice how different "parts" of the screen seem to take longer to fade in/out than others -- it's a effect of the palette transition choices Tecmos made. If you just want to see it yourself, here are two videos of gameplay from Youtube:

https://www.youtube.com/watch?v=ueeKMQSS4bw (emulator with dumb rounding/smoothing filter applied)
https://www.youtube.com/watch?v=uzU53dVHGSo (actual NES)

The text fade I did in the FF2j/e intro replacement for Neo Demiforce was really easy: it was a single colour in the palette I had to deal with: white to black and black to white. I opted for $0F (black) --> $00 (grey) --> $10 (lighter grey) --> $20 (even lighter) --> $30 (white). Fading out was the exact opposite. Delays (for smoothness/speed of the fade) were done by simply waiting for VBlank. I thought about trying to fade in/out the logo too (on reset, and when starting), but when I started thinking about all of the above, I opted to just do text only. I think the 3 or 4-stage "simple" way looks best visually, barring maybe an exception for black-to-white and white-to-black which could involve one or two more steps -- but it means more code, and other graphics on screen (of other colour) have to "wait" for a phase without any change (again, see Ninja Gaiden).

Edit: lots of other replies above me that say basically the same thing. Seems we're pretty much all in agreement ;)

kuja killer
Posts: 80
Joined: Mon May 25, 2009 2:20 pm

Re: Basic fade in or out of a palette

Post by kuja killer » Thu Aug 30, 2018 9:54 pm

Here's from megaman 3
you just JSR to 1 of the 2 "global labels" at the very start and that's it. :)

https://pastebin.com/UAu39SqF

User avatar
Banshaku
Posts: 2328
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: Basic fade in or out of a palette

Post by Banshaku » Fri Aug 31, 2018 12:05 am

Thank you everyone for the examples, that will help to make a basic one for my prototyping needs.

@koitsu

One on my list of favorite games as an example: you chose you samples well :lol: After seeing the fade in/out from the video, I'm so used to what we can do with current hardware (or even just dos VGA256) that my "rose tinted memories" of it gave me the impression that I could have done something more smooth then that.

I will first test a basic one and see if it's enough for my needs. If not, then I will have to decide on a case by case basic then.

calima
Posts: 1017
Joined: Tue Oct 06, 2015 10:16 am

Re: Basic fade in or out of a palette

Post by calima » Fri Aug 31, 2018 3:59 am

Nobody mentioned it yet, but neslib contains one.

User avatar
FrankenGraphics
Formerly WheelInventor
Posts: 2032
Joined: Thu Apr 14, 2016 2:55 am
Location: Gothenburg, Sweden
Contact:

Re: Basic fade in or out of a palette

Post by FrankenGraphics » Fri Aug 31, 2018 4:06 am

fair warning about neslib:s fade routine if you use the $xD column, it will erroneously try to handle it by using the $x0 column instead. This is good practice, but what it does wrong is that it translates the values to a brighter end result. Notice that $x0 and $0d do not align with the typical brightness scale - they are offset in reverse directions! An $xD handling clause should mind this difference.

Or i might remember it slightly wrong: it might just be that neslib does not translate the then $xD, now $x0 colours back to $xD when fading in again.

Either way, colours will be off.

since both $0F and $0E are considered safe, maybe using $0E specifically for $xD colours as the final black can be a good way to remember what it should translate back to, since it needs to get black before the others. Or you can make a special case for $x0 and $xD combined where $2d is seen as the step between $00 and $0f, always.
Last edited by FrankenGraphics on Fri Aug 31, 2018 6:40 am, edited 1 time in total.
http://www.frankengraphics.com - personal NES blog

User avatar
Kasumi
Posts: 1289
Joined: Wed Apr 02, 2008 2:09 pm

Re: Basic fade in or out of a palette

Post by Kasumi » Fri Aug 31, 2018 4:56 am

Here's another post by Kasumi that describes the logic for hue fades. https://forums.nesdev.com/viewtopic.php ... 05#p204005
That particular post has a wrong assumption, though, so read the next two as well.

User avatar
DRW
Posts: 1914
Joined: Sat Sep 07, 2013 2:59 pm

Re: Basic fade in or out of a palette

Post by DRW » Fri Aug 31, 2018 6:51 am

In my adventure game, I use the simple way:

For each palette value:
If value >= $10 then subtract $10 from the value.
Else set value to $0F (black).

Do this four times and the whole palette is black.

The $0D problem shouldn't be an issue because there's really no reason to use the gray colors in the $xD column in the first place. The shades in $x0 are pretty much identical.
Available now: My game "City Trouble".
Sales website: https://megacatstudios.com/products/city-trouble
Gameplay: https://youtu.be/Eee0yurkIW4
Download website: http://www.denny-r-walter.de/city.htm

tepples
Posts: 21752
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Basic fade in or out of a palette

Post by tepples » Fri Aug 31, 2018 7:07 am

Colors $20 and $30 white behave differently in fades, and this can be if you want to mark something as "super-bright".

In Thwaite, I special cased color $00 (dark gray) to become $02 (dark blue) in fades. The loop looked like this:

Code: Select all

palloop:
  lda cutscene_palette,x
  sec
  sbc fadeAmount
  bpl palNotNeg
  cmp #$F0
  bne palNotF0
  lda #$02
  bne palNotNeg
palNotF0:
  lda #$0F
palNotNeg:
  sta PPUDATA
  inx
  cpx #$20
  bcc palloop

User avatar
Sumez
Posts: 909
Joined: Thu Sep 15, 2016 6:29 am
Location: Denmark (PAL)

Re: Basic fade in or out of a palette

Post by Sumez » Fri Aug 31, 2018 7:29 am

My solution to the rough fade-out from simple implementations is to just do the fadeout fast enough that it doesn't matter.

Of course it depends on what your goal is. If you want a slow fadeout for atmospheric reasons (like fading into a flashback or whatever), it's of course useful to go the extra step to mind the difference between different hues, etc. and making use of color emphasis, to make the fade smoother.

But for my own purposes, which I guess is similar to what most people need, I just want a less abrupt transition between completely different scenes (like when entering a menu or finishing a stage), and for this purpose I figured it's actually better for the player to make it fast anyway, and even a rough fadeout is a lot smoother than a straight cut-off. As a rule of thumb I always execute my cheap-o fadeout whenever I switch between game states.

User avatar
Banshaku
Posts: 2328
Joined: Tue Jun 24, 2008 8:38 pm
Location: Fukuoka, Japan
Contact:

Re: Basic fade in or out of a palette

Post by Banshaku » Fri Aug 31, 2018 8:15 am

Since I'm using a lot more C code than before and what made me move into that direction is neslib, maybe I should check again some of the method used. I mostly use from neslib the gamepad method, the idea of loading palette in nmi if flag is set and now the RLE screen loader. I guess I'm slowly using back a few functions from it ^^;; The fade function, I wasn't sure it I could use it or not so maybe I should try it then.

@calima

That's true, there was one. I don't know why I removed it from my test files ^^;;

@FrankenGraphics

I will be careful if I try back the neslib one then. thanks.

@DRW

That seems straightforward. I will keep note of it if I decide to make a quick one. Thanks.

@Sumez

Right now this is just for quick transition between 1 image to another, to fade out the text then fade-in it back so a very simple type of fade.

edit:

Forgot to say thank you for the link to @kasumi and for the example from @Tepples. thanks!
Last edited by Banshaku on Fri Aug 31, 2018 8:24 am, edited 1 time in total.

User avatar
FrankenGraphics
Formerly WheelInventor
Posts: 2032
Joined: Thu Apr 14, 2016 2:55 am
Location: Gothenburg, Sweden
Contact:

Re: Basic fade in or out of a palette

Post by FrankenGraphics » Fri Aug 31, 2018 8:16 am

DRW wrote:The $0D problem shouldn't be an issue because there's really no reason to use the gray colors in the $xD column in the first place. The shades in $x0 are pretty much identical.
This seems to be a fairly widespread notion, but fortunately for us artists, it's actually not true. I think the idea of them being close to the same has gotten popularized because, for some reason, fceux doesn't make a distinction between $00 and $2d, so $2d looks way too bright.

The difference between $10 and $3D is minute, so much that on some tv:s and contrasts, you might not be able to see the difference. $3D is still my preference for a desaturated highlight in some cases, because it is simply that little bit brighter, which makes it that little bit more comparable to the brightest pastel hues.

But $2D is the only gray that i'd call dark gray. The difference is significant, or with low contrast - still noticeable.
I've tested this on 6 different screens, 4 crt:s, 1 "hd ready" tv and one modern hdmi monitor. The crt test was side by side at the same time.

using bisqwits savtool, which i think yields a result i can recognize (even if the hues are different from my PAL unit), you get the following:
comparison.png
comparison.png (8.01 KiB) Viewed 3912 times
(1: $0c, 2: $2d, 3: $00, 4: $1c)

Notice how $00 has a brightness comparable to colours $1x, while $2d has a brightness that falls somewhere halfway between $0x and $1x. This is in line with with my screen experiences. So $2d this is the closest we can get to a gray "shadow", and i'm keen on using it as such. There are lots of cases where $00 simply doesn't cut it.

For the compatibility discussion that tends to follow whenever $2d is mentioned, i should probably link to the discussion (and its solutions) we just had last month. :wink:
Last edited by FrankenGraphics on Fri Aug 31, 2018 8:52 am, edited 1 time in total.
http://www.frankengraphics.com - personal NES blog

tepples
Posts: 21752
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Basic fade in or out of a palette

Post by tepples » Fri Aug 31, 2018 8:44 am

A change to use $2D as the step between $00 and $0F and correct for things going to $0D is straightforward:

Code: Select all

  ldx #0
palloop:
  lda cutscene_palette,x
  sec
  sbc fadeAmount
  cmp #$0D
  beq isBlack
  ora #$00
  bpl haveColor
  cmp #$F0
  bne isBlack
  lda #$2D
  bne haveColor
isBlack:
  lda #$0F
haveColor:
  sta PPUDATA
  inx
  cpx #$20
  bcc palloop
But by this time, we're hitting a worst case close to 36 cycles per color, which is about half of NTSC vblank. So if you're pushing substantial data to the PPU other than palette and OAM in the same frame as a fade step, it might be wise to calculate the fade step into your transfer buffer.

Code: Select all

  ldy #0
  lda popslide_used
  clc
  adc #32+3
  sta popslide_used
  lda #$3F  ; destination address high
  sta popslide_buf,x
  lsr a  ; A = 31, correct value for length-1 byte
  sta popslide_buf+2,x
  tya ; destination address low
  sta popslide_buf+1,x
palloop:
  lda cutscene_palette,y
  sec
  sbc fadeAmount
  cmp #$0D
  beq isBlack
  ora #$00
  bpl haveColor
  cmp #$F0
  bne isBlack
  lda #$2D
  bne haveColor
isBlack:
  lda #$0F
haveColor:
  sta popslide_buf+3,x
  inx
  iny
  cpy #32
  bcc palloop

Post Reply