Idea for creating lag-free Tetris on non-CRT displays

[The Idea Guy]

Hi. First post, so go easy on me.

As many of you will know, since YouTube started recommending videos from one of the world Tetris championships a few years ago, there has been a rise in popularity of people playing Classic Tetris on the original NES hardware (in many cases with a view to competing in online and in-person tournaments).

I myself was inspired by the late Jonas Neubauer (RIP) to start streaming Tetris games (https://www.twitch.tv/tetrisstream) using original NES hardware.

Again as many of you will know, due to the lag with LCD, plasma, and OLED screens between receiving an image from the NES and displaying it to the player, tournaments and tournament practice play are conducted using CRT screens which have almost no lag. Indeed, I picked up an old CRT TV for this very purpose.

The problem with this solution is that CRT TVs are bulky, heavy, take up a lot of space, and create a bit more eye-strain than modern displays. Also, since nobody makes them any more, as the years pass they will become increasingly hard to find in good condition.

My idea is for a couple of different workarounds, one of which I am pretty sure is possible, the other of which may or may not be (I hope that the devs here will know).

First workaround is with emulation: I am sure that accurate (close-to-perfect) NES emulation is not difficult on modern PCs. However, would it be possible to have an emulator run in such a way that it knows that the player sees its output slightly late, and thus allows the player an additional period of a few milliseconds to carry out inputs before locking the pieces in position and beginning the next shape drop in Tetris?

Additionally, could rules be added so that if a shape vertically passes another such that it can no longer be moved to the left of right of that other piece, then so long as the player provides a left or right input within a certain number of milliseconds of that happening, the game allows it to pass "through" the blocking piece as if the input had been received a few milliseconds earlier? This would seem to be all that is needed to give players a lag-free gaming experience on a non-CRT display.

Second workaround is with a rom-hack: would it be possible to create a modern cartridge which did the same thing as outlined above, and which played in an original unmodified NES?

If someone were to make one of the above happen, players could stream and practice for tournaments using an LCD display before playing and competing on CRT screens on tournament day, without having to adjust to differences between each playing environment before or after.

I apologise in advance if any of that is plain stupid. It is possible that I have missed something fundamental in this analysis, though after giving it careful thought, I believe this should at least be possible in theory.

Cheers & beers!

[Controllerhead]

I admire your enthusiasm, but what you describe breaks the laws of physics.

NES Tetris, or any NES game, usually has one "lag frame" in which input is detected and the next game state is calculated while the "previous" frame is being drawn. The calculated picture is then drawn during the next frame, the process repeats. Button press to drawn gamestate ends up being anywhere from about 16-32 milliseconds(ish) from when a button is actually hit to when the top of the affected frame begins to draw on a CRT. In human terms, this might as well be instantaneous.

A modern emulator paired with a high refresh rate monitor will be fine for most people. Turning off vSync might make it draw a touch faster but at the price of chopped up looking frames. I was heavy into speedrunning for a while, and while CRT's do feel slightly snappier, a proper emulator / joypad / monitor setup is all but indistinguishable in practice.

Anyway, NES Tetris itself isn't the fastest of Tetris games. The real challenge of playing at a high level is mashing the dpad fast enough to actually move the piece! To address this, i made a romhack to speed up the game and sharpen the controls considerably, mostly just for programming practice and my own amusement:
https://www.romhacking.net/hacks/4973/

Anyway, you can't really make joypad detection any faster in NES Tetris. I've looked into it =p
I hope this answers your questions!

[Dwedit]

This is already a thing in emulators, and it's called RunAhead.

Normally, when you play a game like Super Mario Bros in an emulator, you can measure a difference between when you tell the player to jump, and when the jump actually starts. Using the Frame Advance and Jump buttons, you can see that if you hold "A", it takes two presses of Frame Advance before you see Mario jumping. We say the game has 1 frame of "internal lag".
This is because a NES game will blast out the new screen content during vblank time, display that frame, read input, run game logic, and decide what gets drawn next frame.
Only the Atari 2600 does not have this internal lag.

But using a special trick called RunAhead, you can eliminate the internal lag. This will show a future frame from 1 frame ahead instead of the current frame, and you will see Mario jump immediately on the next frame.

The basic process it uses is here. Instead of Running one Frame, it does this:

* Run Frame (don't display it) <- your joystick input is used here
* Save State
* Run Frame (display it) <- joystick input is irrelevant here
* Load State

After running this sequence, the game state is the same as if you ran one frame. But the screen image is ahead by one frame.

Additionally, Windows has a problem with the DWM. It adds 3 frames of lag to EVERYTHING, unless the program is using Exclusive Fullscreen mode.

[The Idea Guy]

Thank you guys!

I appreciate that feedback. While I understand that the lag with a well-set-up LCD display is small, I can personally notice it quite a lot when I play Tetris on displays other than my CRT.

While it might seem on the surface that I am trying to break the laws of physics, it still occurs to me that any lag (however small) could be negated by allowing a player to move a Tetris piece through another one to the left or right for just a few milliseconds after this would normally be impossible. Would that in itself require extremely lengthy and fundamental alterations to the game code do you think?

Anyway I am certainly going to try out your romhack Controllerhead. I've dropped you a PM about that.

Also Dwedit that information is good to know. Great forum. Very pleased to have found this place!

[Controllerhead]

it still occurs to me that any lag (however small) could be negated by allowing a player to move a Tetris piece through another one to the left or right for just a few milliseconds after this would normally be impossible. Would that in itself require extremely lengthy and fundamental alterations to the game code do you think?

Ok, i'm going off of my own memory here, but anyway:

The game sets up the background and sprite changes in the previous frame in a period called vBlank when the PPU (and CRT) are not actually drawing the screen. All games are like this. The next thing it does is read the controller. Then it calculates all changes to the gameboard etc etc. When that's done, it waits for the next vBlank for the drawing changes again.

Usually the game calculations are rather quick, and only take about 25% of the CPU time (until scanline 60 or so). I suppose you could add an artificial delay before reading the controller to add a few more milliseonds of possible response time... but not so fast though!

When the game gets a line clear, or especially a Tetris, it ends up taking much more CPU time. The problem is, if the game isn't done calculating before it tries to draw the next frame, it crashes. If you add in a check to make it not crash and finish calculations, the game will lag. I dealt with these issues when making the animations and piece spawning go faster.

So... you might be able to get away with adding some delay before the controller read. It also may crash the game sometimes. If you prevent that by finishing the game calculations, the game will lag. I suppose you could check what state the game is in before adding the delay, but, that sounds like a ton of work for minimal gain and a possibly unstable experience.

If you do decide to go this route, i'd imagine the "delay reduction effect" you could get away with would be extremely minimal, talking like 3-4ms or 1/4 of a frame. I don't think it's worth it?

ALSO: Usually emulators don't read controllers like consoles do, they are read immediately with code when they are polled on console, but, this isn't the case with emulation. So, this change might have zero effect on emulators, so...

TL;DR- It might be possible(ish) for an extremely minimal gain... and probably wouldn't affect emulators

[The Idea Guy]

Again, thank you. I really appreciate your care and patience in explaining this to me.

Although you have done a great job of pointing out the problems associated with delays, something that might be worth keeping in mind is that I am not actually proposing any additional delay being added; rather I am proposing a change to the game's rules: as you know, pieces can not move through other pieces in Tetris. ...I was just wondering how hard (if even possible) it might be to change that rule such that a piece can move through another piece left or right for up to X milliseconds after they have drawn level with each other in the vertical plane.

I'm thinking that this in itself could be a crude but effective way of mitigating the effects of display lag. What do you think about that?

[Controllerhead]

I was just wondering how hard (if even possible) it might be to change that rule such that a piece can move through another piece left or right for up to X milliseconds after they have drawn level with each other in the vertical plane.

Sorry, i'm not sure what you're asking?

Anyway, the game runs in steps. 60 of them per second. You can think of each step as 16 milliseconds (16.66667 etc). The controller is read, the game board is calculated, and everything is drawn the next step, the process repeats. It isn't really possible to do anything in between that time?

[The Idea Guy]

Sorry, i'm not sure what you're asking?

I'm asking something rather odd, so I am not surprised that some clarification is required.

...Imagine you are playing Tetris and you experience a tiny bit of lag because your (LCD) display isn't showing you the screen image quite as fast as your CRT would have done. Now imagine that as you are playing, a piece starts to drop and you realise that you want to move it to the far left (or right) of the screen, but by the time your brain makes that assessment it's just a tiny bit too late, and the shape has already fallen so far that a piece already nested in the stack prevents you from moving it across left or right to where you want.

Now, normally, you would just be screwed. BUT... ...if the rules of the game were modified so that you can move this new piece through the stack left or right as if the stack were not there (at least for a few milliseconds after the falling piece had drawn level with it), you would still be able to play as if the lag had not caused you any problem. Do you see what I mean?

[Controllerhead]

So ...you want to disable the left / right clipping checks so you can move through a populated space of the gameboard?

I mean, you would first end up inside of the populated space you were trying to go through?

[The Idea Guy]

So ...you want to disable the left / right clipping checks so you can move through a populated space of the gameboard?

I mean, you would first end up inside of the populated space you were trying to go through?

Yup, and yup.

I guess the moment you emerged from the co-occupied space, the clipping checks would apply again as normal so it were only ever one-way, and if you stayed inside the occupied area for more than X milliseconds it would automatically spit you out on whatever unoccupied side was available.

Also clipping would only ever be disabled for no more than the time it takes for a piece to fall a single block at max (pre-killscreen) speed, and not be disabled at all if it were determined that there was insufficient space for the new piece to be positioned after moving through an existing part of the stack.

The more I think about this, the more I realise that it is something that would probably be possible only on emulators, since the additional processing required to implement these additional rules in realtime would over-burden the processor of original NES hardware.

Also, my idea probably has a fairly high chance of being judged to be stupid and pointless by the wider community.

[The Idea Guy]

oops, double posted by mistake instead of doing an edit

[Controllerhead]

The more I think about this, the more I realise that it is something that would probably be possible only on emulators, since the additional processing required to implement these additional rules in realtime would over-burden the processor of original NES hardware.

Mmmm no, it could be done. Plenty of processing power to do that. I was able to add hard drop without any issue continually calculating up to 17/18 new vertical positions in one frame.

The details would be annoying to work through. How many frames of leway do you have? How many game board blocks would be possible to clip through? One? More? How about if you were rotated as a flat line piece, would it just jolt you to the other side of the board if there were a valid space?

It's a wacky idea man.

Speaking strictly gameplay theory here, this shouldn't really ever be a problem. You generally want to keep your board as flat as possible. Having a jut in the middle is a huge no-no. You want to build 1-2 block wells on either side, and for NES Tetris, usually the right because you can move the vertical line piece over in one less dpad press.

I can make it into the 20s a bit, but not for long... usually 19 takes me down pretty quick because i can't move the damn pieces fast enough, which is why i made the romhack in the first place =p

[The Idea Guy]

Hmm, all good to know. Thanks. I'll give the details some further thought, since you've raised a few interesting and natural questions.

Right now I am on the PAL version. ...I have both a PAL and an NTSC NES, but I'm waiting for the NTSC Tetris cart to arrive in the post.

I can handle level 15/16/17 OK, which I believe is the same as level 18 on the NTSC version, but yeah, I'm sure 19 is hard (equivalent to 18 on PAL I believe). I can last a fair while at that speed, but I find myself going line by line then rather than building for tetrises.

It will be interesting to see what my high score is on the NTSC version. I've managed 250,000 on the PAL one, which I have calculated should let me score at least double that on NTSC, but it's a bit tricky to compared them perfectly in that respect.

As I am 99% sure you know, they are much more different from each other than most PAL/NTSC variants tend to be. It's very frustrating having to wait nearly two months for international postage during this stupid lockdown BS.

[Controllerhead]

I can handle level 15/16/17 OK, which I believe is the same as level 18 on the NTSC version, but yeah, I'm sure 19 is hard (equivalent to 18 on PAL I believe). I can last a fair while at that speed, but I find myself going line by line then rather than building for tetrises.

Oh wow, i loaded up PAL Tetris, you're right, 19 moves every frame at full kill screen speed!
That happens much later on NTSC (29 i think?).

As I am 99% sure you know, they are much more different from each other than most PAL/NTSC variants tend to be.

I had no idea they were so different. I'm in the US so i don't usually experiment with PAL games. Cool to know =)

[Dwedit]

I suggest you try running NES Tetris in RetroArch (full screen), using RunAhead 1. It's amazing how much less input delay there is between that and running FCEUX in a window.