Netplay strategies

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

User avatar
zeroone
Posts: 939
Joined: Mon Dec 29, 2014 1:46 pm
Location: New York, NY
Contact:

Netplay strategies

Post by zeroone »

Any suggestions for implementing Netplay?
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Netplay strategies

Post by tepples »

First you have to traverse whatever network address translation (NAT) layers lie between the players. Common methods for this include ICE and IGD.

Once the machines are connected, they send input data back and forth. Over this connection, the machines exchange input data, each with a timestamp saying on which frame it was valid. Because these packets represent real-time data, the emulator disables output buffering on the TCP socket with TCP_NODELAY.

When a button is pressed locally, the emulator does not act on it instantly. Instead, it uses input data from about four frames back, giving time for the other player's input data to arrive. (This delay should be a bit over half the ping.) If input data is late in arriving, the emulator predicts the other player's input (usually by assuming that the input has not changed) and emulates that, but it saves the state beforehand so that it can re-emulate missed frames should the other player's input data differ from the prediction.
hackfresh
Posts: 101
Joined: Sun May 03, 2015 8:19 pm

Re: Netplay strategies

Post by hackfresh »

Typically you want to use UDP as your method for sending packets for these reasons...


http://gafferongames.com/networking-for ... dp-vs-tcp/


Also most NES games aren't set up to take advantage of prediction since re-emulating the frame really isn't an option in a some games. Especially if more than a few frames have gone by.


The basic structure is

1. Determine number of frames to delay input (typically half the ping rounded up to the nearest integer or +1 frame for margin) on local machine by measuring average ping time over a time interval.
2. Synchronize emulator startup/loading of rom
3. Ignore first X frames where X = frame delay since you can't possibly have any input from the other player.
4. Send input every frame with the frame it is to execute on. Frame to execute = current frame + delay
5. Save own input to buffer for local machine
6. On current frame check get own input from buffer and check to make sure input from other player has arrived. If it hasn't arrived emulation freezes until it does
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Netplay strategies

Post by tepples »

hackfresh wrote:Typically you want to use UDP as your method for sending packets for these reasons...


http://gafferongames.com/networking-for ... dp-vs-tcp/
By the time you've implemented reliable delivery of input packets over UDP, with error detection and retransmission and the like, you'll end up having reimplemented most of TCP. Or what am I missing?
Also most NES games aren't set up to take advantage of prediction since re-emulating the frame really isn't an option
How not? It's not like the NES has some huge @$$ amount of RAM like the quarter megabyte of a Super NES save state. There's 20K at the most because games using SOROM, SXROM, ETROM, and EWROM boards tend to be single-player and turn-based. And with the Wii Menu-inspired nemulator emulating over a dozen NES instances at once on modern PC hardware, speed shouldn't be too much of a problem.
If it hasn't arrived emulation freezes until it does
I was describing the technique used by GGPO. In twitch games, I imagine that prediction and reemulation disrupts the experience less than a momentary freeze.
User avatar
koitsu
Posts: 4201
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Netplay strategies

Post by koitsu »

tepples wrote:By the time you've implemented reliable delivery of input packets over UDP, with error detection and retransmission and the like, you'll end up having reimplemented most of TCP. Or what am I missing?
Nothing. You're absolutely 100% correct. And for sake of example, WoW and many other modern games use TCP. The "use UDP because it's fast" mentality is only applicable in one scenario, and that's the handshaking nature of the protocol -- which for large payloads isn't even used any more given how the TCP sliding window algorithm works (RFC 1323): you no longer need to send an ACK for every single packet). TCP selective acknowledgement (SACK) per RFC 2018 further extends this in the case of packet loss.

Use UDP if you want, but you're going to end up reimplementing pieces/parts of TCP like Tepples said. Those reimplementations may be better for your application -- it's up to you to decide and do the analysis -- but it's time not well-spent, IMO.

The biggest problem I find is that the programmers using the underlying syscalls/code for sockets do not actually understand 1) how individual descriptor flags affect what is going across the wire, and 2) have no real familiarity with packet analysis to determine what is going on and correlating that behaviour with actual code in their application. (The latter is somewhat difficult, which is why using a language like C makes this easier -- anything that is abstracted is going to make that task even harder). A common example are packets that arrive out-of-order due to intermediary routers and load balancing on the Internet (and to some degree even NAT routers); there's a common misconception that if machine A sends packets to machine B in the order of 1,2,3 that machine B will receive them in the order of 1,2,3. I will not pontificate on all this this past this point; my statements (especially about the code aspect) sound anecdotal but honestly I'm really preaching fact.

All that said: if you find something online somewhere that actually documents the design and architecture of a realtime gameplay protocol (TCP or UDP, I don't care which), I'd love to read it. I have yet to see anyone publish anything like this. All I've seen are commercial companies writign their own proprietary methodologies (which makes sense to some degree), and open-source nutbags saying "look at some crap I put on github; the code is the documentation" (wrong). Possibly this type of thing is discussed in an actual published game development book. I have a few, but they're all dedicated to graphics, level design, algorithms, and things of this nature.

The buffering methodology Tepples describes (re: the emulator working on input 4 frames behind what's active) is commonly used, but the problem is that 4 frames is sometimes too little, or in other cases too much. It's going to vary based on several variables, absolutely none of which your emulator/game/whatever has control over.

My general suggestion to people is: don't bother implementing netplay in emulators. You're entering a completely different world of completely different pain. And I have seen people implement things (including in commercial games) so utterly wrong that it's baffling.
User avatar
zeroone
Posts: 939
Joined: Mon Dec 29, 2014 1:46 pm
Location: New York, NY
Contact:

Re: Netplay strategies

Post by zeroone »

With those network buffering strategies, there are at least 2 different architectures that could be implemented:

1) Master-slave: The master runs at realtime and the slave has a delayed view of the master. The slave transmits button statuses to the master and they take effect there first. Network lag would only be perceptible on the slave.

2) Symmetric: Both emulators transmit button statuses to each other and each can be potentially delayed waiting for the other. Its a configuration where they are both equally kept in sync by each other. Network lag could manifest itself on either.

Any thoughts on which is better or if there are other architectures? Thanks.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Netplay strategies

Post by Dwedit »

There's also history-changing, where you retroactively apply lagged joystick input. Made possible by logging input, saving state every frame, and running faster than full speed.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
Myask
Posts: 965
Joined: Sat Jul 12, 2014 3:04 pm

Re: Netplay strategies

Post by Myask »

1 is also known as server/client.

Remember that you can have more than two players. In that case, server-client has the server take the brunt of the communications load compared to symmetric.

3) Dedicated server
Like 1) except all players are slaves and the server is a computer that isn't playing...probably not that useful for NES-emulators. Has advantage of being harder to hoodwink/cheat; doesn't have to (but can) render nor process input, so had more processing power available...in the days when 3d graphics cards were not the norm.
4) Terminal-mainframe
Like 3, except the only thing clients send/receive is button presses/graphics information. (Or like 1.)

Again, probably not terribly useful in this age nor for NES emulators...though the idea of a way to encode NESvideos particularly would be nice for other purposes...hmm.
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Netplay strategies

Post by tokumaru »

Surely there's a library of some sort to deal with the low-level stuff, right?
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Netplay strategies

Post by tepples »

koitsu wrote:All that said: if you find something online somewhere that actually documents the design and architecture of a realtime gameplay protocol (TCP or UDP, I don't care which), I'd love to read it.
The design of GGPO is buffering plus history-changing.
The buffering methodology Tepples describes (re: the emulator working on input 4 frames behind what's active) is commonly used, but the problem is that 4 frames is sometimes too little, or in other cases too much.
I chose four as a starting point because it's an approximation of the ideal buffer depth, which is a little over half a ping time.
My general suggestion to people is: don't bother implementing netplay in emulators.
If emulators don't include netplay, then how[Digression detected; taken to another topic]
Мяск wrote:3) Dedicated server
This has at least two advantages. First, NAT traversal can be simpler because the server has a public IP address. Second, the copy of the emulator running on the server can encode the game's output as a live video stream, so long as games' publishers don't use copyright to shut down the streams.
tokumaru wrote:Surely there's a library of some sort to deal with the low-level stuff, right?
GGPO used to be available as a library by Tony Cannon, but I don't know how it's licensed. And I'm not quite sure how to find out, seeing as GGPO's web site has become a soft redirect to FightCade, a version of the FBA emulator.
User avatar
zeroone
Posts: 939
Joined: Mon Dec 29, 2014 1:46 pm
Location: New York, NY
Contact:

Re: Netplay strategies

Post by zeroone »

Is there an easy way to stream NES video output? If the clients are just remote views into a server, that would make life easier. E.g. only transmit tile, nametable, palette data and PPU register changes per frame with their associated timing.
User avatar
rainwarrior
Posts: 8734
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Netplay strategies

Post by rainwarrior »

I've enjoyed emulators with netplay occasionally, especially over LAN, so I think it's not a bad feature to consider for one.


You only need a symmetrical experience if the game is competitive in a way that short scale timing makes a difference. Letting the "server/master" run normally with all the lag/unreliability pushed onto the other players can sometimes be much more straightforward to implement than other techniques. The "slave" has their input acknowledged only when the "master" later tells them it happened.

UDP is often very suitable for networked games, but generally for this case the game engine is built in such a way that data loss is acceptable. If you miss a few seconds of input, the data coming in is enough to place the player where they should be without the game having to go back in time and re-simulate, etc. In the general purpose emulation idea, the game state diverges irrecoverably if you don't have identical inputs, so loss of anything is forbidden. If your game is designed for networked play, though, you have a lot of different ways to deal with lag/unreliability/divergence. If you can recover well from short-term divergence, dealing with lost UDP data is often better than dealing with the extra latency of TCP (all latency causes divergence, unless you halt the game for everybody). It's a question of which method will cause a greater disparity for players, and on what time scale.

(Strategies for networked game design is a whole topic of its own I won't get into here, since we're just talking about emulating, right now.)

In the asymmetrical case, you can at least send save-states from "master" to "slave" as a recovery method if data is lost. (Obviously you can't "merge" savestates, so this isn't an option for fair/symmetrical play. You have to go back and re-emulate.) If you've already ruled out competitive play, you probably might as well go with TCP to make your implementation simpler.
User avatar
Myask
Posts: 965
Joined: Sat Jul 12, 2014 3:04 pm

Re: Netplay strategies

Post by Myask »

^^^I do not believe there is a NES video format/protocol; that was what this bit addressed:
Myask wrote:Again, probably not terribly useful in this age nor for NES emulators...though the idea of a way to encode NESvideos particularly would be nice for other purposes...hmm.
I think that deserves its own topic, though.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Netplay strategies

Post by tepples »

zeroone wrote:Is there an easy way to stream NES video output? If the clients are just remote views into a server, that would make life easier. E.g. only transmit tile, nametable, palette data and PPU register changes per frame with their associated timing.
That could work in theory. But in practice, you'd have to develop an app for Windows desktop that receives and interprets this data, develop an app for OS X that receives and presents this data, develop an app for X11/Linux that receives and presents this data, develop an app for iOS that receives and presents this data, develop an app for Android that receives and presents this data, develop an app for Windows Runtime (8.1/10, 8.1 RT, and Phone 8.1/10 Mobile) that receives and interprets this data, and develop an app for each major set-top box platform that receives and presents this data. This is why most people just encode to video and stream it through a service like Twitch: the bandwidth wasted by video's theoretical inefficiency costs less than developing and deploying the native viewers.
hackfresh
Posts: 101
Joined: Sun May 03, 2015 8:19 pm

Re: Netplay strategies

Post by hackfresh »

I guess I haven't seen how prediction would look on sports game on the nes where things happen every frame. If too many frames go by and the actual input is different isn't it going to look like the game got "rewound"

If the emulator predicts that I didn't press a button to say pass the ball but I actually did press a button to pass the ball and it has to go back and re-do that if too many frames have gone by. Correct? That seems like a worse experience


From that same article on why you shouldn't use TCP

http://gafferongames.com/networking-for ... dp-vs-tcp/

" The problem with using TCP for realtime games like FPS is that unlike web browsers, or email or most other applications, these multiplayer games have a real time requirement on packet delivery. For many parts of your game, for example player input and character positions, it really doesn’t matter what happened a second ago, you only care about the most recent data. TCP was simply not designed with this in mind.

Consider a very simple example of a multiplayer game, some sort of action game like a shooter. You want to network this in a very simple way. Every frame you send the input from the client to the server (eg. keypresses, mouse input controller input), and each frame the server processes the input from each player, updates the simulation, then sends the current position of game objects back to the client for rendering.

So in our simple multiplayer game, whenever a packet is lost, everything has to stop and wait for that packet to be resent. On the client game objects stop receiving updates so they appear to be standing still, and on the server input stops getting through from the client, so the players cannot move or shoot. When the resent packet finally arrives, you receive this stale, out of date information that you don’t even care about! Plus, there are packets backed up in queue waiting for the resend which arrive at same time, so you have to process all of these packets in one frame. Everything is clumped up! "



Kailerra which Nestopia and other emulators used to help implement netplay has worked well. It uses UDP to provide a less laggy experience. We've used it for our online NES leagues to great success. It figures out the needed frame delay by taking a ping average or a certain period of time.

It has a server based client as well as direct peer to peer based client (the peer to peer) is the one we use. It does require people in the league to port forward in order to be able to host games. The kaillera C++ source is out there. You can see their reasoning for using UDP in the faq. The typical frame delay is 1-5 frames depending on where the two players are located.

In the dial-up days the frame delay was 11 frames which made for a very different gaming experience as you can imagine.

http://www.kaillera.com/faq.php

Now obviously kaillera was developed YEARS ago so maybe the same logic doesn't apply. The code for kailerra is open source however the comments are VERY sparse in the code and there is no high level document so you are left to mostly reverse engineer it.




I also helped work on getting the netplay working for an updated implementation of NES game on a different platform and most of their netplay options end up sending the information via UDP either in unreliable or reliable mode. I'm currently not at liberty to post the code for proprietary reasons but I don't think it would be much help for the emulator. It used the same basic principles I outlined in my first post.
Post Reply