It is currently Mon Oct 16, 2017 11:06 pm

All times are UTC - 7 hours



Forum rules


Related:



Post new topic Reply to topic  [ 95 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next
Author Message
 Post subject: Re: Graphics in Excel
PostPosted: Mon Apr 20, 2015 2:04 am 
Offline
User avatar

Joined: Sun Jul 01, 2012 6:44 am
Posts: 337
Location: Lion's den :3
For MSU1 audio to play, one major prerequisite has to be met (at least on real hardware like the sd2snes), which is to unmute the SPC700, and disable echo. Also, you have to wait for the Audio Busy bit to clear before setting the audio state to play.

Here's a code snippet I posted on another forum quite some time ago (the posting is in German, but the code itself should be self-explanatory, and has English comments). It's confirmed working on both higan and sd2snes. LMK if you want the ROM and/or full source, which I'd have to dig up from somewhere. :wink:

As for the data port, it essentially works the same, except that it's up to you how to process the streamed data, of course. A good reference is e.g. ikari's video player, sources are included in these (snes/msu1 subfolder).

_________________
Some of my projects:
Furry RPG!
Unofficial SNES PowerPak firmware
(See my GitHub profile for more)


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Mon Apr 20, 2015 4:21 pm 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
Mmmkay, so I tried adapting ikari's video player code to my program and nothing at all happened. So I tried using your code and I'm getting somewhere because it's crashing Bizhawk now - which I've learned happens when it hits a WAI command and never gets an interrupt, which due to the way I hacked it out means it's taking the "NoMSU" branch. So, I haven't made it recognize that I'm trying to use MSU1 yet. Bizhawk is just a shell for bsnes v87, so it definitely should work, right...?

I decided to try it in higan and magically, it works. Works on my SD2SNES as well. So I guess somehow Bizhawk lacks MSU1 support despite using bsnes.

I'm grateful for the help trying to get this put together but I can't believe that the actual specification page on byuu's site that all this knowledge seems to come from is just plain gone. Without your help I'd have never figured it out. Did I miss something? I was just reading some heated conversations right on this forum no more than a year ago getting the fine details of it worked out. Seems like an exciting new development like this should have resulted in a bunch of interesting new hacks in the meantime yet MSU1 barely even brings in a handful of google results. Do I dare start relying on it for more than mere demos?

(Speaking of those conversations, did they ever reach some kind of agreement about the audio frequency/clock speed? I was extremely concerned to see the issue of video and audio de-syncing brushed aside as unimportant - a tiny desync becomes absolutely intolerable very quickly.)

Anyways. The audio is hideously out of sync the way I have it written now... It starts playing several seconds before the video begins to load. As far as I can tell from my code, the first frame should appear on screen 4-5 frames after the audio begins playing, so I'm assuming there's some kind of long buffering delay or something from loading the audio data. When looking at ikari's video player it looked like he had it set up to start the audio as the first frame is drawn, but I'm not sure that's any better if you still have a significant delay right after the audio starts.

Was trying to take a closer look in higan but I'm shocked to discover it doesn't even seem to have a frame counter so I can see how much of a delay I'm even dealing with... Why oh why doesn't it just work in Bizhawk. I think I may have to go bother adelikat about that, if he's forgiven me for the last time I wasted his time.

The video is looking amazing though. On my TV it's like 3/4 of an 8x8 tile row away from fullscreen, you can hardly tell it isn't a full picture. Frame processing is at ~200/2682 after maybe a day running full speed. Gives me time to figure out the data stream, though I am worried I won't have a processor left when it's done.

EDIT: Oh, I see:
Code:
   lda #$01   ; Set audio state to play, no repeat.
   sta MSU_CONTROL
   lda #$00
-   inc a
   wai
   sta MSU_VOLUME
   cmp #$FF
   bne -
   ; The MSU1 will now start playing.
   ; Use lda #$03 to play a song repeatedly.

So it's taking a bunch of frames to gradually ramp up the volume. I guess I can build that into my normal frame loop, if I even need it.

EDIT: Audio is nicely synced and I got rid of that annoying little line of pixels at the bottom by blanking both tilemaps first. Literally all I need now is to learn to DMA frames from MSU1 and I've got it.


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Apr 23, 2015 3:10 pm 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
Frame processing is approaching 50%. I really need to speed things up somehow but I'm already a little disturbed by the quality I'm getting out in places. Certain scenes have really large areas of almost-but-not-quite the same colour and it looks to me like those just become terrible the moment you reduce them to the SNES's maximum colour depth, as the smooth gradient of colour becomes a splotchy broken mess. This also kind of highlights the tilemap grid for those brief moments, which is less than ideal.

I haven't attempted any manual correction for this effect yet. In certain scenes it might be as easy as changing one or two palette colours, I will give it a try later.

98% of the video looks great though. I enjoy how the MSU1 just keeps on reading data after it reaches the end of the file - on emulator it does nothing but on SD2SNES it just starts playing the other ROMs on the card as though they were video, which is very very colourful and poooossibly seizure-inducing I'm not sure. I'll patch it out when I'm done but for now I'm enjoying the show.

...

Serious question: I'm pondering making use of the MSU1 for the other main project I've been working on. I don't dare to dream that it could someday be worthy of burning into a cartridge, but I'd like to keep that possibility open just in case I actually finish something for once. I'm afraid to start planning on it though without knowing how practical the MSU1 is to actually build.

It sounds straightforward enough to me, but it doesn't sound like the intended normal-cartridge implementation of MSU1 has been built yet. I got the impression it is supposed to be simpler than what ikari had to do for SD2SNES. I've recently discovered that I live near people with the means to build high quality reproduction cartridges, out of newly made parts instead of reclaimed used stock (and accurately labelled as reproductions to boot). I'm eager to discuss the possibility of MSU1 support with them, but I'd like to know what I'm getting into first from a hardware perspective.

P.S. I am aware this thread hasn't been about Excel for a long time now and I am sorry


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Apr 23, 2015 3:19 pm 
Offline

Joined: Sun Apr 13, 2008 11:12 am
Posts: 6273
Location: Seattle
Real-world hardware that can ONLY stream video or audio at once is easy. Real-world hardware that can do both from the same media becomes a royal PITA.

A more practical option might be putting two compactflash cards into a single cart, one for audio, one for the CPU. Or maybe better would be a compactflash card for the CPU and one of those MP3-player-on-a-chips to stream audio.

(P.S. PM tepples a split point)


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Apr 23, 2015 3:31 pm 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
lidnariq wrote:
Real-world hardware that can ONLY stream video or audio at once is easy. Real-world hardware that can do both from the same media becomes a royal PITA.

Wait, so, the way MSU1 functions in higan and SD2SNES is not how it's intended to work? Or do I misunderstand? Are you saying that you can't DMA data to VRAM using the MSU1 while a track is playing? Or did they deliberately emulate the maximum attainable functionality regardless of how difficult it would be to implement? I mean - how bad is the SD2SNES, when it has to not only stream data and audio simultaneously, but then write SRAM back to the same card too?

I think I'm starting to understand what byuu was talking about now with the "open-ended design allowing for flexibility in deployment" or however it was worded. It makes sense but it's a bit of a pain to not just have a full blown chip design that explicitly lays it out for you. I guess nothing worth doing is easy...


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Apr 23, 2015 4:03 pm 
Offline

Joined: Fri Jul 04, 2014 9:31 pm
Posts: 781
The data and audio are streamed from two separate memory spaces, right?. So in a real system you could well have two different physical storage devices for them - in fact that would seem to make the most sense. The sync issue was dealt with (I think it was dealt with...) by making the audio timing track the SNES master clock, which implies that the MSU1 itself does no internal A/V synchronization, which implies that the hardware interfaces for data and audio don't have to be tightly coupled. It seems to me that this means you just need 2x"easy" instead of 1x"royal PITA".

At worst you could just copy what the SD2SNES does, with an FPGA addressing a single moderately high-bandwidth memory device. Since save games wouldn't also have to go on said device, the reset-to-save hack wouldn't be necessary any more... mind you, I have no idea what the components would cost...

Ramsis wrote:
disable echo

Why?


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Fri Apr 24, 2015 1:49 am 
Offline

Joined: Sun Apr 13, 2008 11:12 am
Posts: 6273
Location: Seattle
Khaz wrote:
Or did they deliberately emulate the maximum attainable functionality regardless of how difficult it would be to implement?
That.

Basically, in order to be able to stream two independent things at the same time from the same medium requires a lot of RAM and a fairly large amount of logic glue. By promising both, it's kind of a pain to implement something similar without either
1- using two separate pieces of media, one just for audio, one just for the CPU
2- or a comparatively large FPGA with a large amount of RAM.

Quote:
I think I'm starting to understand what byuu was talking about now with the "open-ended design allowing for flexibility in deployment" or however it was worded.
"flexibility in deployment" means "difficult to deploy". If the interface were instead just "Here's a CF card/deserialized SD card/whatever" it would, at absolute worst, condemn the hardware instantiation to irrelevance in a decade or two.

That's why the SegaCD / PC-Engine CD-ROM interface is hairier than the MSU1 one; it reflected the hardware rather than being abstracted away.


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Fri Apr 24, 2015 1:55 am 
Offline
User avatar

Joined: Sun Jul 01, 2012 6:44 am
Posts: 337
Location: Lion's den :3
93143 wrote:
Ramsis wrote:
disable echo

Why?

Sorry, no clue whatsoever. :P

_________________
Some of my projects:
Furry RPG!
Unofficial SNES PowerPak firmware
(See my GitHub profile for more)


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Mon Jun 01, 2015 3:29 pm 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
So I recently learned about some functionality in Excel I wasn't previously aware of, that inspired me to actually make a more useful tilemap editor. I think what I came up with is going to save me a lot of time, so I thought I'd share it.

There are a few macros to be used in the same way as my previous spreadsheets:

ReadMap: Reads a text file of a tilemap (.dw formatted), either 32x32 or 64x64, and loads it into the MAP EDITOR tab.

WriteMap: Writes the tile map currently in the MAP EDITOR tab to a text file in .dw format.

PaletteShift: Gives three different ways to manipulate the palette numbers of every tile on the sheet. Run it and you'll get a popup window explaining how it works more specifically.

DrawEntireMap: IF you have a tile set in the TILE SET tab (the same way the tile sets are in my original spreadsheets - it reads the cell colours directly), then this macro will render the tile map according to the tile set, and output the results into MS Paint. You must have Paint open for this to work. (The image will also be drawn into another excel worksheet, but you should delete that worksheet as soon as it's done. Not every computer can seem to handle excel when the cells are at minimum size like that.)

UndoLastChange: Undoes the last edit to the map, SEE NEXT SECTION:

The fun part of this is that I discovered you can program macros to be executed on certain events within Excel. In this case, I have macros set to run when you click on cells within the MAP EDITOR worksheet!

The top part of the sheet is the 64x64 map itself. If you left-click on a cell, the value in "LEFT CLICK:" down below will be written to it. If you right-click, the value in "RIGHT CLICK:" gets written. You can drag to select a range of cells, and they will all be assigned the value in LEFT CLICK. You can't drag a range of cells with the right mouse button, but you CAN select a range with the left button (which will change it to LEFT CLICK), and then right-click that selection to change it to RIGHT CLICK.

You can change the values of LEFT and RIGHT CLICK manually using the other fields at the bottom: The box to the bottom left is your tile numbers (right now it's built for 16x16 tiles so there's no odd numbered rows or columns, sorry, can work on that). Click the boxed, numbered cells to set H/V inversion, palette and priority. Alternately, you can copy in a number that's currently on the tilemap by clicking "< COPY TILE INTO LEFT (or RIGHT) CLICK", then clicking the tile on the tilemap. The "COPY" cells will highlight themselves in red to show when they're active.

Finally, since this functionality requires disabling the normal on-click behaviour, there's also a big "DISABLE MACROS" button. When it turns red and says "MACROS DISABLED", everything returns to normal Excel until you click it again. The main thing this allows you to do is select a block of cells on the tilemap, copy it and then paste it elsewhere.

Finally, there's the UNDO STACK. Since Excel can't "undo" a macro action, and since it's very easy with this method for a stray click to screw up a map, I implemented my own undo feature. Every time you click or drag on the tilemap and make a change, it's stored in the UNDO STACK. Whenever you run UndoLastChange, it undoes the last change (the first item in the list). The keyboard shortcut is currently set up as CTRL+U, but you can change that yourself under Macros, Options...

If people have feedback I would welcome it. My continued and sincere apologies to the non-Excel crowd. I figure after a bit more touchup work, I should go back and edit the first post of this thread to put everything in one place...

EDIT: To-do list:
-8x8 tilemap support
-improve drawing (tile set/palette reading/importing, and direct file output because this method is stupid but it was just kinda proof-of-concept)
-Merge with tile editor spreadsheet?

Not that anybody is interested here, but I've realized a few more improvements on this version and I'm currently revising it. The next one should be much better in many ways.


Attachments:
File comment: Tile Map Editor for Excel
MapReader.xlsm [314.78 KiB]
Downloaded 75 times
Top
 Profile  
 
PostPosted: Thu Jun 04, 2015 1:59 pm 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
Okay. Massively overhauled the map editor since my last post. Summary of the new state of things:

ImportTileMap, ExportTileMap, PaletteShift, UndoLastChange are unchanged (aside from the names).

DrawMapToBitmap: Does direct-to-bitmap output now, but is curiously somehow slower than my previous stupid method. It's about a 3MB file so maybe excel is just really slow at one-byte-at-a-time binary writes. It takes just under an hour at worst but I can't think of any reasonable way to speed it up further.

DrawTileSet: New macro; Renders the TILE SET sheet according to one of the palettes in the Palettes sheet.

ImportPalettes, ImportTileSet: New macros; Imports palettes or a tile set from a text file. ImportPalettes requires them to be formatted specifically (.dw where each line is one palette). Tile Set just reads in data (either .db or .dw) until the sheet is full or until the end of the input file is reached.

The MAIN overhaul though was how the Map Editor functions. The annoying click-cells-to-control-stuff region at the bottom is gone, replaced by a userform! Double-click any cell in the "MAP EDITOR" worksheet to bring up the "Map Tools" window (or alternately run the "ShowMapTools" macro manually). This has all the same functionality as before but in a separate little window so you can move around the map more freely. Brief summary follows:

Each mouse button has a Tilemap-Word associated with it. LEFT and RIGHT CLICK buttons at the top determine which mouse button's Tilemap-Word you're editing with the other controls on the panel. Left click a cell on the map to assign it the value of LEFT CLICK; right-click a cell to assign it the value of RIGHT CLICK. Drag-select a range of cells to assign them all the value of LEFT CLICK. Right click that selection to change them all to RIGHT CLICK.

When you click the COPY TILE button, then select a cell on the worksheet, the contents of that cell will become the new RIGHT- or LEFT-CLICK word, depending on which button is toggled at the top. UNDO just runs UndoLastChange (as does Ctrl+U).

DISABLE MACROS works the same as before, to disable editing mode so you can copy/paste cells and interact with the sheet normally. Also note that the click-to-edit functionality on the MAP EDITOR will ONLY work while the MapTools window is open. When the window is closed, everything returns to a normal Excel spreadsheet.

I decided against merging the two spreadsheets for now, just for the sake of less work and keeping things more organized. I'm satisfied for my own purposes again, so I'm only going to take on a task like that if other people are interested. As always, let me know if you find it useful and if you have problems with it I'm happy to help.


Attachments:
File comment: Updated Tile Map Editor
MapToolsV2.xlsm [548.67 KiB]
Downloaded 84 times
Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Sep 03, 2015 9:39 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19084
Location: NE Indiana, USA (NTSC)
Khaz wrote:
Code is now on Github.

Code:
if bytelist[0] != b'B' or bytelist[1] != b'M':
    print ("This is not a bitmap.  Aborting.")
    quit()

Is there a reason why you parse Windows bitmaps manually rather than using Pillow (Python Imaging Library)?


Top
 Profile  
 
 Post subject: Re: Graphics in Excel
PostPosted: Thu Sep 03, 2015 10:03 am 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
tepples wrote:
Khaz wrote:
Code is now on Github.

Code:
if bytelist[0] != b'B' or bytelist[1] != b'M':
    print ("This is not a bitmap.  Aborting.")
    quit()

Is there a reason why you parse Windows bitmaps manually rather than using Pillow (Python Imaging Library)?

Yes, the reason is that this program was originally written in Excel VBA, where I don't think I had a choice but to decode the bitmaps manually. When I made the transition to Python I was basically only concerned with translating what I already had to that language. Also, this is the first time I have heard of Pillow.

Considering that the parsing of the bitmap takes a trivial amount of time in the process I'm not too concerned if it's not done the most efficient way. If anyone happens to find a way to speed up the actual quantization loop without losing quality though, that would be huge and please tell me.

In general if anyone wants to take my program and improve it, please, by all means help yourself. It is my only Python project and I'm sure I did a ton of things the stupid way.


Top
 Profile  
 
PostPosted: Thu Sep 03, 2015 10:25 am 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5707
Location: Canada
PIL (or Pillow) is simply a widely used image library for python. It's not part of its standard library, you have to download and install it separately.

Writing a simple BMP parser is comparable to the effort required to install PIL, so I think this is a wash. :P


Top
 Profile  
 
PostPosted: Thu Sep 03, 2015 10:33 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19084
Location: NE Indiana, USA (NTSC)
I imagine that it's a lot easier to \Python27\scripts\pip install Pillow (see full instructions) or sudo apt-get install python-imaging (if on Debian or Ubuntu) than to write your own parser for PNG, JPEG, and compressed BMP.


Top
 Profile  
 
PostPosted: Thu Sep 03, 2015 10:37 am 
Offline
User avatar

Joined: Thu Dec 25, 2014 10:26 pm
Posts: 309
Location: Canada
tepples wrote:
I imagine that it's a lot easier to \Python27\scripts\pip install Pillow (see full instructions) or sudo apt-get install python-imaging (if on Debian or Ubuntu) than to write your own parser for PNG, JPEG, and compressed BMP.

Yeah, I'd imagine you're right. I chose simple uncompressed bitmap as an input format specifically because it's so easy to decode and I already knew how to...


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 95 posts ]  Go to page Previous  1 ... 3, 4, 5, 6, 7  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: kevtris, noko_bombette and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group