granular synthesis analysis tool

Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music.

Moderator: Moderators

tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

lidnariq wrote:
tepples wrote:It might be easier just to fit a 2-pole linear predictive model to the sound using the Levinson-Durbin algorithm, determine its pole frequency, and find the noise pitch that has the closest pole frequency.
That would make sense... My background is more in DIP than audio
If the "I" in DIP refers to two-dimensional images, I might still be able to help you. Ever heard of deconvolution to reverse a blurring operation? If you have, I could explain what I mean using related terms.
The other problem is that the different values written to $400E have different loudnesses, too. Which is why I'm searching the entire 512-entry space of frequency and volume rather than just the 32-entry space of frequency.
I took differential equations, linear algebra and group theory in college. My professors always used to recommend solving problems by changing the domain: "go somewhere, do something else, and come back". The "somewhere" in question might be the Laplace domain, the z-transform domain, the Fourier domain, etc. In this case, you could 1. change the volume of the input sample such that the sum of squares matches that of your reference samples, 2. match against reference samples' power spectra, and 3. change the volume back.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

I can't get the necessary dependencies working on Cygwin at all. Would someone post some WAV or NES files demonstrating the program?
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Post by lidnariq »

Dwedit wrote:I can't get the necessary dependencies working on Cygwin at all. Would someone post some WAV or NES files demonstrating the program?
I thought I'd included them in the .7z -- the .wav files were converted and are in the noise.obj (rename it to noise.nes) -- they map as:

cymrsmp.wav - A
hat.wav - B
bd.wav - select
pwrsnare.wav - start

Note that all of these are not sampled at 48kHz, so all of them are erroneously pitch-shifted by my tool.
tepples wrote:If the "I" in DIP refers to two-dimensional images, I might still be able to help you. Ever heard of deconvolution to reverse a blurring operation? If you have, I could explain what I mean using related terms.
Sure! That'd be useful.
tepples wrote:In this case, you could 1. change the volume of the input sample such that the sum of squares matches that of your reference samples, 2. match against reference samples' power spectra, and 3. change the volume back.
Right -- the problem is that the highest frequencies of noise (0-3 = can be thought of us 4, 8, 16, and 32 point FIR ~lowpass all-ones filters) all have their first zeroes at frequencies so much higher than we can hear (447kHz, 224kHz, 112kHz, 56kHz respectively) that there's basically no useful attenuation in the audible spectrum to match those by anything except volume -- they're basically just white. (And in the 24kHz bandwidth my sound card can produce, they functionally are white)
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

I'd like to hear what the program outputs for drumloops.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Post by lidnariq »

Dwedit wrote:I'd like to hear what the program outputs for drumloops.
No good. It keeps the snare and hat sounds; roughly as expected, the bd and tom don't convert. Cymbals convert so-so.

http://eamp.org/noise.nes

now, in addition-
Down - sample # 9350 from freesound (http://www.freesound.org/samplesViewSingle.php?id=9350)
Left - sample #3455 from freesound (http://www.freesound.org/samplesViewSingle.php?id=3455)
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples »

Dwedit wrote:Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
In a traditional NES music engine, that would correspond to treating the cymbals as "instruments" and the hi-hats as "sound effects".

Anyway, here's the deal:
Linear prediction tries to find a filter that "predicts" the next sample given the last few samples. Analysis (used in compression) is like convolution, giving the signal a more "white" spectrum; synthesis (used in decompression) is like deconvolution, reconstituting the original spectrum. Once you have the filter, you can take its Z-transform to find the "poles" and "zeroes", which are related to which frequency bands a filter increases or decreases.
User avatar
neilbaldwin
Posts: 481
Joined: Tue Apr 28, 2009 4:12 am
Contact:

Post by neilbaldwin »

tepples wrote:
Dwedit wrote:Still cool how it sounds like it plays the cymbals and hi-hats simultaneously on the DOWN sample.
In a traditional NES music engine, that would correspond to treating the cymbals as "instruments" and the hi-hats as "sound effects".
I have my drums on a virtual track to achieve this effect. It's applied to all 4 voices though, not just noise. Everything gets rendered to virtual APU registers until the very end of the code where it's written to the hardware. That way I can write/overwrite/modify anything that gets written to the (virtual) registers so I end up with a snapshot of activity during each refresh.

The only problem I have is that there's no real priority system other than the numerical order of the tracks, which the virtual drum track is highest and therefore always takes precedence.

This discussion has given me some ideas of how to change my code to make it more flexible. :)
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Post by lidnariq »

To dig up this post again from the dead, bregalad was pestering me about it on the wiki, and I said "eh... needs to be rewritten in C"

So I did. It now deals correctly with sample rates other than 48kHz. It now assumes an "ideal" white spectrum by directly calculating the spectrum of the lowpass caused by the noise hardware rather than comparing to a sampled reference. The gain calculation seems incorrect, but it's the same one I was using before.

source code + demonstration rom:
preanalyze.zip
(3.72 KiB) Downloaded 163 times
noise.nes- samples play on A,B,select,start,down,left. Down and Left are a comparison of the old analyzer to the new one respectively.

To build this you'll need fftw3 and libsndfile. I don't have a windows machine to build this on but it should be much easier than fighting with PDL.

I release it to the public domain, or if you're in a place where that's not allowed, under the WTFPL.
Last edited by lidnariq on Sun Feb 23, 2020 4:19 pm, edited 2 times in total.
User avatar
blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

Post by blargg »

The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool. Thinking about the big picture, the following comes to mind:

With traditional percussion, composer uses those that best elicit the desired "feeling" when listening. Presumably the development of drums themselves was shaped by this as well, so we have drums that form a good sound palette.

With this tool, sound palette is drums put through tool, forming a sound palette that might not be optimal anymore. Traditional drum sounds are somewhat limited by the physics of drums, so the best palette using them is shaped by these limitations. If different limitations were present, a different palette might be better. Converting a palette based on the physical limitations of drums to the limitations of the NES is unlikely to give the best palette for the NES.

Making new drum sounds on the NES and tuning them until they sound good is likely to yield the best drum palette for a NES. Here, it's not their similarity to physical drums that's important (though that may play a role), it's just their effect on the listener that's refined.

Anyway, just a thought that came to mind when scanning this thread and listening to the demo.
User avatar
Bregalad
Posts: 8056
Joined: Fri Nov 12, 2004 2:49 pm
Location: Divonne-les-bains, France

Post by Bregalad »

Well I didn't ask for it to be converted to C, but thank you for doing it.
I tried to compile it under windows but failed miserably. I downloaded both libraries you are mentionning, one comes in the form of an installer and I'm not sure where it gets installed, and the other I was able to get. Anyways, gcc still says both .h files are missing and can't compile the thing.

(you know, no offense but not everyone here is a PHD in computer science).

@blargg : you have a perfect point. I'm more interested in this tool for curiosity than anything else. It could be used as an inspiration tool to make drums manually too (instead of "blindly" copy it's output).
You can make noise splash which doesn't sound like real drums at all but goes well along a song. Castlevania is a good example of this, it comes with 2 kind of noise splashes for all percussion, and this works well. Wizard & Warriors II would be another example of this.

Also, I think the drum kit in Ferrari Grand Prix Challenge (from Neil :) ) and Blaster Master sounds absolutely amazingly close to a real drum kit with kick, snare and hats. It's hard to make a crash cymbal without having the other drums stop playing unfortunately.
Useless, lumbering half-wits don't scare us.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Post by lidnariq »

blargg wrote:The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool.
Why do we need yet another DPCM converter? It's just another tool, this one unique. Does it produce perfect results? I'd be surprised if it ever did, but it can be informative and a good starting place.
Bregalad wrote:(you know, no offense but not everyone here is a PHD in computer science).
Sorry about the tone; you reminded me that I really shouldn't have left it as the pile of crap it honestly was.
User avatar
blargg
Posts: 3715
Joined: Mon Sep 27, 2004 8:33 am
Location: Central Texas, USA
Contact:

Post by blargg »

lidnariq wrote:
blargg wrote:The demo isn't really convincing of this approach being better than just making drums manually, though I may be missing the point of the tool.
Why do we need yet another DPCM converter? It's just another tool, this one unique. Does it produce perfect results? I'd be surprised if it ever did, but it can be informative and a good starting place.
Sorry, I wasn't meaning to bash your tool, I was just sharing a thought that came to mind as I read about it/listened to the demo. When I'm making things, I also consider them from a practical standpoint, without reference to their research/future value, because it often leads to new ideas that can make it more practical, rather than entirely research-oriented. Maybe I misunderstood it, but I took it to be converting drums to noise only, no DPCM. Thus I was talking about manually coming up with noise writes to make a drum, not DPCM at all.

I think your tool is interesting, and could easily be used as the basis for something. For example, converting a drum, then manually tweaking the output. Or using the code as a basis for one that converts things for other channels. I was just considering it in the context of converting drum samples without any further tweaking.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Post by Dwedit »

Demo program fails on FCEUX.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Post by lidnariq »

Dwedit, it works for me with FCEUX and all the other emulators I can get my hands on. Did it get downloaded erroneously as text?
Post Reply