Musepack Forums

Musepack Forums (https://forum.musepack.net/index.php)
-   Development (https://forum.musepack.net/forumdisplay.php?f=11)
-   -   Improved seeking for mpc files (https://forum.musepack.net/showthread.php?t=276)

snowgoon 24 March 2006 10:28 am

Improved seeking for mpc files
 
Does anyone have any thoughts on the seeking implemented in libmpcdec (or in_mpc, etc.)?

I'm thinking of testing an improvement to the seeking in mpc files on Rockbox. The key features of this "improvement" are:

1. Use a/the seek table as much as possible
2. Don't use fseek when jumping forward to the required frame (when the seek table is empty), just fill the buffer as normal (I'm not sure my assumption that a few "large" freads are faster than lots of small fseeks and small freads will hold up in Rockbox). Does anyone know?
3. Only decode the scale factors in the n previous frames (I found n=64 is good) and skip over the rest of the frame - this will be faster than decoding all of the frame
4. Set any scale factors that haven't been reset to -128 which avoids any loud artifacts (if using a smaller n).

I've tested all this in (yikes!) C# and it seeks to work ok, and will eventually go ahead and test it in Rockbox (if I can find a MS version of cygwin so I can compile it ;-)).

Any thoughts appreciated.

Lefungus 24 March 2006 07:29 pm

In the very first version of libmpcdec (when it was still called libmusepack), a seek table hack was implemented to speed up seeking. Unfortunately, it was also a source of random artifacts, even with a very large number of "preroll" frames (I don't remember the numbers). Most of the time, nothing was heard, but sometimes you could get horrible sounds. Eventually, the whole thing was removed.

However, if you happen to know how to reduce or attenuate those artifacts, that would be a very nice improvement, especially on portable devices.

In an ideal world, you would post a patch against SVN version of libmpcdec, and people would review/test it :)
Libmpcdec is used on both PC and Rockbox (with some tunings/modifications).

About the fseek versus fread issue, any reading on a portable device draws powers, so I assume less is better.
However, it's useless to read too small buffers, you need to tune it against the disk reading granularity.

Shy 24 March 2006 09:35 pm

I'll send you a file to test this on to see if the reduction of the screeching works properly.

snowgoon 24 March 2006 11:34 pm

Thanks for the replies. This method worked a treat on the test file. Thanks Shy.

Not sure if I need to explain the bit about setting scale factors to -128 - I meant scale factor indexes for each subband. The sign bit indicates if the scale factor for that band has been reset. If it hasn't then scale factor 128 is used (instead of some random index) ensuring that the subband is at least very quiet.

snowgoon 25 March 2006 10:53 am

After playing around with this a bit more I found that the maximum length of frame dependencies was 31 (ie 32 frames linked), and from the look of the original seek and encoder code this is the designed limit. And after retesting the few files I've got on hand (don't ask) I found that n=32 (prior frames) works fine without the need to set the scf indexes to -128. However setting n<32 results in (loud) artifacts without using the -128 hack. Maybe the artifacts heard with n=32 (or n>32) were caused by bugs in the seeking code? (I found heaps in my own code). I will look a bit more...

Whether this is correct or not, I stand by my other suggestions as possible improvements to the seeking implementation... Now I just have find some time to write some patches :-)...

stel 25 March 2006 03:35 pm

It sounds like you are making good progress snowgoon.
I'd like to see mpc seeking implemented on Rockbox, and willing to help with the testing.
I play around with a bit of development in C, so I might be able to help here to.
Let me know if I can be of any help.

snowgoon 29 March 2006 04:11 am

I have some important updates on the seeking issue. Sorry if the following is a bit technical, but this is development thread after all!

Inspired by death (!), I decided to test the accuracy of the decoder output after seeking. To test it I decoded an mpc file to wav by seeking to each individual frame from some arbitrary starting point and comparing the result to the output without doing any seeking. I found that only 33 out of 43548 frames in the test file were correctly decoded. The problem is that the synthesis doesn't stabilize until after the first frame is decoded after seeking. To fix this, I started the SCF scanning at -33 frames instead of -32, and then requantize and resynth the last frame before the seek point. After rerunning my test this resulted in only 16 invalid frames (a second test file decoded perfectly).

Suspecting a bug in the encoder I went look for one, and found several dubious lines relating to the scale factor encoding. After several hours of debugging nothing seemed to make a difference until I noticed that the differential SCF runlength counters are only 8 bits!

I therefore have two new patches - a new version of the seeking patch for libmpcdec and one for the sv7 encoder. Together these ensure 100% correct decoding after seeking.

Lefungus 29 March 2006 05:35 pm

Patchs attached would be nice. Even better, patchs using "diff -u".

[edit]
And do you really mean bit-accurate decoding after seeking to frame n, with both patchs applied ?

snowgoon 29 March 2006 11:16 pm

I mean bit accurate. This was after about 43548 (x2) seeks in my test file, ie. one to a random point in the file and one to the frame being decoded, for every frame.

I will work on the diff patches, can someone give me ftp access? Or tell me if I can attach files to posts?

[edit]
The relevant patched files are in:

http://snowgoon.musepack.net/experimental/sv7_patch.zip
http://snowgoon.musepack.net/experim...ch_v1_2_3b.zip

snowgoon 30 March 2006 12:53 pm

I have updated the libmpcdec version to be rockbox friendly:

http://snowgoon.musepack.net/experim...ch_v1_2_3c.zip

and a version for rockbox itself:

http://snowgoon.musepack.net/experim...tch_1_2_3c.zip

Sorry, still no diffs... :-)

The Rockbox version seems to work reasonably on my iAudio, put it's not perfect. Takes about 20 seconds or so to seek forward 4-5 minutes in a song, but <1 second when seeking <3-4 minutes forward... Strange. Maybe some yields() would help?

Comments, suggestions or complaints are welcome :P

preglow 31 March 2006 10:26 am

Quote:

Originally Posted by snowgoon
The Rockbox version seems to work reasonably on my iAudio, put it's not perfect. Takes about 20 seconds or so to seek forward 4-5 minutes in a song, but <1 second when seeking <3-4 minutes forward... Strange. Maybe some yields() would help?

I don't think adding any yields would helpat all. The seeking logic is in the codec thread, after all, and letting other threads execute while you're trying to seek shouldn't help performance.

But anywho, I think this sounds really great, and I'm looking forward to trying it out when I have more time.

stel 31 March 2006 07:15 pm

Wow snowgoon, you work fast.
I've just compiled the rockbox version for use on the iRiver H320 and initial tests seem good. I can seek to 6-7 minutes without having a long deley. Going >7 minutes the delay before playback starts increasing. I had to wait approx. 20 seconds to seek 14minutes.

IMHO, a worthwhile addition.

snowgoon 01 April 2006 11:32 am

A couple of new versions:

http://snowgoon.musepack.net/experim...ch_v1_2_3d.zip
http://snowgoon.musepack.net/experim...tch_1_2_3d.zip

Fixes:
* id3v2 tagged files now play and seek (no, they're not my files - I borrowed them from a friend ;-))
* hopefully faster big endian seeking (and maybe faster decoding too).

I can't find a solution to the slow seeking problem with large seeks - I may or may not keep looking :-)

snowgoon 02 April 2006 12:34 pm

I've added the -128 hack to the patches to avoid anyone damaging their ears when playing back incorrectly encoded files. I've written a tool to repack mpc files to correct the scale factor bug, in which case the -128 hack won't be needed, but it needs more testing to ensure that it doesn't affect the output bitstream of a sequentially decoded file - ie. mpc files decoded to wav should be the same before and after repacking. Is there anything already around to do mpc bitstream conversion?

http://snowgoon.musepack.net/experim...tch_1_2_3e.zip
http://snowgoon.musepack.net/experim...ch_v1_2_3e.zip

snowgoon 03 April 2006 08:30 am

Update on the slow seeking on Rockbox:

The problem occurs when the filebuf goes empty, ie. if the seek point is not within the api's buffer. Once the buffer is empty the api starts filling the buffer in 16k chunks. The problem is reduced significantly by setting the chunk-size to 512k before the seek, however I don't think this is the optimal solution.

http://snowgoon.musepack.net/experim...tch_1_2_3f.zip

snowgoon 18 April 2006 09:38 am

Patch submitted to Rockbox here: http://www.rockbox.org/tracker/task/5172

stel 18 April 2006 07:52 pm

Snowgoon,

Have you done what I think you've done with the latest patch?
It looks like it's optimised to the point where I'm getting as good as if not lower/better boost ratio with mpc as I do with mp3
MPC - 5%
MP3 - 7%

Still testing, but seeking seems to be spot on.
Excellent work, I'm definitely one happy user.

:D

snowgoon 19 April 2006 12:49 am

Cool! I'm glad someone is happy ;-) I suspect there are a couple of bugs in the patch as it doesn't work on iPod mini (apparently).

Changes made:

- lookup tables for huffman decoding, like in the standalone decoder
- cache the next dword of the bitstream
- added IRAM attributes to the lookup and huffman tables

Moos 19 April 2006 12:35 pm

Man, you made great works, and for sure you'll make more than one user happy : )
We are plenty of Musepack Rockbox users, and potentially more and more everydays.

Preglow (our musepack Rockbox man) will try to commit your patch in Rockbox sources and this way make *lot* of people happy.

Please let us know if you succed to improves the things again.
You can join us on Rockbox IRC, and I'm sure you'll have feed-backs+some ways to take for improve your patch.

Thanks again man, hopefully your patch will be commited really soon.

R.A.F. 12 August 2006 10:31 pm

Can someone compile Snowgoon´s improvements and upload them somewhere here (inofficially to the Rockbox-homepage)? - As it seems, that this patch won´t be implemented after V3.0 is released.


All times are GMT. The time now is 01:12 pm.

Powered by vBulletin® Version 3.8.11 Beta 2
Copyright ©2000 - 2019, vBulletin Solutions Inc.