Musepack Forums  

Go Back   Musepack Forums > Main > Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 24 March 2006, 10:28 am   #1
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default Improved seeking for mpc files

Does anyone have any thoughts on the seeking implemented in libmpcdec (or in_mpc, etc.)?

I'm thinking of testing an improvement to the seeking in mpc files on Rockbox. The key features of this "improvement" are:

1. Use a/the seek table as much as possible
2. Don't use fseek when jumping forward to the required frame (when the seek table is empty), just fill the buffer as normal (I'm not sure my assumption that a few "large" freads are faster than lots of small fseeks and small freads will hold up in Rockbox). Does anyone know?
3. Only decode the scale factors in the n previous frames (I found n=64 is good) and skip over the rest of the frame - this will be faster than decoding all of the frame
4. Set any scale factors that haven't been reset to -128 which avoids any loud artifacts (if using a smaller n).

I've tested all this in (yikes!) C# and it seeks to work ok, and will eventually go ahead and test it in Rockbox (if I can find a MS version of cygwin so I can compile it ;-)).

Any thoughts appreciated.
snowgoon is offline   Reply With Quote
Old 24 March 2006, 07:29 pm   #2
Lefungus
Procrastinator
 
Lefungus's Avatar
 
Join Date: Jul 2004
Posts: 131
Default

In the very first version of libmpcdec (when it was still called libmusepack), a seek table hack was implemented to speed up seeking. Unfortunately, it was also a source of random artifacts, even with a very large number of "preroll" frames (I don't remember the numbers). Most of the time, nothing was heard, but sometimes you could get horrible sounds. Eventually, the whole thing was removed.

However, if you happen to know how to reduce or attenuate those artifacts, that would be a very nice improvement, especially on portable devices.

In an ideal world, you would post a patch against SVN version of libmpcdec, and people would review/test it
Libmpcdec is used on both PC and Rockbox (with some tunings/modifications).

About the fseek versus fread issue, any reading on a portable device draws powers, so I assume less is better.
However, it's useless to read too small buffers, you need to tune it against the disk reading granularity.
Lefungus is offline   Reply With Quote
Old 24 March 2006, 09:35 pm   #3
Shy
Admin
 
Shy's Avatar
 
Join Date: Jul 2004
Posts: 369
Default

I'll send you a file to test this on to see if the reduction of the screeching works properly.
Shy is offline   Reply With Quote
Old 24 March 2006, 11:34 pm   #4
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

Thanks for the replies. This method worked a treat on the test file. Thanks Shy.

Not sure if I need to explain the bit about setting scale factors to -128 - I meant scale factor indexes for each subband. The sign bit indicates if the scale factor for that band has been reset. If it hasn't then scale factor 128 is used (instead of some random index) ensuring that the subband is at least very quiet.
snowgoon is offline   Reply With Quote
Old 25 March 2006, 10:53 am   #5
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

After playing around with this a bit more I found that the maximum length of frame dependencies was 31 (ie 32 frames linked), and from the look of the original seek and encoder code this is the designed limit. And after retesting the few files I've got on hand (don't ask) I found that n=32 (prior frames) works fine without the need to set the scf indexes to -128. However setting n<32 results in (loud) artifacts without using the -128 hack. Maybe the artifacts heard with n=32 (or n>32) were caused by bugs in the seeking code? (I found heaps in my own code). I will look a bit more...

Whether this is correct or not, I stand by my other suggestions as possible improvements to the seeking implementation... Now I just have find some time to write some patches :-)...
snowgoon is offline   Reply With Quote
Old 25 March 2006, 03:35 pm   #6
stel
Member
 
Join Date: Mar 2006
Location: UK
Posts: 3
Default

It sounds like you are making good progress snowgoon.
I'd like to see mpc seeking implemented on Rockbox, and willing to help with the testing.
I play around with a bit of development in C, so I might be able to help here to.
Let me know if I can be of any help.
stel is offline   Reply With Quote
Old 29 March 2006, 04:11 am   #7
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

I have some important updates on the seeking issue. Sorry if the following is a bit technical, but this is development thread after all!

Inspired by death (!), I decided to test the accuracy of the decoder output after seeking. To test it I decoded an mpc file to wav by seeking to each individual frame from some arbitrary starting point and comparing the result to the output without doing any seeking. I found that only 33 out of 43548 frames in the test file were correctly decoded. The problem is that the synthesis doesn't stabilize until after the first frame is decoded after seeking. To fix this, I started the SCF scanning at -33 frames instead of -32, and then requantize and resynth the last frame before the seek point. After rerunning my test this resulted in only 16 invalid frames (a second test file decoded perfectly).

Suspecting a bug in the encoder I went look for one, and found several dubious lines relating to the scale factor encoding. After several hours of debugging nothing seemed to make a difference until I noticed that the differential SCF runlength counters are only 8 bits!

I therefore have two new patches - a new version of the seeking patch for libmpcdec and one for the sv7 encoder. Together these ensure 100% correct decoding after seeking.
snowgoon is offline   Reply With Quote
Old 29 March 2006, 05:35 pm   #8
Lefungus
Procrastinator
 
Lefungus's Avatar
 
Join Date: Jul 2004
Posts: 131
Default

Patchs attached would be nice. Even better, patchs using "diff -u".

[edit]
And do you really mean bit-accurate decoding after seeking to frame n, with both patchs applied ?
Lefungus is offline   Reply With Quote
Old 29 March 2006, 11:16 pm   #9
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

I mean bit accurate. This was after about 43548 (x2) seeks in my test file, ie. one to a random point in the file and one to the frame being decoded, for every frame.

I will work on the diff patches, can someone give me ftp access? Or tell me if I can attach files to posts?

[edit]
The relevant patched files are in:

http://snowgoon.musepack.net/experimental/sv7_patch.zip
http://snowgoon.musepack.net/experim...ch_v1_2_3b.zip
snowgoon is offline   Reply With Quote
Old 30 March 2006, 12:53 pm   #10
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

I have updated the libmpcdec version to be rockbox friendly:

http://snowgoon.musepack.net/experim...ch_v1_2_3c.zip

and a version for rockbox itself:

http://snowgoon.musepack.net/experim...tch_1_2_3c.zip

Sorry, still no diffs... :-)

The Rockbox version seems to work reasonably on my iAudio, put it's not perfect. Takes about 20 seconds or so to seek forward 4-5 minutes in a song, but <1 second when seeking <3-4 minutes forward... Strange. Maybe some yields() would help?

Comments, suggestions or complaints are welcome :P
snowgoon is offline   Reply With Quote
Old 31 March 2006, 10:26 am   #11
preglow
Member
 
Join Date: Mar 2006
Posts: 1
Default

Quote:
Originally Posted by snowgoon
The Rockbox version seems to work reasonably on my iAudio, put it's not perfect. Takes about 20 seconds or so to seek forward 4-5 minutes in a song, but <1 second when seeking <3-4 minutes forward... Strange. Maybe some yields() would help?
I don't think adding any yields would helpat all. The seeking logic is in the codec thread, after all, and letting other threads execute while you're trying to seek shouldn't help performance.

But anywho, I think this sounds really great, and I'm looking forward to trying it out when I have more time.
preglow is offline   Reply With Quote
Old 31 March 2006, 07:15 pm   #12
stel
Member
 
Join Date: Mar 2006
Location: UK
Posts: 3
Default

Wow snowgoon, you work fast.
I've just compiled the rockbox version for use on the iRiver H320 and initial tests seem good. I can seek to 6-7 minutes without having a long deley. Going >7 minutes the delay before playback starts increasing. I had to wait approx. 20 seconds to seek 14minutes.

IMHO, a worthwhile addition.
stel is offline   Reply With Quote
Old 01 April 2006, 11:32 am   #13
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

A couple of new versions:

http://snowgoon.musepack.net/experim...ch_v1_2_3d.zip
http://snowgoon.musepack.net/experim...tch_1_2_3d.zip

Fixes:
* id3v2 tagged files now play and seek (no, they're not my files - I borrowed them from a friend ;-))
* hopefully faster big endian seeking (and maybe faster decoding too).

I can't find a solution to the slow seeking problem with large seeks - I may or may not keep looking :-)
snowgoon is offline   Reply With Quote
Old 02 April 2006, 12:34 pm   #14
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

I've added the -128 hack to the patches to avoid anyone damaging their ears when playing back incorrectly encoded files. I've written a tool to repack mpc files to correct the scale factor bug, in which case the -128 hack won't be needed, but it needs more testing to ensure that it doesn't affect the output bitstream of a sequentially decoded file - ie. mpc files decoded to wav should be the same before and after repacking. Is there anything already around to do mpc bitstream conversion?

http://snowgoon.musepack.net/experim...tch_1_2_3e.zip
http://snowgoon.musepack.net/experim...ch_v1_2_3e.zip
snowgoon is offline   Reply With Quote
Old 03 April 2006, 08:30 am   #15
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

Update on the slow seeking on Rockbox:

The problem occurs when the filebuf goes empty, ie. if the seek point is not within the api's buffer. Once the buffer is empty the api starts filling the buffer in 16k chunks. The problem is reduced significantly by setting the chunk-size to 512k before the seek, however I don't think this is the optimal solution.

http://snowgoon.musepack.net/experim...tch_1_2_3f.zip
snowgoon is offline   Reply With Quote
Old 18 April 2006, 09:38 am   #16
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

Patch submitted to Rockbox here: http://www.rockbox.org/tracker/task/5172
snowgoon is offline   Reply With Quote
Old 18 April 2006, 07:52 pm   #17
stel
Member
 
Join Date: Mar 2006
Location: UK
Posts: 3
Default

Snowgoon,

Have you done what I think you've done with the latest patch?
It looks like it's optimised to the point where I'm getting as good as if not lower/better boost ratio with mpc as I do with mp3
MPC - 5%
MP3 - 7%

Still testing, but seeking seems to be spot on.
Excellent work, I'm definitely one happy user.

stel is offline   Reply With Quote
Old 19 April 2006, 12:49 am   #18
snowgoon
Member
 
Join Date: Mar 2006
Posts: 12
Default

Cool! I'm glad someone is happy ;-) I suspect there are a couple of bugs in the patch as it doesn't work on iPod mini (apparently).

Changes made:

- lookup tables for huffman decoding, like in the standalone decoder
- cache the next dword of the bitstream
- added IRAM attributes to the lookup and huffman tables
snowgoon is offline   Reply With Quote
Old 19 April 2006, 12:35 pm   #19
Moos
Member
 
Join Date: Apr 2006
Location: Paris
Posts: 1
Default

Man, you made great works, and for sure you'll make more than one user happy : )
We are plenty of Musepack Rockbox users, and potentially more and more everydays.

Preglow (our musepack Rockbox man) will try to commit your patch in Rockbox sources and this way make *lot* of people happy.

Please let us know if you succed to improves the things again.
You can join us on Rockbox IRC, and I'm sure you'll have feed-backs+some ways to take for improve your patch.

Thanks again man, hopefully your patch will be commited really soon.
Moos is offline   Reply With Quote
Old 12 August 2006, 10:31 pm   #20
R.A.F.
Member
 
R.A.F.'s Avatar
 
Join Date: Feb 2005
Location: Nuremberg / Germany
Posts: 7
Send a message via ICQ to R.A.F.
Default

Can someone compile Snowgoon´s improvements and upload them somewhere here (inofficially to the Rockbox-homepage)? - As it seems, that this patch won´t be implemented after V3.0 is released.
R.A.F. is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mppenc 1.16/libmpcdec 1.2.3 Seed General 8 11 December 2006 03:03 am
How to restore / convert some files? danieldc MPC for Windows 3 02 October 2005 08:23 pm
All MPC files broken after file restore deus62 MPC for Windows 3 19 August 2005 05:55 pm
How to re-encode mpc files at reduced quality? drmrbrewer General 6 31 July 2005 07:28 pm
mpc files 48khz replay davidm MPC for Windows 1 07 February 2005 08:02 pm


All times are GMT. The time now is 02:16 pm.


Powered by vBulletin® Version 3.8.11 Beta 2
Copyright ©2000 - 2017, vBulletin Solutions Inc.