Personal View site logo
Make sure to join PV on Telegram or Facebook! Perfect to keep up with community on your smartphone.
Please, support PV!
It allows to keep PV going, with more focus towards AI, but keeping be one of the few truly independent places.
GH2 - 2.5fps AVCHD 24h Timebuster/TimeHBusteR 2.0 settings - The day is not over!
  • 323 Replies sorted by
  • driftwooddriftwood

    All of the fb2s should be frame limit divided by 8. Find the ideal frame limit first ;-)

    Forget FB1s for now. Forget SBs. Concentrate on IQ/Q, Frame Limit, FB2s.

    Also, don't bother with Bitrate settings, ptools handles that based on your AVCHD Compression setting.

  • duartixduartix

    Sharing a most spectacular Failure.

    I thought I'd share it because there are a few things to learn from the analysis and because it was really really close to working. My first approach for TimeHBusteR was the same as Timebuster 1.3: Marrying the GOP to shutter speed, getting rid of B-Frames and trashing P-Frames. All in all it looked pretty simple:

    1080i60 GOP Size=12

    Encoder setting 1 1080i/p=2

    1080i Scaling B=0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF

    The IQ wasn't without all sorts of issues, reduced resolution, horizontal blocking and there were these very small but noticeable and very annoying interlacing artifacts. And it was a spectacular failure because only today I understood how close it came to working but also how it can never work...

    So what's happening here? First since this is an interlaced stream, there is something unusual: Only one of the slices of the I-Frames is encoded as I (#12/1), the other is encoded as P(#12/2). Now the P-Frames are real trash as expected, but in an unexpected twist, the P-Slice that's accompanying the I-Slice (EDIT) should be really good with a fixed 22 QP. This should give the frames of interest for timelapsing good IQ, right? I'm on the beach right now, but then it hits me when I look at it on StreamEye...Half the picture is a P-Slice and most of that is either skipped blocks or inter blocks, predicted from... yes, from a trashed P-Frame. So I die on the beach. :( And so should interlaced!!!

    (update) Probably the P-Slice is using the Scaler Table for P-Frames and that is a more likely reason for all the IQ issues I mentioned before...

    P.S. Today I made some slow but steady progress, thanks @driftwood .

    (edit file attached)

    failure.png
    1802 x 874 - 598K
  • duartixduartix

    Cheers,

    This is a sketch of what TimeHBusteR Base could be, isn't not close to final, but it serves to point the direction. It's got @Lpowell 's scaling tables, the biggest Frame Limit that makes sense, GOP married to 1/2.5s (in both 24p and HBR, later it will only target HBR, but for now that I'm still heavy testing, it's like this) and tries to use a QP=20 with an initial QP=24

    Now does anyone have an idea why the quantizer in the video tests settles on 23 after the first second and I can't get it any smaller? The frames don't seem limited and they grow quite healthily as I raise the ISO (this has been most of what I've been testing, I'm trying to get some numbers into an Excel sheet that will ultimately will spit out which QP to use depending on: recording mode, footage characteristics, target ISO, pretended recording length and card size.

    I don't have a clue why the quantizer on the footage is not 20 as requested... :(

    setj.zip
    585B
  • driftwooddriftwood

    IQ should should be 20. then lower Q accordingly.

  • duartixduartix

    Thanks Nick @driftwood, but I'm a bit lost on that one... You mean Initial Quantizer should be 20?

  • driftwooddriftwood

    yeah always start from there.

  • duartixduartix

    I'm almost 99% sure I tried QP=IQ=20 last night without results, but I'll re-check it tonight with something like QP=18, IQ=20.

    Thanks Nick.

  • LPowellLPowell

    @duartix 20 is the default for IQ, 720 QP, and 1080 QP, so if you enter it manually, you should see no change in behavior.

  • duartixduartix

    @lpowell Thanks for the tip Lee, that hadn't crossed my mind. I've got the camera here with me, I'm uploading a new set with QP=16, IQ=20.

    And...not much luck! :(

    I just changed Quantizer for 1080 modes to 16 and Initial Quantizer to 20. Do I need to mess with 720p QP too?

    • Frame--Type QP---Min---Max--Range---DC Skipped QST-High

    00000 (IDR| P ) 28 28 28 1 9 1535

    00001 ( P | P ) 28 28 28 1 9 4415

    00002 ( P | P ) 28 28 28 1 9 4797

    00003 ( P | P ) 27 27 27 1 8 4574

    00004 ( P | P ) 27 27 27 1 8 5261

    00005 ( P | P ) 26 26 26 1 8 4918

    00006 ( P | P ) 26 26 26 1 8 5581

    00007 ( P | P ) 25 25 25 1 8 4560

    00008 ( P | P ) 24 24 24 1 8 4216

    00009 ( P | P ) 24 24 24 1 8 4993

    00010 ( P | P ) 24 24 24 1 8 5322

    00011 ( P | P ) 24 24 24 1 8 2023

    00012 ( I | P ) 23 21 25 5 7 1184

    00013 ( P | P ) 23 23 23 1 7 3470

    00014 ( P | P ) 23 23 23 1 7 3999

    00015 ( P | P ) 23 23 23 1 7 4507

    00016 ( P | P ) 22 22 22 1 7 3371

    00017 ( P | P ) 22 22 22 1 7 4088

    00018 ( P | P ) 22 22 22 1 7 4584

    00019 ( P | P ) 22 22 22 1 7 4978

    00020 ( P | P ) 22 22 22 1 7 4985

    00021 ( P | P ) 22 22 22 1 7 5320

    00022 ( P | P ) 22 22 22 1 7 5235

    00023 ( P | P ) 22 22 22 1 7 1533

    00024 ( I | P ) 22 20 24 5 6 923

    00025 ( P | P ) 22 22 22 1 7 3048

    00026 ( P | P ) 22 22 22 1 7 3366

    00027 ( P | P ) 22 22 22 1 7 3756

    00028 ( P | P ) 22 22 22 1 7 4136

    00029 ( P | P ) 22 22 22 1 7 4549

    00030 ( P | P ) 22 22 22 1 7 5037

    00031 ( P | P ) 22 22 22 1 7 4949

    00032 ( P | P ) 22 22 22 1 7 5294

    00033 ( P | P ) 22 22 22 1 7 5510

    00034 ( P | P ) 22 22 22 1 7 5554

    00035 ( P | P ) 22 22 22 1 7 1733

    00036 ( I | P ) 22 20 24 5 6 938

    (edit) Maybe it's the initial value, I'm changing them respectively to 19/17 and retesting...

  • duartixduartix

    No success...

    It's not affecting HBR nor 24p.

    QP is still reported as 22 for P-Frames and 20-24 for I. :(

    Can't see what else I need to change...

  • LPowellLPowell

    @duartix If the encoder doesn't think it will have enough bitrate to support your manual QP setting, it will force the encoded QP up to a level it can support. 20-24 is in the default QP range for the encoder.

  • duartixduartix

    @LPowell : OK, it makes sense, so how can I "convince" it into thinking it has enough bitrate (because it does have, and much more than it needs). Does it involve any other settings than the ones I changed? Is it something as simple as raising the related modes bitrate? Like Video Bitrate FSH/FH or 24H/L?

  • LPowellLPowell

    @duartix Sorry, this is where you get into tricky issues of balancing bitrate, buffer sizes, encoder settings, and reliability under stress. Welcome to the pitfalls of patch development!

  • duartixduartix

    Unfortunately this is where I usually start copying other people's settings and adapting to my needs. :(

    Ironically, reliability which is probably the biggest PITA, is the least of my worries.

    Now, I've raised Video Bitrate 24H/L/FSH/SH/FH/H, Video Buffer, Video Buffer 24p, 1080p24 FB2, 1080p24 Frame Limit, 1080p24 High Top/Bottom Setting, 1080i Top/Bottom Setting, Other Modes Top/Bottom Setting and...

    NADA! StreamParser is still reporting QP=23... WTF??? This is what the setings look like.

    seth.zip
    639B
  • duartixduartix

    @LPowell : One last question before I give up on this... Is there any plausible reason for me to pursue an I-Frame QP lower than 22? Or wouldn't anyone notice the difference in IQ?

  • duartixduartix

    Well, it took me a good part of the weekend to figure out what the problem is and why the QP doesn't stick... Surprisingly it's related to GOP.

    Since I can't get my simple settings to work (there's an immediate freeze in the video recording as if the camera was doing WB bracketing in video) so I tried a lot of other people's VBR settings on Q18. They all work fine at Q18 but I'm trying to change the 24p GOP to 5 (347º shutter on 24p and 360º on HBR25 at 1/5s) and 1080i GOP to 12 (360º shutter on HBR30) and that's where the problems start. :(

    I tried @balazer 's Cake 95 and as soon as I set 24p GOP=5 or 1080i GOP=12, the QP doesn't stick anymore and goes on vacation to a constant 22.

    I tried @bkmcwd 's natural_v1.11. Setting 1080i GOP=12 makes the QP on HBR30fps go up to 27-28 on I and 29-30 on B. However when setting the 24p GOP=5 it does respects the QP in 24p.

    Finally I tried @lpowell 's FlowMotion 1.11 and these are the only settings that respect the QP in both 24p and HBR30fps.

    Does anyone have an idea what the problem could be? I mean, I'm using a 1/2.5 shutter speed in these tests. It's not full intra, there's a lot of temporal redundancy for either B or P or both to do their efficiency magic, so bitrate and buffers shouldn't be challenged... Why doesn't the QP work as it should?

  • LPowellLPowell

    @duartix I don't think PTool 3.64d works well with GOP lengths that are not divisible by 3. This was one of the reasons I changed the 720p GOP from 5 in Flow Motion v1.0 to 6 in Flow Motion v1.1.

    When you set a manual QP for 1080 or 720 modes, the encoder will attempt to use it but will resort to a higher QP if it predicts that it lacks the bitrate to handle it. Since you're manipulating P and B-frame sizes in a non-standard way, the encoder may think it needs considerably more bitrate than it actually does.

  • duartixduartix

    That would be a plausible reason @Lpowell , but I hadn't yet done my stuff on the Scaling tables. It was just QP=18 and GOP=5|12.

    Well, I'll be trying that now, I'll see what happens. As a matter of fact I'll have to weight using B-Frames as a bitrate saviour since I cannot trash P-Frames because of the dual I/P nature of the keyframes in 1080i.

  • balazerbalazer

    Have you tried an auto quantizer setting?

  • duartixduartix

    Cheers @balazer!

    No I haven't, it's against my religion... ;)

    But I already know what's breaking it and it's not nice. It's not GOP either after all (well at least not alone)... I set Encoder Setting 1 1080i/p = 2 on @lpowell 's FlowMotion to supress B-Frames and the QP blew up! Surely it couldn't be B-Frames! But it was! I took Cake 95, set it to use B-Frames by Encoder Setting 1 1080i/p = 3 and QP obligingly respected my autoritah.

    Why? I have no idea. The bitrate wasn't being taxed, so I suppose the buffers should also be healthy... Anyway, due to the interlaced nature of the keyframes encoding, I'm condemned to use good P-Frames in HBR, so this time it will be the B-frames' turn to be trashed.

  • duartixduartix

    24Timebuster Base 1.4 is near ready and so is timeHBusteR Base (I guess I'll start at version 1.4 also).

    I'm running some tests but it's all looking good so far. The GOP will now be married to the sweet spot at 1/2.5s SS based on my motion fluidity tests. There was a ~6% sacrifice in bitrate in 24p by using B-Frames to respect the QP plus a ~20% sacrifice to accommodate 1/2.5s (instead of 1/2s), however this can now be gained precisely by setting a little higher QP so it will still record 24h.

  • duartixduartix

    Releasing 24h Timebuster 2.0 and 24h TimeHBusteR 2.0!

    Due to some respectable user demand suggesting to keep timelapse on 24p but because my motion fluidity tests suggested otherwise (using HBR modes), 24h Timebuster 2.0 is now comprised of 2 main variants:

    a) 24h Timebuster 2.0 - with the following changes (from TimeBuster 1.3):

    • The GOP for 24p is now married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. As a benefit it's still very close to 2fps and it's the closest to a 360º shutter we can get on 24p.

    b) 24h TimeHBusteR 2.0 (NEW, uses HBR 30)

    • Similar philosophy to 24h Timebuster 2.0 Base but as the name suggests, uses HBR mode instead of 24p. There is another difference, P-Frame quality is fully preserved as this kind of frames is used for one of the interlaced slices on Key Frames.

    • GOP married to 1/2.5 on HBR 30p (NTSC) and to 1/5s on HBR 25p (PAL). The big benefit is that both are using a 360º shutter. Uncheck "1080i50 and 1080p24 GOP Size=5" before merging if you don't plan to use Shutter Speed (SS) =1/5s and want to better preserve the merger's 24p mode and HBR 25.

    Both variants are now:

    • Using B-Frames. There is a 6% penalty in bitrate usage but this was necessary to make sure the QP was respected. I believe the gain from setting the QP at will is just too great to ignore, since not everyone wants to record for 24h straight. QPs as low as QP=16 were tested and theoretically it could go as low as QP=12 @ISO160 when merged with FlowMotion 1.11 on regular tripod mounted footage.

    • GOP married to 2.5fps. The bitrate hit is a little above 20% but the recording length can be easily compensated by raising the QP. The reason behind this was getting as close as possible to 360º.

    • User definable GOP - GOP can be set at will according to intended shutter speed (SS), use the attached "GOP-SS-table.png" to look up the right value. Defaults settings are SS=2.5fps and GOP 1080i=10 for 24h Timebuster 2.0 and SS=2.5/5fps and GOP 1080i50/i60=12/5 for 24h TimeHBusteR 2.0.

    • User definable QP - QP can be set at will according to desired IQ/recording length. I've gathered quite an amount of information so that in the future I can make an Excel tool to estimate the best QP from Card Size/Footage Snapshot/Desired ISO/Recording Length. Default settings are Q24/23 for 24h Timebuster/TimeHBusteR.

    As before, there is the Base Release to be merged at wish with popular settings (load Base first in PTTools and then "ALT+Click" your choice of merger) and there is also the FlowMotion Release (already merged with FlowMotion 1.11). I've merge tested against @lpowell 's FlowMotion 1.11, @balazer 's Cake 95, and @bkmcwd 's GOP3ZILLA and no undesirable effects were observed. The key is using a VBR settings definition with a very high "1080p24 Frame Limit", especially if you are going for a low QP (<22). If you want to merge with Nick @driftwood 's settings, I advise to go for the Sedna Q20 Pack.

    The day is not over. :)

    24h Timebuster 2.0.zip
    2K
    24h TimeHBusteR 2.0.zip
    3K
    GOP-SS-table.png
    867 x 439 - 51K
  • aljimenezaljimenez

    Thank you for sharing your work @duartix

  • RawBRawB

    Thanks Duartix for all the hard work and time you spend in making these new Time(H)Buster patches! I will give them both a try and then choose my favorite. When I have a nice TimeLapse made I will show it in this topic. Maybe other can do the same. -Cheers-

  • duartixduartix

    Attention@all!

    24h Timebuster (the 24p version) seems borked with some heavily posterised frames (mostly on moving subjects). However you can still make full use of it (there are always full quality frames at the same spots within in each GOP but you have to be careful when blending, to make sure you ignore the bad ones.

    Done some extra testing last night and it looks like a mix off causes that mostly can't be avoided whilst keeping the settings core functions. In the HBR version (24h TimeHBusteR), even though it also happens as small interlaced lines (also on moving subjects), the effect is almost unnoticeable when compared to the 24p variant. This happens basically because the frame changes aren't in full synch with the I-Frames in the GOP. They are in synch in frequency but not in displacement, so when a change occurs, it's encoded with a heavily quantised P/B-Frame and it only gets encoded with proper IQ when the next I-Frame arrives, luckily around 1-2 frames later, however it means it cant be immediately played as it came from the camera. This also happened to some extent @2fps but it wasn't as noticeable and nasty in Timebuster 1.3 when there wasn't any B-Frames.

    I'll be studying ways to minimize this issue, and the lesson learned is that I will never test again against static footage alone. :(

Add a Comment