Personal View site logo
Windmotion - upsampling to 4:4:4
  • zserozsero

    I've just seen this post over at doom9 forums about a guy making a super high quality filter to upsample to 4:4:4 16-bit. Especially for GH2 / GH3 owners. Trying to invite him over here. http://forum.doom9.org/showthread.php?p=1610007

  • 263 Replies sorted by
  • OedipaxOedipax

    Interesting. Maybe I'll fire up a Windows 7/8 VM and try it out.

  • tetakpataktetakpatak

    Bravo zsero, huge potential could be hidden in there. I hope the guy will join here soon, many would like to test it and important feedbacks would be returning to the guy all the time (btw, that forum by doom9 does not seem to be very bubbly.....two weeks without any reactions)

  • spacewigspacewig

    @zsero Have you installed and tried it?

  • zserozsero

    No, I'm waiting for the developer to come up with it's new release and to support Cineform 10-bit as well. At the moment it supports the following formats:

    4:4:4 in RGB avi utvideo

    4:4:4, 4:2:2, 4:2:0 YUV H.264, 10bit H.264 mkv

    4:2:2, 4:2:0 YUV ProRes mov

    4:2:2, 4:2:0 avi utvideo

    4:4:4 16/8 bit internal format (3 YUV avi utvideo files with wav)

  • OedipaxOedipax

    It could stand to have some documentation. I got a Windows 8 VM running, installed the necessary apps/codecs/scripts, but you end up with a folder full of batch files for each individual process and not much idea of how to go about processing footage. Plus several of the scripts aren't included in the beta because they are for the paid pro versions.

    Edit: Nevermind, I see he already has a GUI going in the newer version. Good luck figuring out the alpha ;)

  • matt_gh2matt_gh2

    @zsero I'm not too familiar with 4:4:4. So what is the benefit of upsampling your footage to 4:4:4?...is it about having finer gradations of color correcting the colors in our images? Is this similar to setting your After Effects timeline to 32 bit vs 8 bit in order for images to hold up better when grading?

    So how does it work? Do you upsample your footage to 4:4:4 first and then import into your video editor like Premiere Pro? Thanks

  • reanrean

    Hello, I am the developer. zsero invited me here. The program is not for upsampling really, but it may be used for this thing. It is planned as any camera quality restoration tool. I used some internal versions within last two years for my own commercial videos from a low bitrate camera (17mbps H.264 4:2:0) and currently with Sony NEX-7 (28mbps H.264 1080p50). Currently I have planned to share it with a community. So a new version is developing with some useful things.

    Restoration includes:

    • H.264/MPEG/MJPEG codec artifact fixing (dithering, deblocking, motio-compensated denoised and other tricks).
    • Other camera things fixing including color cast, digital noise, halos, oversharping, etc).

    Also it is useful as a service tool to rename and batch convert original clips from a camera and from a sound recording device. Internally I use it to make films. It supports 1080p by a design. And... upsampling is really a small additional feature.

    I tested an oldest internal version with many videos, from the internet, including latest Sony cameras, Panasonic x900, GH2, GH3, Canon and Nikon DSLR. I always get great visual results even with a 100mbps videos from hacked GH2. Sorry, I cannot show you any result now, because have no GH2 and licensed clips.

    Currently the program is under development. Really I working every day a week with it. After a stable beta version will be ready, I will publish examples, screenshots, videos, html documentation, etc.

    If you have a question, ask be here. But I am really very busy now, because hard coding Windmotion now, so I apologize in advance for not being able to answer multiple questions or clarifications.

    I don't recommend anyone to download Alpha 2 version now, because you will get a next published version very soon. It will be a big different to use. Also, I would be grateful for permission to use your video from hacked GH2 or other cameras. I am particularly interested in high bitrate samples. These samples will be used to create examples and a documentation.

  • tetakpataktetakpatak

    @rean Welcome here, mate! Your project might find lots of resonance and interest here. Just to make sure:

    I always get great visual results even with a 100mbps videos from hacked GH2. Sorry, I cannot show you any result now, because have no GH2 and licensed clips.

    Did you mean with "videos" original out-of-camera MTS files from hacked GH2?

  • zserozsero

    @matt_gh2, yes, you do it before, since once you've imported it into Premiere and edited, this information is lost. So you do this similarly to how you do this in 5DtoRGB or in Cineform. It's not about the floating point thing, it's about rescaling the channels.

    About the actual theory, I don't feel qualified to tell anything, I'd better redirect you to these articles:

    http://en.wikipedia.org/wiki/Chroma_subsampling

    http://blogs.adobe.com/VideoRoad/2010/06/color_subsampling_or_what_is_4.html

  • zserozsero

    Did you mean with "videos" original out-of-camera MTS files from hacked GH2?

    I'm sure that's what he means!

    @rean, I believe from this forum you'll find lots of volunteers to send you all the different clips from GH2s! Especially the ones which give problems to others, like the "diagonal rain" in Premiere.

  • reanrean

    Did you mean with "videos" original out-of-camera MTS files from hacked GH2?

    Of course, I found some originals from Vimeo Plus downloads (MTS, mov) and from other forums. Currently I have a huge collection of many camera original files. They are from unknown license, so I cannot use them to share. I have my own videos from my NEX-7, but, anyway, I will publish any final examples after a stable release. Because internal algorithms are still under development and real finish quality is not stable now. Also huge lossless video needs a lot of server space to keep these files. So I don't want to publish any results now. Currently the program is available for public less than 3 weeks...

  • zserozsero

    @rean, I've posted it in two high-bitrate hack topics, where people suffer from a "diagonal digital rain" issue. I'm sure they'd be happy to share originals with you.

    http://www.personal-view.com/talks/discussion/4828/driftwood-settings-experimental-series-2-an-boom-intravenus-cluster-v8

    http://www.personal-view.com/talks/discussion/5758/gh2-flow-motion-v2-100-mbps-fast-action-performance-series-2

  • BurnetRhoadesBurnetRhoades

    Reading about this amazing sounding software, the new Metabones adapter, RED's plans for indie 4K distribution to theaters nationwide all in one week's time...am I dreaming?

  • spacewigspacewig

    @rean Have you compared the results of restoring hacked with unhackd GH2 footage?

    I'd be happy to share original MTS files that you can then publish. Just let me know how to get it to you.

  • reanrean

    @spacewig No, I have no compare. I have no GH2. Restoring of an "original" image is not possible, so results will looking different in compare. Unable to get information where it is not available (using a lossy codec). However, the image can be adjusted closer to the feel of the feelings from the real image. There are a variety of ways. Widely known is, for example, Dithering, to masking low-level digital artifacts. Missing bits of information in the program are obtained by interpolation using motion prediction and interpolation using the noise reduction. Aliasing is removed with the use of algorithms based on neural network training. And so on.

    Since all this is very difficult to develop alone, I use the GPL code, already implemented on a lot of Avisynth plugins. Therefore, the program has its own specific limitations, related to limitation of all these components. I added the methodology for applying the algorithms, using my personal experience to handle video and photos.

    About sharing. Thank you. I will contact you in PM.

  • richg101richg101

    Watching this topic. sounds interesting.

  • reanrean

    Some results from Sony FS100 ($6,550.00 U.S. List Price) on PreAlpha3 (version in development):

    Chroma channel artifacts (before, after and difference):

    links are removed

    Here the processing includes: H.264 predication errors fixing (28mbps), black level correction, small color correction, non-halo-producing micro-contrast sharp, denoise, dithering, conversion from 4:2:0 to 4:4:4.

  • bmorgan83bmorgan83

    @rean, might want to put some forewarning that the advertising on the site you linked those images may not be work safe.

  • BurnetRhoadesBurnetRhoades

    ... the advertising on the site you linked those images may not be work safe.

    Heh, some of it might not be legal!

  • zserozsero

    Yes, that's quite a disgusting site, @rean, I'd recommend you to use imgur.

    A very handy open source program to upload all kind of things to image sharing websites (as well as for lot of other ones as well) is ShareX: https://code.google.com/p/sharex/

  • MeierhansMeierhans

    WOW! Is there any chances to get your code inside a After Effects plugin? Your results look promising, but the fact that you need to apply the "effect" before your start grading means one would either need to convert all his raw footage, or do another round trip after edit lock, convert all used clips, replace... Both is not really nice way to work. Another question would be how does the Avisynth noise reduction compare to Neat Video? In case Neat Video is better, could you chain it in between at the right place?

  • zserozsero

    @Meierhans,

    If he'd integrate it into AE, it'd all be limited to whatever decoder AE uses. For example look at the "digital rain" problems, the Adobe decoder produces this bug. Maybe Adobe will fix it, maybe not. The only workaround is not to use Adobe as a decoder, but something like 5DtoRGB.

    So this tool is an alternative to 5DtoRGB / Cineform HD Link, not a plugin. It can use whatever decoder it needs, if we find one to be buggy, @rean can just change the decoder. Not with Adobe.

    Also, I think it's perfectly fine to import a footage into an editing friendly format (what is not h264). In my opinion the best is Cineform, since it's free and it's AVI based, unlike ProRes and DNxHD, both what are MOV based. So you can have a totally QuickTime free workflow on windows (a big big advantage!).

  • BurnetRhoadesBurnetRhoades

    How is being Quicktime free an advantage in Windows? Is there an equivalent to Quicktime reference movies in any AVI player? That alone would prevent me from ever being Quicktime free in Windows.

  • reanrean

    Heh, some of it might not be legal!

    Oops, sorry. I have moved it to another site. zsero, thanks for the recommendation.

    Is there any chances to get your code inside a After Effects plugin?

    No. It is impossible to use it "inside" and also as a plugin due Avisynth architecture limitations. Also different stages (~20) are used with a special visual control for every stage. Сurrently the program is for prepoduction only.

    Probably you can export your project files, then import it into Windmotion, process it there and get it back again. What a file format After Effect uses for project files? Probably I can create import and export for it.

    Warning. It is not a correct way to use Windmotion, but you can try it for your old projects. The new projects should use this scheme:

    1. Get a *.mts source.
    2. Import it into Windmotion.
    3. Proccess it in a batch mode.
    4. Export it to a file format you need for your software.
    5. Your own tasks.
    6. Remove any temporary files.
    7. Keep original *.mts source and Windmotion project files (~3 kb per a source) to a backup disk array or a tape.

    Because Windmotion uses text files for projects, there is no needs to keep temporary and end results. You can restore them later in any time.

    To restore a project in a future:

    1. Restore all *.mts source and Windmotion project files.
    2. Repeat all previous steps (just mouse clicks and waits for the result. No any editing!)

    Also, if you have a mistake in any stage, you can try to correct it later. There is no any "source destroy" stage, and a need to repeat all the steps again like in editors.

    Windmotion has a command principle, used in a node-based video software. I have no reason to use a editing software like Adobe or plugins, because I cannot change any middle stage later. Using “editors” is not a correct way for video production now. All the future video software will be command- or node- based.

    Another question would be how does the Avisynth noise reduction compare to Neat Video? In case Neat Video is better, could you chain it in between at the right place?

    Avisynth has no noise reduction. It is a platform for other plugins. Currently I use different plugins and special algorithms to use them specially for the program tasks. Also Windmotion has a possibility to choose a better way to denoise in every special case. It is very creative stage. It is not a simple solution "click and get a result".

    So it is a Neat Video alternative solution. You can try it to get a compare. For me personally, I really have no reason to use Neat Video, because Windmotion suits me for any noised sources I have.

  • zserozsero

    @Burnet:

    Is there an equivalent to Quicktime reference movies in any AVI player?

    It's called frame serving, and there is more than one product for it. My favourite is the Debugmode Frameserver: http://www.debugmode.com/frameserver/

Add a Comment