BMD Pocket Cinema Camera Stereoscopic Collaboration

_gl

After the success of our old Panny GH Collaboration, let's see if we can sync two BMD Pocket Cinema Camera (BMPCC) for stereoscopic 3D. I've got two on backorder (ETA 2 weeks), so I'll be able to run sync tests. If you're interested in shooting quality 3D on the Pocket or have one to try stuff on, join in!

The Pocket is only $500 USD / £390 UKP this summer (very limited availability, buy now!) .

Frame Sync is the big problem:

Syncing two cams for 3D cannot be done in post - they need to be synced at the pixel level, ie. each camera should begin each frame at exactly the same time. In other words, this is a fractional-frame problem; but you can only correct whole frames in post (not counting pixel interpolators like Twixtor which can cause unacceptable artifacts in 3D).

Pro cameras are synced with 'Genlock' - basically a connection between the two cameras that tightly synchronises frame capture. Without this, the cameras have a random gap between frame starts, anywhere up to 1/2 frame. So if you shoot 24p, the cameras will be randomly out up to 1/48th of a second, and you have no way to predict what that error will be at any given time. In 3D this is catastrophic, at least if there is any significant motion in the frame. The delay pushes moving objects forwards or backwards in 3D, and can also cause a weird shimmering effect. And forget high motion which looks awful.

Solutions:

1) hardware mod: do true genlocking by linking the cameras' capture timers or master clocks somehow at the circuit board level). This is perfect, but tricky/risky and requires skill and experience.

2) Simul-power workaround (this is what we use on the Panny GHs, and works on many others): you power-up both cameras at exactly the same time (usually with external power and a dual-switch or a custom circuit). Many cameras synchronize their frame capture from a global running clock, which is initialised at boot and never changes. So it doesn't actually matter when you trigger record - the first frame capture is always locked to that clock (ie. it waits for the next frame 'slot' before starting recording). By simul-powering the bodies, those clocks start in sync on both bodies and frames are synced.

we don't know if this works on the Pocket yet.

Downside: as no two digital clocks run at exactly the same speed, the bodies slowly drift out of sync. How fast depends on how closely matched the bodies are - people usually try to get consecutive serial nos., hoping that the eletronics components are likely from the same batch and so a close match. My badly matched (2 month apart) GH2's are good for about 4-5 mins before the drift becomes bad. Apparently some drift far less. So you have to keep power-cycling.

Problem: The Pocket only powers up when a button is pressed ('soft' power). It needs to power instantly as soon as external juice is on (one idea, permanently tape the power button down - does it work?).

3) LANC. The Pocket apparently responds to the LANC 'record' command (but not power cmds). If we're really lucky and this resets the frame capture timers (unlikely), it might be enough.

4) Firmware:

If the capture timers are NOT reset by capture start, BMD could add this to a firmware update. Then LANC-initiated record would always start in sync (subject to gradual drift) - much nicer than having to power-cycle all the time. This might be super-easy for BMD to do.

Of course some kind of true hardware sync option (maybe via the USB or LANC ports) would be best! BMD could probably do it if they wanted to - contact them if you want this: http://www.blackmagicdesign.com/uk/support (without demand it's unlikely to happen):

Exposure / Zoom sync:

Frame sync is the most important, but it would also be much nicer if exposure (iris, ISO, shutter-speed), focus etc settings could also be synchronised, ideally in real-time (one body acts as master, one as slave). Firmware or LANC command additions could do this.

With the pocket's small size, ultra-light and compact SBS (side-by-side) 3D rigs are possible. With 'body stacking', you can even get the lenses as close as around 70mm! With Speedboosters and fast lenses you can do lowlight and shoot wide (my target), and you have 10bit ProRes and 12bit RAW. Let's make it happen!

David_Wilson

Very very good idea to begin this effort. We bought two BMPCCs pretty much as soon as they were released, primarily for stereo work. These are wonderful little cameras. If handled with care, they are capable of making beautiful images. There are, however, certainly limitations to the BMPCC and (at least for our purposes) the ability to control shutter (frame) sync is one of them.

As GL suggests, the form factor of the camera is good for mounting side by side. There are 1/4-20 mounting points directly over and under the lens mount so building a side by side mount for the camera is pretty straight forward. We also have built a beam-splitter rig that works quite well. But the shutter sync issue remains...

Powering the cameras simultaneously does seem to work in our experience and we have produced some quite good results. We have no plans, however, to take the BMPCC stereo rig to the horse-races any time soon.

With regard to GL's thoughts, some things we've observed:

A permanent press on the power button does not work in that a long press on the power button cycles the camera on and then immediately off.

It is certainly possible to start the cameras simultaneously with LANC commands but sadly, the LANC start command does not reset the camera's internal clock. It would be wonderful if BMD were able to write that capability into a future firmware update.

The work-around we have been using for shutter sync is through the use of the Applied Logic DUAL CAMERA LANC CONTROLLER http://www.appliedlogiceng.com/index_files/Page1431.htm.

This board is seems very well conceived, engineered and fabricated and is relatively inexpensive at 240 USD. The board offers control of Zoom out, Zoom in, Focus Auto/Man, Focus Near, Focus Far, Photo Capture, Record Start/Stop, and Power On, and Power Off. Sadly, of course, the Power On, and Power Off does not work with BMPCCs. The board does not shutter-sync the cameras but through a set of LEDs one can tell when the camera is within 5 milliseconds which we find is good for most applications this side of a horse-race.

Very much looking forward to others thoughts and observations on the possibility of syncing these splendid little cameras.

_gl

Great info David.

OK, so taping the power button down won't work. Except that maybe if it is held before the camera gets power, it might not make it shutdown (depends how they programmed their button scanning). Could you try that?

So, LANC cannot reset the clocks, but it can be used to measure sync differences. That's very useful. I've not used it before (I will program an AVR to try it), but I've just been told that the cameras determine the protocol timing (makes sense), so I guess you can measure that from a custom controller and so figure out real-time sync. I will try that too.

There may yet be some way to get simul-power to work. Or maybe something else resets the clocks, like changing lenses, framerates etc.

If the firmware can reset the capture clock in theory, then BMD could enable this with almost no work. I'd be very happy to beta test firmware like that - does anyone have a good BMD contact? John Brawley?

_gl

A second idea - as the power button is 'soft', something is already running when the camera is off. It may just be an interrupt that wakes the CPU, but it's possible that the master clock is already running from the moment you connect power. If so, then just delivering juice to the bodies, even when switched off, may be syncing them at that point.

On the GH's, David Cole found that simul-power sync was only reliable with manual lenses. The bodies do some kind of lens calibration at boot with native lenses (probably focus and/or iris), and that probably introduces a random delay (which also suggests that the capture clock is only initialised after this calibration, so the firmware can apparently reset the clock).

We'll need to see if this is true for the Pocket aswell.

David_Wilson

Good idea to try pressing the power button and then applying power to the camera. The results are different but unfortunately not so helpful in that on applying power with the on-off button depressed, the camera simply remains off. In addition, the camera does not turn on when releasing the button. The camera seems to only turn on with a quick pressing of the on-off button with power already applied. The camera is shut down with a long pressing of the button.

With regards to David Cole's discoveries concerning manual vs automatic lenses with the GH, we seem to find that we are able to achieve equally passable sync with both manual lenses and a pair of automatic Lumix 14mm pancake lenses we have on-hand.

_gl

OK, maybe there's another way to trick it.

re. lenses I will confirm when I get my bodies. I have the 14mm and 20mm primes here, and I can tape the contacts to simulate manual lenses. (I also have the Lumix 3D lens which I tape up to shoot raw stills, it's mostly awful for video though).

David_Wilson

Many many thanks... any ideas that occur to you we would be very happy to try out here while you wait for your cameras.

MarcusWolschon

Try the LENS-RELEASE button in sync to sync the cameras. It does a proper lens shutdown and power-on via the SPI interface to the lens.

crunchy

@_gl "A second idea - as the power button is 'soft', something is already running when the camera is off. It may just be an interrupt that wakes the CPU, but it's possible that the master clock is already running from the moment you connect power. If so, then just delivering juice to the bodies, even when switched off, may be syncing them at that point."

This seems possible. In this case, if the power is delivered to both cameras simultaneously, the cameras should remain in synch (for some period, of course) even if they are waken up afterwards by shortly clicking power buttons. David Wilson may check it if his cameras have common external power supply.

Concerning manual and "auto" lenses, the situation is not so clear. Mostly I get synched shots with a pair of GH2s (or GH1s) with Panasonic lenses. However, there are still many shots (usually less than 50%) which are not in synch. I switch on the cameras with predefined time-delay in order to get longer "good-synch" interval. The missynch between the cameras is monitored via video out signal.

David Cole measured the missynch between the GH cameras and reported that he's got bad synched videos even when video out signals from the cameras were perfectly in synch. On the other hand, I've got badly synched videos only when the video out signals were not in synch (during recording). Later on I discovered the reason. David Cole used NTSC video out format and I used PAL video out signal (in camera's menu). When using PAL settings, the video out signal changes from 60i (in steady-mode) to 50i (during recording video). If the cameras are actually not in synch, it will be perfectly seen in missynched video out signals as well. On the other hand, if using NTSC video out signal, nothing changes in video out signal when starting recording (even though the cameras are not in synch anymore).

To make the story shorter: when using two GH2s (GH1s, GH3s?, GH4s?), set video out signal to PAL. If there is no difference between video out signals from the cameras when they start recording, the cameras are actually in synch, period. No matter what kind of lenses you have on the cameras...

On the other hand, if you want repeatable guaranteed synch with GH1, GH2 (probably GH3 and GH4 as well) cameras with any kind of lenses, use NTSC type of camera and 60p mode (e.g. 720p60, 1080p60). If the cameras are in synch after power-on, the recorded video will remain in synch as well. There might be a difference of one or two full frames between the takes, but this is easily solved in post (when comparing sounds from both cameras).

In short, manual lenses are not the limitation.

_gl

Very valuable info @crunchy. I don't actually measure sync on my GH2's. I only recently started designing a circuit for it though (with a nice OLED display and everything), but if the Pockets work out I'll use LANC instead of VSYNC extraction.

(on a tanget: Of course what I'd really like is HDMI decoding (you could also measure sync from that), but it's beyond my skillset. The idea is to build a small board that accepts two HDMI ins from cameras, and does custom image processing to a single HDMI out to show eg. anaglyph previews (and other types) on any cheap HDMI monitor. You could extend that with histograms etc. I can do the image processing, but HDMI signal handling seems very complex, and demands high-speed processors and suitable electronics, I have no experience with either.)

Like you I find that with my Panny primes that sync is hit and miss - but when it works, it's usually perfect. As I suspected the lens calibration, I developed a routine where I pre-focus and expose my shot (I always use manual exposure), then power cycle the bodies first before recording. The idea was that as both lenses are already on the same focus & iris, the calibration time on both should be much closer. It seems to be working for me, in that I get good sync more often (which I determine from the audio waveforms of a click at recording start). I also take more time these days after power-up to let everything settle, so it may be a combination of things, or maybe the lenses really have nothing to do with it at all. I should have done scientific sync test on my GHs, but I just never got around to it as it worked well enough-ish (but I will with my Pockets).

@MarcusWolschon, I didn't realise that m43 lenses use SPI (I've just coded SPI comms on an AVR for something else). That might give us another way in. Has anybody reverse-engineered the protocol?

It would be much easier to hack the lens pins with manual lenses (not sure how that works out with a Speedbooster fitted?).

Can someone try the LENS RELEASE to see if it resets the capture clock on a single body? The best way to test it (and dual-body sync) is with an old CRT monitor (has to be CRT) and Peter Wimmer's free software: http://www.3dtv.at/Knowhow/Synctest_en.aspx , you can just see what's happening to sync in real-time.

_gl

Ah @MarcusWolschon, you've already done a page on m43 pins/protocol! Awesome: http://marcuswolschon.blogspot.de/2013/12/reverse-engineering-micro-four-thirds.html

43 (not m43) protocol: http://preamp.org/revenge/four-thirds-communication-protocol

Various other pins (Nikon/Canon) for comparison: http://www.dvxuser.com/V6/showthread.php?247706-micro-4-3-lens-electronic-pinout-for-the-focus-control/page2

David_Wilson

A quick test with a single BMPCC confirms Marcus's thought that the lens release button resets the camera's clock. At least I can say that the lens release button appears to put the camera though the same protocol as camera start. If I can get a moment this afternoon I will two cameras and the Applied Logic board to see if a common reset with the lens release buttons of two cameras will put them in sync. I have never set up Peter's synctest but hopefully the Applied Logic board's led read-outs will provide some information.

I should point out, however, that the lens reset button understandably only works with an automatic lens.

_gl

If you get the Sync Tester running, there's a few things to try. You only need a single camera for these.

I think this is right (correct me if I'm wrong as I haven't used the sync test method yet): no need to measure anything, you should just be able to see the CRT beam moving gradually up or down as the camera drifts. You want a fast shutter speed (to eliminate motion blur). Try to get the beam moving as little as possible. You might have to try different frame rates to get close to the beam refresh rate? I believe the default is 60Hz, so 30fps might be a good match.

We're looking for anything that resets the capture clock, ie. that places the vertical beam position visually somewhere else suddenly:

power up, observe the beam position, then quickly power down and up again (keeping external power connect all the time). Is the beam still where you expect it?

If yes, power cycling does NOT reset the clock, and connecting juice may sync bodies. You could verify this with two bodies - if you can feed juice to them at exactly the same time, but then power them up at slightly different times - are they actually in sync (beam positions in the same spot on both)?

same for Lens Reset - does it actually reset the capture clock (beam position changes)?
try to find any other setting that resets the clock. For example changing shutter speed, Rec709 <> film, Prores <> RAW, focus punch-in (as these may change the sensor mode) etc.
if you (or anyone) has a LANC controller, try if any of the commands it listens to reset the clock. It only supports a limited cmd set (see my next post later).

If lens release or some lens pin manipulation works, we can probably simulate a lens connection with a little microchip - maybe just signalling the 'lens ready' pin would be enough (or something like it)

_gl

Thinking about it, you can do most of these tests with your Applied Logic readout. Here I would leave one camera connected and running, and try the above on the other cam. If the sync suddenly changes the capture clock is reset.

You can also try manually start-stop recording one body several times. Does it change the sync?

kavadni

This might be of interest to you at some point
http://www.intersil.com/content/intersil/en/products/audiovideo/video-ics/video-sync-separators/EL4511.html

_gl

Thanks @kavadni. From skimming the datasheet, it seems much more capable than the standard composite sync separators, but it still only deals with analogue signals, correct? The Pocket doesn't have analogue, so the only option would be HDMI.

If going that route I would want to do full image processing on two HDMI inputs to give live anaglyph and other stereo previews on a single HDMI monitor (and you could also display the sync directly on it). But I don't know how to do the HDMI interfacing ...

_gl

Here are the supported LANC commands, courtesy of http://forum.blackmagicdesign.com/viewtopic.php?f=18&t=14201 :

LANC(0x18, 0x33); // REC

LANC(0x28, 0x43); / /AUTO FOCUS

LANC(0x28, 0xAF); // AUTO IRIS

LANC(0x28, 0x45); // MANUAL FOCUS FAR

LANC(0x28, 0x47); // MANUAL FOCUS NEAR

LANC(0x28, 0x53); // IRIS+

LANC(0x28, 0x55); // IRIS-

If BMD could add ISO, shutter speed and a few other key params, we could remote control and match both bodies perfectly. The same controller can measure sync from LANC, and could simulate a lens connection if necessary.

Theoretically it's probably even be possible for BMD to send a capture sync signal via LANC to the other body.

_gl

Another question, the Pocket has timelapse functionality right? Could we trigger this simultaneously on both bodies? How is it activated?

I included microcontroller-based timelapse functionality on the custom controller for my GH2 rig: http://www.dvxuser.com/V6/showthread.php?221553-GH1-Stereoscopic-Rig-Collaboration&p=2359206&viewfull=1#post2359206 (that was the old hyper rig, it's all now on a much shorter bar from DealExtreme).

But that was triggering the stills shutter, so the trigger came from an external clock (my controller) and wouldn't drift between two bodies. Would this even work on the Pocket without massively drifting?

kavadni

@_gl ... actually that was not the link I intended ... check this one
http://www.analog.com/static/imported-files/data_sheets/ADV7842.pdf

While I cannot find the forum that explained it, this device
http://www.lumagen.com/testindex.php?module=hdmiext_details
apparently this particular device (probably most repeaters) has a suitable IC which has the sync pulse available ... Thus ... 2 extenders provide the HDMI passthrough ... and then you feed the pulses into a comparator of some description ... Arduino/AVR or whatever

I would want to do full image processing on two HDMI inputs to give live anaglyph and other stereo previews on a single HDMI monitor (and you could also display the sync directly on it).

With my GH2s, I used two by HDMI-SDI converters and a Matrox MC-100

_gl

With my GH2s, I used two by HDMI-SDI converters and a Matrox MC-100

Cool. So the MC100 can do it, handy. Shame it's only SDI (I'd prefer to avoid the converters, more to carry around and power). I mainly shoot outdoors, carrying my rig myself, often all day, so small, light & simple is ideal (that's why the Pocket appeals). It's also not cheap, ~£350 UKP / 500 USD. How does it work in practice?

... actually that was not the link I intended ... check this one

Cool. I found some Analog Devices HDMI chips like that before - but it seems pretty complex. AFAIKT this one has two inputs, but they're used for switching between rather than processing simultaneously? It also seems to require additional RAM modules, you probably have worry about HDCP (even though there should be none from the cameras, but you probably still have complications from it at the other end) etc.

Another option though is buying two HDMI->analog converters, and then digitizing & processing that. Some people have already done that for DIY Ambilight TV illumination (where you place some LEDs behind the TV enclosure, and sample the video colours at the image edges, then spread thm via the LEDs to your wall). With component (rather than composite) you could probably get HD?

I would prefer a small HDMI-based box that could do it all ideally. If anybody's interested in this, we could start another thread?

kavadni

@_gl

How does it work in practice?

It works well, the actual path was (GH2 -> BMD HDMI-SDI x 2) -> Matrox MC100
-> FramePacked SDI into BMD Mini Recorder into MacBook via thunderbolt. (I then use an application called 3D Stereo Cat)
and
-> FramePacked HDMI into an LG Stereo Monitor and view on set with Glasses

It works absolutely great ... Everything is perfect with 2 x EX3 (Genlockable).

Personally I have given up on measuring sync and offsetting startup times. Results can be achieved, but on set not at a level of reliability that I feel I can charge for a service.
My rig is a (robert rig) and I can fit two by ex3 on it. I am waiting for an affordable genlock camera, then I will proceed with 3D.

_gl

@kavadni cool, I've read up on the MC100 a bit more and see that it does a number of other useful jobs too. For me (non-paid art work), it's overkill as I don't need anything SDI related (yet, maybe by the next upgrade a couple of years down the line :)) and wish there was just a simple HDMI box that could do anaglyph at least.

re. sync I hear you, you cannot have reliability issues on shoots with deadlines. Otherwise with patience the GH2's with simul-power have worked out great for me. I have lost a few shots and some aren't perfect, but with sync monitoring (which I didn't have) that wouldn't have happened. I don't know how close you can get with simul-power for high action shots, as I don't do that. For most other shots, you can get genuinely excellent results.

_gl

Pocket teardown - very easy disassembly:

(why did he leave the lens on?? : )

A hardware sync hack would be great. Anyone want to look for sync pins or clock hacking?

Also you could remove the black plastic of the grip to flatten it. That way you could overlap ('stack') two bodies with far less depth separation, to get the lenses closer together ...

_gl

I did some BMPCC 3D Rig measurements for various layouts in Photoshop (attached). It also shows all the ports & buttons.

With flattened grips, you could stack the bodies closer vertically and get the 70mm interaxial (plus any connectors). Although it may be fine as-is as the grips are already fairly shallow, at least with a reasonable min. distance from the closest subject. You wouldn't see half the screen of one cam, but you can still get to the controls, and you might use external monitoring anyway

_gl

(from the video) the main CPU is a Spartan-6 FPGA: http://www.xilinx.com/products/silicon-devices/fpga/spartan-6/ .

There seem to be a number of unconnected/unsoldered points on the front board, by the grip. Debug points? Or maybe interesting signal taps?

I dont know much about clocks/crystals. Anybody spot one? How about the two silver things right above the FPGA?

_gl

You could also probably slave the buttons from one camera to the other. Then all settings would by synced.

Howdy, Stranger!

Categories

Tags in Topic

Top Posters