fulldome.org

immersive projection environments

Stereoscopic domemaster images

Hi everybody,

I'm posting here some tests renderings using a domemaster lens shader for mental ray that allows for 3D steroscopic images.

It's still a work in progress, and I'd like to make this an open effort, posting all the info, research, user interface, and the source code (later on).

A quick intro about myself. I've been working on computer graphics for too long now, and while I used to be a developer (15 years ago), I'm now mainly doing 3D content. I started evaluating the possibility of steroscopic domemaster images after I was asked by Terry Galloway at the Chabot Space and Science Center in CA to do some research.

I started with Daniel F. Ott Angular fisheye shader, and expanded (a lot) from there.
Daniel's shader can be found here, and it is also included in the DLL I will post soon.

This shader starts from the camera position, and creates two virual cameras (left and right, renderable one at a time) that rotate, constantly looking at the specific point of the dome matching the current rendered pixel.
The shader supports horizontal and vertical domes, and any degree of tilt in-between.

Here are two samples of Center, Left, and Right images, in vertical and horizontal mode.

It's not easy to see the differences here, but look below at the test stereo pairs.

The Center image would match Daniel's shader, but it has a different orientation, as I use some tricks to rotate the coordinate system 90 degrees. The current orientation matches the camera viewport, so it's extremely intuitive to setup the camera.

Of course, withouth some restrictions, this system will creates distortion points and areas where the 3D effect is wrong, reversed, or misaligned. The shader allows the use of maps to control the cameras separation (reduce or eliminate 3D effect), head rotation (force some areas to be looked with the head looking straight), and head tilt.

These are sample maps that I'm using for testing.

Using the Turn and Separation maps above, you can see what happens when rendering a simple grid (Red=Right, Green=Left).

The image above is something I would consider for horizontal or slightly tilted domes, where above/behind the viewer head the 3D effect is eliminated to allow the top of the dome to be looked at without turning the head, and the back with fully turned head.

But maps can be used creatively to control any area of the dome. Here, the Turn map above is used in combination with a simple gradient as a Separation map to have 3D only in the front part of the dome.

I think the math is almost there. The User Interface might need some adjustments, and maybe some automatic correction of distorted areas can be used instead of the maps, but the major issue now is to find a way to create proper maps.

Unfortunatley, I don't have any dome at home to try it, so I rely on simulated previews using a 3D concoction in 3ds Max, but that won't let me verify if the head tilt and rotation is set correctly. I think those values can be only found by experimentation on real domes.

These are some samples from the horizontal and verticals samples above. Relax (don't cross) your eyes to see the 3D effect. It helps is you are a bit shortsighted and take you glasses off :)

Horizontal:

Vertical:

I will post more tomorrow. If you have any comments, please post.

Rob

▶ Reply to This

Replies to This Discussion

Permalink Reply by Roberto Ziche on September 16, 2011 at 12:30am

Hi Uwe,

I'm posting the simulator and the concoction I used to preview the effect of the 3d settings.

The simulator is missing the sample dome maps. Just use any domemaster L/R image you create.
One is for single images test, the other for animated ones.

All the paths used will need to be adjusted. Won't work until you do that.
Check the Asset Tracking dialog to see the paths I'm using.

Keep in mind these are very rough tools. Writing a full set of instructions woudl drive me nuts :)

There is a two step process to use them.

Once you have oriented/animated the cameras, tilted the domes properly (one cam/dome only, the other will match the first), run Batch rendering to create the stereo pair, then Video post to composite the relaxed-eye image in the frame buffer.

---

The other scene is the concoction I use to see the effect of the 3D settings parameters. The only objects you can select and move are the "parameters". The blue teapot is your object, the gray one is the apparent postion of the object.

You can change the eye separation (one eye, the other moves automatically), the screen distance, the convergence point (which in my tool is 1/2 the dome diameter setting), and the teapot distance.

Keep an eye on is the apparent infinite point. It can be used to your advantage :)

---

About your idea of using center and one of the L/R cameras. I think that might work, but there is a shift of the center that might cause some distortion. Maybe nothing that the viewer would notice.

I don't see any other side effect, as all the parameters would act on one camera only, but still maintain their exact functionality.

I guess the only way to verify is to try and compare with a L/R normal pair on a real dome.

Rob

Attachments:

▶ Reply

Permalink Reply by Roberto Ziche on October 6, 2011 at 1:53pm

Hi Chris,

Not for Mac. I don't have a Mac compiler. One of these days I will post the source code. Then everyone can help compiling different versions.

I have to get a clear Ok from my employer first.

Rob

▶ Reply

Permalink Reply by Andrew Hazelden on October 24, 2011 at 1:27pm

Hi Robert.

I would like to compile a Maya on Mac x64 build of your stereoscopic domemaster lens shader.

Have you considered adding support for the mental ray 3.9 stereoscopic rendering feature to your domemaster mental ray shader? This feature could speed up stereoscopic rendering by about 11% on most scenes.

I recently did a test render using a Maya on Mac version of the regular domeAFL lens shader and used Haggi's (stereoscopic) settings shader to create my first stereoscopic fisheye rendering. (This approach lacks features like turn and separation compensation.) Attached is the result.

Regards,

Andrew Hazelden

▶ Reply

Permalink Reply by Roberto Ziche on October 27, 2011 at 1:56am

Hi Andrew,

I'll be posting the source code soon... or as soon as work give me a break :)

I believe you can use the 64-bit version I posted on maya too. I never tried but it's a simple mental ray plugin. Should work.

I too want to see what the mental ray 3.9 stereo support does. No time for that either :( But will it work with the moving camera of the shader?

Rob

▶ Reply

Permalink Reply by Andrew Hazelden on October 31, 2011 at 3:27pm

Hi Robert.

I created the supporting files for your Stereoscopic Domemaster lens shader to work with Maya on Windows. I edited the MI include file to add Maya specific options and created a Maya MEL attribute editor template and custom Hypershade node icons. I also put together a readme.txt file with installation instructions and the basic information on how to use the shader.

Download the Maya version of the domeAFL FOV Stereo lens shader.

Note:

This was quite a learning experience for me and I welcome any feedback or suggestions.

I don't have access to a real Dome to do extensive testing so all I could do for quality assurance testing was view the rendered fisheye images in a new Maya scene with simulated dome geometry and anaglyph 3D glasses.

Maya Installation Instructions
1. Unzip the domemaster3D.zip archive.

2. Copy the appropriate "domeAFL_FOV_Stereo.dll" file from either the "Windows 32-bit LIB" or "Windows 64-bit LIB" folder or to your mental ray LIB folder:

C:\Program Files\Autodesk\Maya2012\mentalray\lib\

If you are running a 32-bit version of Maya install the 32-bit DLL. If you are running a 64-bit version of Maya install the 64-bit DLL.

3. Copy the "domeAFL_FOV_Stereo.mi" mental ray include file to:

C:\Program Files\Autodesk\Maya2012\mentalray\include\

4. Copy the Maya AE Template file "AEdomeAFL_FOV_StereoTemplate.mel" to either the Maya AETemplates folder or to your user account's Maya script folder:

C:\Program Files\Autodesk\Maya2012\scripts\AETemplates\

My Documents\maya\2012\prefs\scripts

5. Copy the Hypershade icons from the "Icons" folder to your Maya icons directory or to your user account's Maya icons directory:

C:\Program Files\Autodesk\Maya2012\icons\

My Documents\maya\2012\prefs\icons\

6. The next time you start Maya you will find the "domeAFL_FOV_Stereo", "domeAFL_FOV", and "domeAFL_WxH" lens shaders in the Hypershade. Look in the create bar under the mental ray > lenses section.

Using the Domemaster Stereo Lens Shader in Maya

Connect the "domeAFL_FOV_Stereo" lens shader to a camera's .miLensShader input.

A quick way to make the connection in the Hypershade is to hold down the Control key on your keyboard and, using the Middle-Mouse button, drag the domeAFL_FOV_Stereo lens shader onto the camera's node icon. This will automatically make the node connections.

In the Hypershade work area, if your domeAFL_FOV_Stereo node has a solid red color background that means you need to change your renderer to Mental Ray. You can change mental ray to be your default renderer by going to the Window > Settings / Preferences > Preferences menu. In the Preferences window click on the rendering section. Select "mental ray" from the the preferred renderer pop-up menu and click "Save".

You can connect your own custom texture maps to the Separation Multiplier Map, Turn Multiplier Map, and Head Tilt Map attributes.

------------------------------------------------------------------------------------------------------------------

I'm looking forward to compiling a Mac version of this shader because I had to do the mental ray Maya shader conversions using the Parallels Desktop emulation software.

P.S.
I'm not sure how mental ray 3.9's native stereoscopic feature would handle the moving camera feature used in this shader. Also I don't know how you would pass your custom view settings for the left and right camera to mental ray because from what I read in the docs it appears that shaders are not able to find out which view is being rendered.

Roberto Ziche said:

Hi Andrew,

I'll be posting the source code soon... or as soon as work give me a break :)

I believe you can use the 64-bit version I posted on maya too. I never tried but it's a simple mental ray plugin. Should work.

I too want to see what the mental ray 3.9 stereo support does. No time for that either :( But will it work with the moving camera of the shader?

Rob

▶ Reply

Permalink Reply by Roberto Ziche on November 4, 2011 at 2:30am

Thanks Andrew. This is great for all the Maya users :)

Rob

▶ Reply

Permalink Reply by Meinolf Mueller on December 7, 2011 at 7:30am

Hello Rob,

your tool is very helpful, thanks!

I tested the shader regarding to the math output of your simulator. But it doesn't fit exactely. When I create an object wich is 3 meters far from the camera and I want it to be on screen convergence with no parallax I should type in the DomeRadius field also 3 meters (if the unit system is in meters), right? The camera and eyeseparation is exactely the same amount.

Could you provide the math behind your simulator?

This simulator opened my eyes on two things that can actually be used to our advantage to create manageable finite size worlds. First, the non-linear movement of the apparent object. And this can be used to establish areas closer to the camera that have enhanced depth effect. Second, the presence of a point where objects will appear at infinite. Knowing that point, we can have objects set at reasonable distances from the scene center for our background.

A snapshot of the simulator...

I will post more on this tool once I verify that my math and my predictions are actually working.

Rob

▶ Reply

Permalink Reply by Timo Schumacher on December 7, 2011 at 9:08am

Hi Rob,

I am currently working on a vray dom camera plugin that should do more or less the same thing you do with mental ray. I was wondering if you can tell me what the math is behind the creation of the stereo effect. Awesome would be the sourcecode... any help would be great!

Thanks in advance

Timo

▶ Reply

Permalink Reply by Roberto Ziche on December 7, 2011 at 11:28pm

Meinolf,

I tried the shader and the simulator. The shader works correctly. I placed a camera in an arbitrary position in space, and an object at another arbitrary position, and found the distance between the two (I was testing the most generic case).

I then rendered Left and Right images with the dome radius set to that distance, and the object rendered in the same position.

The simulator has a totally different math because it's purpose is different. But if you set the object called Convergence over the bluish teapot object, the apparent object position (gray teapot) will appear on the Screen object, no matter where it's positioned on the axis. The eye distance won't make a difference.

Let me know if I misunderstood your question.

Rob

▶ Reply

Permalink Reply by Roberto Ziche on December 7, 2011 at 11:29pm

Timo,

I'm planning to finally post the source code during the holidays break. You are welcome to make the Vray version. I will try and use a site where everyone can contribute and post different version for different renderers. (Google Code?)

Rob

▶ Reply

Permalink Reply by Timo Schumacher on December 8, 2011 at 3:29am

Hi Rob,

sounds cool! Well Google code would be a good place for that and I would like a mercurial hg repository. Since branches creation and working side by side is much easier then only svn.

I am already very far with my vray plugin... I am rendering fisheye projection and I already have the standard stereo projection stuff working so far. And I am working on the math to create rotated cameras right now. The company I am writing it for just got there own miniDome ;) and I can't wait to test it in there. Do you have a paper which your stuff is based on which I could look at... I think when I got the matrix and vector stuff done I am almost there.

Timo

▶ Reply

Permalink Reply by Roberto Ziche on December 11, 2011 at 11:29pm

Timo,

I'm posting a scene and a script you can use to see the math behind the lens shader. The formulas used here are the exact same used in the C code.

The scene setup is done to simulate the actual mental ray camera space.

Load the scene, run the script. It will reset the scene, and then you can adjust all the params from the dialog that pops up. The camera shows you the mental ray camera orientation, and the Red and Lime green represent the Right and Left virtual cameras created by the lens shader.

With this you can see the effect of the settings on the virtual cameras. Just use Phi and Theta to move the target point on the dome surface.