Layers Project Notes

Author

flip phillips (fxpppr)

Modified

September 22, 2026

Coordinate Systems & Spatial Conventions

This document serves as the canonical reference for all spatial coordinate frames, transformation matrices, vector conventions, and camera models used across the layers project—spanning RenderMan scene generation, OpenEXR metadata, Wolfram Language paclets (EXRIO, EXRCamera), and the neural field engine (nerf/).

1. Quick Reference Matrix

Coordinate System Handedness Up Axis Forward Axis Right Axis Units / Bounds Primary Use
World Space Right-handed \(+Y\) \(-Z\) (default view) \(+X\) Metric/Scene units (\(Y=0\) is floor) Geometry placement, lights, NeRF training points
RenderMan Camera Space Left-handed \(+Y\) \(+Z\) (into scene) \(+X\) Metric (\(Z > 0\) is in front of camera) RenderMan internal rasterization, worldToCamera
OpenEXR Camera Frame Left-handed \(+Y\) \(+Z\) (along view) \(+X\) Camera space worldToCamera header attribute
Screen NDC Space Left-handed \(+Y\) (up) \(+Z\) (depth) \(+X\) (right) \([-1, 1]^2\) normalized device coords Ray generation (ndcX, ndcY), pixel projection
Pixel Raster Space 2D Grid Down (\(+v\)) or Up N/A \(+u\) (right) \([0, W-1] \times [0, H-1]\) Image arrays, AOVs, Cryptomatte masks
Wolfram Image3D Space Grid Tensor Dim 1: \(Z\), Dim 2: \(Y\), Dim 3: \(X\) N/A \(+X\) \(\{Z, Y, X\}\) voxel array Volumetric continuous field sampling
Object Space Local Mesh local \(+Y\) Mesh local \(+Z\) Mesh local \(+X\) Centered around origin \((0, 0, 0)\) geometry.Po, OBJ raw vertices

2. World Space (geometry.P, Scene Layout)

The world space coordinate frame is established in renderman/layers.py.

          +Y (Vertical Up)
           │
           │
           │
           └───────► +X (Right)
          /
         /
        ▼
      +Z (Towards Viewer)
  • Vertical Axis (\(+Y\)):
    • The ground plane (cyclorama floor) lies horizontally on the \(XZ\) plane at \(Y = 0\).
    • Elevation and object height increase in the positive \(+Y\) direction.
    • The crazy-sphere and scatter objects rest on the floor, with centers typically located at \(Y \approx 0.52\)–\(0.60\) and bounding extents \(Y \in [0.0, 1.25]\) (up to \(Y \approx 2.2\) with large displacement noise).
  • Horizontal Ground Plane (\(XZ\)):
    • Ground center is at \((X=0, Z=0)\).
    • \(X\) spans left (\(-X\)) to right (\(+X\)).
    • \(Z\) spans back (\(-Z\)) to front (\(+Z\)).
  • Camera Hemispherical Orbit:
    • RenderMan cameras are placed on a geodesic hemisphere strictly in the upper hemisphere: \[\mathbf{p}_{\text{cam}} = (x, y, z) \quad \text{such that} \quad y \ge 0\]
    • Camera up vector is almost universally: \[\mathbf{Up}_{\text{world}} = (0, 1, 0)\] (except for the singular top-down pole view \(|y| > 0.999\), where \(\mathbf{Up} = (0, 0, -1)\)).

3. RenderMan Camera Space & UVN Basis

RenderMan defines camera space with the eye at the origin looking along \(+Z\).

3.1 Construction of the Camera Frame (from NCCARenderman/common/Camera.py)

Given camera position \(\mathbf{E}\) (eye), look-at target \(\mathbf{A}\) (at), and world up vector \(\mathbf{Up}\) (up = {0, 1, 0}):

  1. Viewing Axis (\(N\)): Points from the target back to the eye: \[N = \frac{\mathbf{E} - \mathbf{A}}{\|\mathbf{E} - \mathbf{A}\|}\]
  2. Camera Right Vector (\(U\)): Perpendicular to both world up and the view line: \[U = \frac{\mathbf{Up} \times N}{\|\mathbf{Up} \times N\|}\]
  3. True Camera Up Vector (\(V\)): Orthogonalized up vector: \[V = \frac{N \times U}{\|N \times U\|}\]
  4. Forward Viewing Vector: The camera looks along: \[\mathbf{F} = -N = \frac{\mathbf{A} - \mathbf{E}}{\|\mathbf{A} - \mathbf{E}\|}\]

3.2 Handedness

In Camera.place(ri):

ri.Scale(1, 1, -1)
ri.ConcatTransform(tx)

RenderMan scales \(Z\) by \(-1\), turning camera space into a left-handed system: - \(+X\) is Right - \(+Y\) is Up - \(+Z\) is Forward (into the scene)

4. 4x4 Transformation Matrices

4.1 Row-Vector Multiplicative Convention

RenderMan and OpenEXR follow the row-vector convention (direct vector-matrix multiplication on the right):

\[\mathbf{p}_{\text{cam}} = \mathbf{p}_{\text{world}} \cdot M_{\text{w2c}}\]

where \(\mathbf{p} = [x, y, z, 1]\).

4.2 The worldToCamera Matrix

The \(4 \times 4\) transformation matrix stored in the EXR header (worldToCamera) is structured as:

\[M_{\text{w2c}} = \begin{pmatrix} U_x & V_x & -N_x & 0 \\ U_y & V_y & -N_y & 0 \\ U_z & V_z & -N_z & 0 \\ -\mathbf{E}\cdot U & -\mathbf{E}\cdot V & \mathbf{E}\cdot N & 1 \end{pmatrix}\]

Notice: - Column 1 is the Right vector \(U\). - Column 2 is the Up vector \(V\). - Column 3 is the Forward view vector \(-N\). - Row 4 contains the negative dot products with the eye position \(-\mathbf{E}\).

4.3 The Camera-to-World Matrix (\(M_{\text{c2w}} = M_{\text{w2c}}^{-1}\))

Because the \(3 \times 3\) upper-left submatrix is orthonormal, inverting \(M_{\text{w2c}}\) yields:

\[M_{\text{c2w}} = \text{Inverse}[M_{\text{w2c}}] = \begin{pmatrix} U_x & U_y & U_z & 0 \\ V_x & V_y & V_z & 0 \\ -N_x & -N_y & -N_z & 0 \\ E_x & E_y & E_z & 1 \end{pmatrix}\]

In Wolfram Language (EXRCamera.wl and nerf/NeRFRayGen.wl):

inv = Inverse[matrix];
pos = inv[[4, 1 ;; 3]]; (* Camera World Position E *)
rx  = inv[[1, 1 ;; 3]]; (* Camera Right Vector  U *)
ry  = inv[[2, 1 ;; 3]]; (* Camera Up Vector     V *)
rz  = inv[[3, 1 ;; 3]]; (* Camera Forward (-N)    *)

5. Camera Ray Generation (NeRF Raymarching)

In nerf/NeRFRayGen.wl, primary camera rays are generated from cameraMatrix (\(M_{\text{w2c}}\)):

  1. Field of View & Half-Frustum: \[h_w = \tan\left(\frac{\text{FOV}}{2}\right), \quad h_h = \frac{h_w}{\text{AspectRatio}}\]
  2. Normalized Device Coordinates (NDC) for pixel \((u, v)\) with \(0 \le u < W, \; 0 \le v < H\): \[\text{ndc}_x = \left(\frac{u + 0.5}{W} \cdot 2 - 1\right) \cdot h_w\] \[\text{ndc}_y = \left(1 - \frac{v + 0.5}{H} \cdot 2\right) \cdot h_h\]
  3. World Ray Direction: \[\mathbf{d} = \text{Normalize}\left(\text{ndc}_x \cdot \mathbf{r}_x + \text{ndc}_y \cdot \mathbf{r}_y + \mathbf{r}_z\right)\] \[\mathbf{o} = \mathbf{pos}\] where \(\mathbf{r}_x = U\), \(\mathbf{r}_y = V\), \(\mathbf{r}_z = -N\).
  4. Stratified Samples: Sample positions along the ray are: \[\mathbf{x}_i = \mathbf{o} + t_i \mathbf{d}, \quad t_i \in [t_{\text{near}}, t_{\text{far}}]\]

6. Turntable & Orbit Visualization Space

When rendering turntable orbit sequences (RenderNeRFTurntable) or using the interactive viewer (InteractiveNeRFViewer), the camera orbits in RenderMan’s \(Y\)-up space:

6.1 Spherical Orbit Equation

Given target center \(\mathbf{T} = (T_x, T_y, T_z)\) (default {0.0, 0.6, 0.0}), radius \(R\), elevation angle \(\theta\), and azimuth \(\phi \in [0, 2\pi]\):

\[\begin{aligned} \text{eye}_x &= T_x + R \cos(\theta) \sin(\phi) \\ \text{eye}_y &= T_y + R \sin(\theta) \\ \text{eye}_z &= T_z + R \cos(\theta) \cos(\phi) \end{aligned}\]

With: \[\mathbf{Up} = (0, 1, 0)\]

6.2 Why \(Z\)-Up Failed (Common Pitfall)

If \(\mathbf{Up}\) is incorrectly set to \((0, 0, 1)\) or elevation is placed in \(Z\): 1. The camera rolls 90° on its side (horizon becomes vertical). 2. The azimuth revolves in the vertical \(XY\) plane (tumbling end-over-end through the ceiling and floor) rather than orbiting around the upright object.

7. Volumetric Tensor Space (Image3D, RenderNeRFVolume)

When sampling the continuous neural field into a 3D grid with RenderNeRFVolume:

7.1 Coordinate Lattice

The bounding box \([X_{\min}, X_{\max}] \times [Y_{\min}, Y_{\max}] \times [Z_{\min}, Z_{\max}]\) is subdivided into a regular \(R \times R \times R\) grid: - Default bounds: \[X \in [-1.5, 1.5], \quad Y \in [-0.2, 2.2], \quad Z \in [-1.5, 1.5]\] - Note that \(Y\) is centered at \(Y = 1.0\) to capture the full crazy-sphere resting on the ground \(Y = 0\).

7.2 Tensor Indexing vs World Coordinates

In Wolfram Language:

gridPts = Outer[List, xRange, yRange, zRange];
  • Image3D tensor dimensions are \(\{D_z, D_y, D_x, C\}\).
  • Slice 1 corresponds to \(Z_{\min}\), row 1 corresponds to \(Y_{\min}\), column 1 corresponds to \(X_{\min}\).
  • The vertical axis of the volume aligns directly with the \(+Y\) world coordinate.

8. Tangent Space & Differential Geometry AOVs

RenderMan writes surface differential properties to specific EXR channels:

  • geometry.P: World-space position of the surface point \((P_x, P_y, P_z)\).
  • geometry.Po: Object-space position \((P_{ox}, P_{oy}, P_{oz})\) prior to scene transform.
  • geometry.Nn: Normalized shading normal vector \((N_x, N_y, N_z) \in [-1, 1]^3\) in world space.
  • geometry.dPdu / geometry.dPdv: World-space partial derivatives of position with respect to surface parametric coordinates \((u, v)\).
  • geometry.k1 / geometry.k2: Principal curvatures \(\kappa_1, \kappa_2\) (computed post-render in EXRComputed using screen-space Weingarten equations).

Volume Representation

Volume Representation