--- title: "Getting Started with scimesh" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Getting Started with scimesh} %\VignetteEngine{knitr::rmarkdown} %\usepackage[utf8]{inputenc} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ## What is scimesh? **scimesh** is a fast, headless, GPU-free software renderer for 3D triangle meshes that produces **publication-quality images** for papers, slides, and presentations. It works anywhere R works — no X11, no OpenGL, no GPU required. Applications span any field that works with 3D surfaces: - **Neuroimaging**: cortical surface visualisation (FreeSurfer data) - **Structural biology**: molecule surfaces from PDB files - **Computer graphics**: Stanford models, procedural geometry, textured meshes - **Engineering and simulation**: mesh-based scientific visualisation of any kind All rendering is done in modern C++17 and returns in-memory RGBA images that can be saved to PNG or composed into multi-panel figures. While scimesh can serve as a drop-in renderer backend when rgl/OpenGL is unavailable (e.g., on macOS without XQuartz, on HPC clusters, in CI containers), it is a **general-purpose visualisation tool** — not tied to any specific domain or package. ## Installation ```r # Install from GitHub remotes::install_github("dfsp-spirit/scimesh") ``` For the full viridis colormap family (magma, inferno, cividis, etc.), you may optionally install `viridisLite`: ```r install.packages("viridisLite") ``` But this is not required — scimesh ships built-in `viridis_colormap()` and `diverging_colormap()` functions that use only base R. ## Quick Start ```{r, eval=FALSE} library(scimesh) sphere <- generate_sphere(c(0, 0, 0), radius = 1.2, segments = 32, color = c(0.9, 0.3, 0.2, 1.0)) cam <- camera_auto(sphere, direction = c(1.2, 0.8, 1)) # Flat-shaded sphere img <- render_mesh(sphere$vertices, sphere$triangles, colors = sphere$colors, camera = cam, options = render_options( lights = list( list(position = c(0.5, 1.0, 0.8), intensity = 1.5), list(position = c(-0.5, 0.2, 0.6), intensity = 0.5)))) tmp_file <- tempfile(fileext = ".png") write_png(img, tmp_file) printf("Rendered sphere written to: %s\n", tmp_file) ``` ## Rendering Features ### Render Options Reference All rendering is controlled via `render_options()`. The following parameters are available: | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `width`, `height` | integer | 800, 600 | Output image dimensions in pixels | | `shading` | `"smooth"` / `"flat"` | `"smooth"` | Per-vertex (smooth) or per-face (flat) shading | | `backface_culling` | logical | `TRUE` | Skip triangles facing away from camera | | `background_color` | RGBA vector | `c(0,0,0,0)` | Transparent black by default | | `default_color` | RGBA vector | `c(0.7,0.7,0.7,1)` | Fallback when no per-vertex colors | | `invert_normals` | logical | `FALSE` | Flip face orientation | | `wireframe` | logical | `FALSE` | Render edges only | | `wireframe_color` | RGBA vector | `c(0,0,0,1)` | Edge color in wireframe mode | | `projection` | `"perspective"` / `"orthographic"` | `"perspective"` | Camera projection type | | `specular_color` | RGBA vector | `c(0,0,0,0)` | Specular highlight color (off by default) | | `shininess` | numeric | 0 | Gloss tightness (8–128) | | `ambient` | numeric | 0.3 | Ambient light level (0–1) | | `contrast` | numeric | 1.0 | S-curve contrast multiplier | | `lights` | list or `NULL` | `NULL` | Custom light list (auto-default if `NULL`) | | `fog_enabled` | logical | `FALSE` | Enable depth fog | | `fog_start` | numeric | 0 | Distance where the fog fade begins | | `fog_end` | numeric | 1 | Distance where fog is fully opaque | | `fog_space` | `"world"` / `"ndc"` | `"world"` | Unit of `fog_start`/`fog_end` | | `fog_color` | RGBA vector | `c(0,0,0,0)` | Fog color | | `aa_samples` | integer (1, 2, or 4) | 1 | Ordered-grid supersampling factor | | `ssao_enabled` | logical | `FALSE` | Enable screen-space ambient occlusion | | `ssao_radius` | numeric | 16 | SSAO sample radius in pixels | | `ssao_intensity` | numeric | 0.8 | SSAO occlusion strength (0–1) | | `threads` | integer | 0 | Number of CPU threads (0 = auto) | | `clip_planes` | list or `NULL` | `NULL` | Clip planes (see [Clip Planes](#clip-planes)) | | `near_plane`, `far_plane` | numeric | 0.1, 10000 | Depth range of the camera | The following sections cover the most important options in detail. ### Lighting and Shading scimesh uses a Blinn-Phong shading model with support for multiple light sources, specular highlights, ambient control, and contrast adjustments. #### Ambient Lighting The `ambient` parameter (default 0.3) controls how much light reaches surfaces that face away from the light source. Lower values produce deeper shadows and more contrast: ```r render_options(ambient = 0.15) # deeper shadows ``` #### Multi-Light Setups Explicit lights give you full control over direction, colour, and intensity: ```r render_options( ambient = 0.2, contrast = 1.1, lights = list( list(position = c(0.5, 1.0, 0.8), color = c(1, 0.97, 0.9, 1), intensity = 1.5), list(position = c(-0.5, 0.2, 0.6), color = c(0.4, 0.5, 0.8, 1), intensity = 0.5))) ``` #### Specular Highlights Add a glossy sheen to surfaces: ```r render_options( specular_color = c(0.4, 0.4, 0.4, 1), # white highlight shininess = 64) # tight spot ``` | `shininess` value | Look | |-------------------|------| | 8–16 | Soft plastic | | 32–64 | Shiny surface | | 128 | Glass-like tight spot | #### Contrast Adjustment Pass `contrast` to `render_options()` to apply an S-curve contrast stretch after shading. Values > 1.0 push darks toward black and lights toward white, increasing perceived contrast. The default 1.0 means no change: ```r render_options(contrast = 1.1) # subtle S-curve ``` The formula applied is `(value - 0.5) * contrast + 0.5`, clamped to `[0, 1]`. Typical values are 1.1--1.2 for a gentle boost, or up to 1.5 for a dramatic look. You can also apply contrast as post-processing to an existing image: ```r img <- render_mesh(mesh$vertices, mesh$triangles) img <- image_apply_contrast(img, contrast = 1.1) tmp_file <- tempfile(fileext = ".png") write_png(img, tmp_file) printf("Contrast-adjusted image written to: %s\n", tmp_file) ``` ### Camera #### Auto-Framing `camera_auto()` computes a camera that fits any mesh or vertex set: ```r verts <- matrix(rnorm(900), ncol = 3) cam <- camera_auto(verts, direction = c(1, 1, 1), fov = 45) ``` It accepts either an Nx3 matrix or a mesh descriptor list. #### Manual Camera For full control, use `camera()`: ```r cam <- camera(eye = c(0, 0, 10), center = c(0, 0, 0), up = c(0, 1, 0), fov = 30) ``` #### Orbiting the Camera `camera_orbit()` rotates a camera's eye and up vector around its center by a given angle about an axis. This is useful for generating turntable-style frame sequences: ```r cam <- camera_auto(mesh, direction = c(1, 1, 1)) for (i in seq_len(8)) { cam_i <- camera_orbit(cam, angle_degrees = 360 / 8 * (i - 1)) img <- render_mesh(mesh$vertices, mesh$triangles, camera = cam_i, options = render_options(width = 600, height = 400)) # Construct the file path inside tempdir() file_name <- sprintf("frame_%04d.png", i - 1) file_path <- file.path(tempdir(), file_name) # Write the image to the temporary directory write_png(img, file_path) } message("Frames saved in: ", tempdir()) ``` For more complex trajectories, replace `camera_orbit()` with your own function — it only needs to set `camera$eye` and `camera$up`. The resulting PNG frames can be assembled into a video with ffmpeg or into an animated GIF with your tool of choice. See `examples/R/video_frames_orbit/` for a runnable R example, and `examples/cpp/brain_video/` for a C++ version that renders 48 full-brain frames. #### rgl-Compatible Default View `camera_auto()` accepts an `rgl_compat` parameter. When `TRUE`, it mimics rgl's default view parameters: | Parameter | scimesh default | `rgl_compat = TRUE` | |-----------------|-----------------|---------------------| | FOV | 45° | 30° | | Elevation | 0° (front-on) | 15° above horizon | | Distance basis | bounding box | bounding sphere | The distance formula follows rgl's implementation exactly: `distance = sphere_radius / sin(FOV / 2)`, where `sphere_radius` is half the length of the axis-aligned bounding box diagonal. ```r mesh <- generate_cuboid(c(0, 0, 0), c(1, 1, 1)) # scimesh default: straight front view img_default <- render_mesh(mesh) # rgl-compatible view: elevated, matching rgl's default look cam <- camera_auto(mesh, rgl_compat = TRUE) img_rgl <- render_mesh(mesh, camera = cam) ``` This is especially useful when comparing outputs between scimesh and rgl, or when you prefer rgl's slightly elevated default perspective. ### Projection Type Perspective (default) or orthographic (parallel projection, matching rgl's `view3d(fov = 0)` convention): ```r render_options(projection = "orthographic") ``` ### Anti-Aliasing scimesh supports ordered-grid supersampling (SSAA). Pass `aa_samples = 2L` for 2x2 SSAA (renders internally at double resolution, downsamples by box averaging): ```r render_options(aa_samples = 2L) ``` Values of 1 (off), 2, or 4 are supported. Higher values give smoother edges but use proportionally more memory and time. ### Wireframe Mode ```r render_options(wireframe = TRUE, wireframe_color = c(0, 0, 0, 1)) # black edges ``` Edges are computed via barycentric distance testing inside the triangle rasterizer — no separate line primitives needed. Edge thickness adapts to triangle size so small triangles don't disappear. ### Screen-Space Ambient Occlusion (SSAO) SSAO adds contact shadows in crevices and concavities, dramatically improving depth perception and realism. It's a screen-space post-processing effect — no extra geometry needed: ```r render_options( ssao_enabled = TRUE, ssao_radius = 12, # sample radius in pixels ssao_intensity = 0.5) # occlusion strength (0–1) ``` Higher `ssao_radius` values sample a larger area (more expensive but softer shadows). Higher `ssao_intensity` darkens occluded regions more. Typical settings: radius 8–16, intensity 0.4–0.8. SSAO is demonstrated in the `spot_cow` and `dragon` R examples, and extensively in the C++ examples. ### Depth Fog Atmospheric depth fog fades distant geometry toward a background color, useful for emphasising foreground objects or creating stylised renders. By default the fog distances are given in **world units** (distance from the camera), so a scene looks the same no matter where the camera is placed: ```r render_options( fog_enabled = TRUE, fog_start = 20, # start fading 20 world units away fog_end = 60, # fully fogged 60 world units away fog_color = c(0.9, 0.95, 1, 1)) # pale blue fog ``` The fade is applied linearly between `fog_start` and `fog_end`, measured along the viewing direction. This is independent of the near/far clipping planes (`near_plane`, `far_plane`) and of the projection type. For backwards compatibility, the pre-0.3.5 behaviour of specifying fog in **normalized device depth** (the raw depth buffer, `-1` = near plane, `+1` = far plane) is still available, but has to be requested explicitly: ```r render_options( fog_enabled = TRUE, fog_space = "ndc", fog_start = 0.5, # half way into the depth range fog_end = 1.0) ``` ### Clip Planes Clip planes remove the geometry on one side of a plane, which is useful for cross-sections, cutaways, and "half-brain" style figures. A plane is built with `clip_plane()` and passed to `render_options(clip_planes = ...)`; several planes are combined with a logical AND. A point `p` is kept when `dot(normal, p) + offset >= 0`, i.e. `normal` points toward the part that stays visible. By default the plane is defined in **world space**, so the cut is a fixed feature of the scene and does not move when the camera is moved — which is what makes it usable for orbit videos and multi-view figures: ```r opts <- render_options(clip_planes = list( clip_plane(normal = c(-1, 0, 0), offset = 0))) # keep world x <= 0 ``` The same scene rendered from any camera position always shows the same cut. Pass `space = "eye"` for a plane that is defined **relative to the camera** and therefore travels with it (the classic OpenGL `glClipPlane` behaviour). This is handy for camera-attached cutaways, e.g. removing everything closer than 2 units to the viewer so that the camera always sees inside the mesh: ```r opts <- render_options(clip_planes = list( clip_plane(normal = c(0, 0, -1), offset = -2, space = "eye"))) ``` Both modes can be mixed in the same `clip_planes` list. ### Semi-Transparent Overlays scimesh blends meshes whose colors are not fully opaque. The fourth column of `colors` (or of `face_colors`, or the alpha of `color` for the generated meshes) is the alpha value, and there is nothing to switch on: a mesh with alpha < 1 takes part in the blended pass automatically. The renderer * interpolates per-vertex alpha across each triangle, so smooth fades work, * draws the translucent triangles back-to-front, after the opaque geometry, * depth-tests them against the opaque geometry, so translucent geometry behind an opaque mesh stays hidden and does not wash it out, * treats alpha 0 as fully invisible (`holes`), e.g. for the medial wall. `set_mesh_alpha(mesh, alpha)` is the shortcut for a uniform alpha (`set_mesh_alpha(mesh, 0.1)` for the 10 % brain shell used as a spatial reference). **R example:** ```{r, eval=FALSE} library(scimesh) white <- freesurferformats::read.fs.surface("sub-01/surf/lh.white") pial <- freesurferformats::read.fs.surface("sub-01/surf/lh.pial") nv <- nrow(white$vertices) white_mesh <- list( vertices = white$vertices, triangles = white$faces, colors = matrix(c(0.7, 0.7, 0.7, 1.0), nv, 4, byrow = TRUE)) pial_mesh <- list( vertices = pial$vertices, triangles = pial$faces, colors = matrix(c(0.9, 0.3, 0.2, 0.35), nv, 4, byrow = TRUE)) cam <- camera_auto(pial_mesh, direction = c(-1, 0, 0.2)) img <- render_scene(list(white_mesh, pial_mesh), cam, render_options(width = 1200, height = 900, backface_culling = FALSE, specular_color = c(0.4, 0.4, 0.4, 1), shininess = 64)) temp_file <- tempfile(fileext = ".png") write_png(img, temp_file) printf("Transparent render written to: %s\n", temp_file) ``` **Recipe: a half-transparent medial wall** (in this case with the cortex colored by sulcal depth, so the inside of the hemisphere stays visible): ```{r, eval=FALSE} surface <- freesurferformats::read.fs.surface("sub-01/surf/lh.white") sulc <- freesurferformats::read.fs.morph("sub-01/surf/lh.sulc") cortex <- freesurferformats::read.fs.label("sub-01/label/lh.cortex.label") nv <- nrow(surface$vertices) medial_wall <- !(seq_len(nv) %in% cortex) sulc[medial_wall] <- NA # no data on the medial wall colors <- apply_colormap(sulc, colormap = viridis_colormap(256L), nan_color = c(1, 1, 1, 1), winsor_percentiles = c(2, 98)) if (ncol(colors) == 3) colors <- cbind(colors, 1) colors[medial_wall, 4] <- 0.5 # 50 % transparent brain <- list(vertices = surface$vertices, triangles = surface$faces, colors = colors) img <- render_scene(list(brain), camera_auto(brain, direction = c(1, 0, 0)), render_options(background_color = c(1, 1, 1, 1))) ``` Per-vertex alpha can vary freely, which makes gradients possible: ```{r, eval=FALSE} quads <- generate_plane(color = c(0.8, 0.2, 0.2, 1)) quads$colors[, 4] <- rep(c(0, 1), each = nrow(quads$colors) / 2) ``` **File formats:** PLY vertex colors are read with their alpha (8-bit or float), including RGBA files whose alpha is stored with a different type than the RGB channels. OBJ and STL do not carry transparency, and the FreeSurfer converters take it from the data: `mesh_from_fs(..., detect_transparency = TRUE)` (C++) turns vertices without data into holes, and `convert_fs_mesh(fs_mesh, morph, rgb, nan_alpha = 0.5f)` makes them half-transparent instead. **C++ example:** See `examples/cpp/transparency/` in the repository. **Known limitations:** transparency is resolved per triangle (painter's algorithm), so *intersecting*, *coplanar* or very large translucent triangles can blend in the wrong order, and a translucent surface never occludes anything (it writes no depth). Point rendering (`render_points()`) does not blend at all; use `render_spheres()` (which renders meshes) for translucent spheres. ### Text Labels `text_layer()` adds text to a scene, which is what figures need to name a region, an atom or a panel. Labels are **billboards**: they always face the camera and keep the size given in `size` (in output pixels), so they stay readable from every viewpoint — unlike text that is converted into 3D geometry, which skews as the camera moves. Labels create no geometry, they are drawn after the meshes and lines (so they appear on top of the model), and the font ships with the package (Inter, SIL OFL 1.1), so no system font is required. Positions are either **world space** (the default) or **screen space** (`space = "screen"`, measured in output pixels from the top-left corner): ```{r, eval=FALSE} surf <- generate_sphere(c(0, 0, 0), radius = 1) cam <- camera_auto(list(surf), direction = c(0, 0, 1)) opts <- render_options(width = 500, height = 400, background_color = c(1, 1, 1, 1)) sc <- scene(list(surf), camera = cam, options = opts, texts = list( # world space: anchored above the sphere, follows the camera text_layer(matrix(c(0, 1.3, 0), ncol = 3), "superior", size = 20, adj = c(0.5, 0)), # a direction label that must stay readable: no depth test text_layer(matrix(c(1.4, 0, 0), ncol = 3), "anterior", size = 18, adj = c(0, 0.5), colors = c(0.75, 0.15, 0.15, 1), depth_test = FALSE), # screen space: a panel tag in the top-left corner text_layer(c(12, 10), "A", space = "screen", adj = c(0, 1), size = 28, halo_color = c(1, 1, 1, 0.9)), # screen space: a vertical axis label (rotation in degrees, # counter-clockwise, about the anchor) text_layer(c(16, 210), "sulcal depth", space = "screen", adj = c(0, 0.5), size = 16, rotation = 90))) img <- render_scene(sc) ``` Labelling a brain surface works exactly the same way: the labels carry world coordinates of the region they name, and are hidden when the surface is in front of them (unless `depth_test = FALSE`). ```{r, eval=FALSE} surface <- freesurferformats::read.fs.surface("sub-01/surf/lh.white") sulc <- freesurferformats::read.fs.morph("sub-01/surf/lh.sulc") colors <- apply_colormap(sulc, colormap = viridis_colormap(256L), winsor_percentiles = c(2, 98)) if (ncol(colors) == 3) colors <- cbind(colors, 1) brain <- list(vertices = surface$vertices, triangles = surface$faces, colors = colors) cam <- camera_auto(brain, direction = c(1, 0, 0)) # lateral view # The free surface is roughly 100 mm wide, so the labels sit at x = +-55 mm. sc <- scene(list(brain), camera = cam, options = render_options(width = 600, height = 500, background_color = c(1, 1, 1, 1)), texts = list( text_layer(matrix(c(-55, 0, 0), ncol = 3), "posterior", size = 22, adj = c(1, 0.5), depth_test = FALSE), text_layer(matrix(c(55, 0, 0), ncol = 3), "anterior", size = 22, adj = c(0, 0.5), depth_test = FALSE))) img <- render_scene(sc) ``` Things worth knowing: - `adj` places the anchor on the text box, exactly like `adj` in base R graphics: `c(0, 0)` is the bottom-left corner, `c(1, 1)` the top-right one, and `c(0.5, 0.5)` (default) centres the text on the position. - `rotation` turns the label about its anchor, in degrees, counter-clockwise: 90 writes along a vertical axis (a y-axis label), 180 upside down, and any other angle lets a label follow an annotation line. The rotation happens in the image plane, so it works for screen-space and world-space labels alike. - `offset` shifts the label by a number of pixels after anchoring, which is the easy way to put an atom label *next to* an atom instead of onto it. - `depth_test` (default `TRUE`) hides a world-space label when geometry is in front of its anchor. Use `FALSE` for annotations that must always be readable, and for labels anchored on a surface. - `halo_color` (with `halo_width`) draws an outline behind the glyphs, which keeps labels readable on dark or busy geometry. - `colors` sets per-label colours, `"\n"` starts a new line, and `text_extent()` measures a label so you can right-align a caption or keep two labels from overlapping. - `world_to_screen()` converts world coordinates into the pixels of a rendered image, which is what you need to place a screen-space label next to a location in the scene (or to draw a callout line with `line_layer()`). - No mesh at all is needed for screen-space labels: `render_text()` draws labels onto the background described by the render options. - Replace the typeface with `font_file = "path/to/font.ttf"`, or for the whole session by setting the `SCIMESH_FONT` environment variable. The same feature is available in the C++ API (`TextLayer`, `Scene::add_texts()`, `draw_text()`, see `examples/cpp/text_labels/`) and in the CLI renderer (`--text`, `--text-at`, `--text-size`, `--text-screen`, `--text-rotation`, ...). ### Background and Transparency By default, scimesh renders with a transparent background (`background_color = c(0, 0, 0, 0)`), which is ideal for compositing. For a solid background (e.g., white for papers), set: ```r render_options(background_color = c(1, 1, 1, 1)) # solid white ``` The background color is written to the output image's alpha channel, so transparent backgrounds survive PNG export and can be further composited in tools like ImageMagick or layout packages. ## Working with Meshes ### Procedural Geometry scimesh provides C++ and R functions for generating primitive geometry: ```r cube <- generate_cuboid(c(0, 0, 0), c(1, 1, 1), c(1, 0, 0, 1)) sphere <- generate_sphere(c(0, 0, 0), radius = 1.2, segments = 32, color = c(0.9, 0.3, 0.2, 1)) cyl <- generate_cylinder(c(0, -1, 0), c(0, 1, 0), 0.5, 32, c(0.1, 0.7, 0.3, 1)) cone <- generate_cone(c(0, -1.2, 0), c(0, 1.2, 0), 0.6, 32, c(0.9, 0.7, 0.1, 1)) pyramid <- generate_pyramid(c(0, 0, 0), c(0, 1.5, 0), 1, c(0.7, 0.2, 0.8, 1)) tetra <- generate_tetrahedron(c(-1, -0.5, -1), c(1, -0.5, -1), c(0, -0.5, 1), c(0, 1.2, 0), c(0.2, 0.8, 0.8, 1)) torus <- generate_torus(c(0, 0, 0), 1.0, 0.35, 24, 12, c(0.6, 0.4, 0.2, 1)) plane <- generate_plane(c(0, 0, 0), c(0, 1, 0), 1.2, 0.8, c(0.5, 0.5, 0.5, 1)) ``` See `examples/R/primitives/run.R` for a gallery script that renders all primitives side-by-side in both shaded and wireframe mode. ### Mesh I/O scimesh can read and write standard 3D mesh file formats. All readers return a mesh descriptor list with `vertices`, `triangles`, and optionally `normals`, `uv`, or `colors`. ```{r, eval=FALSE} # Read a Wavefront OBJ file (with optional UVs and normals) mesh <- read_obj("model.obj") # Read a Stanford PLY file (with optional vertex colors) mesh <- read_ply("model.ply") # Read an STL file (binary or ASCII) mesh <- read_stl("model.stl") # Write a mesh to STL tmp_file1 <- tempfile(fileext = ".stl") tmp_file2 <- tempfile(fileext = ".stl") write_stl(mesh, tmp_file1) # binary (default) write_stl(mesh, tmp_file2, format = "ascii") ``` The OBJ reader supports multi-shape files and texture coordinates. The PLY reader supports per-vertex RGB colors. STL writes preserve vertex normals when available. ### Mesh Transforms Translate, scale, rotate, or apply arbitrary 4x4 matrices to meshes: ```r mesh <- generate_cuboid(c(0, 0, 0), c(1, 1, 1), c(1, 0, 0, 1)) mesh <- translate_mesh(mesh, c(5, 0, 0)) mesh <- scale_mesh(mesh, 2.0) mesh <- rotate_mesh(mesh, pi / 4, c(0, 0, 1)) mesh <- transform_mesh(mesh, my_4x4_matrix) ``` ### Face Colors Per-face coloring lets you assign a single color to every vertex of a triangle. This is useful for parcellation overlays or material assignment: ```r mesh <- generate_cuboid(c(0, 0, 0), c(1, 1, 1)) # 12 triangles, each gets a color fc <- matrix(c(1, 0, 0, 1), nrow = 12, ncol = 4, byrow = TRUE) img <- render_mesh(mesh$vertices, mesh$triangles, face_colors = fc) ``` When `face_colors` is provided, it takes precedence over vertex `colors`. ### Texture Mapping Load a texture image, give the mesh UV coordinates and render it: ```r # Requires the 'png' package library(png) tex <- readPNG("texture.png") # H x W x C array, row 1 = top of the image # UV coordinates (Nx2, values 0-1) must match the vertex order of the mesh. # scimesh UVs are image-space: c(0, 0) is the TOP LEFT pixel of the texture. uv <- cbind(c(0, 1, 1, 0), c(0, 0, 1, 1)) # e.g. a quad img <- render_mesh(mesh$vertices, mesh$triangles, uv = uv, texture = tex) ``` Bilinear texture sampling is used for smooth results, and the texel is multiplied with the vertex color (so use white vertices to show the texture unchanged). UVs are read from files by **no** scimesh reader: `read_obj()` and `read_ply()` return geometry (and colors/normals where available), but drop texture coordinates, so you have to supply `uv` yourself. Note also that OBJ files, rgl, OpenGL and tools like Blender store UVs with the *opposite* origin (`v = 0` at the bottom of the image); convert such coordinates once with `flip_uvs()`: ```r # UVs taken from an OBJ file / another 3D tool: v = 0 is the bottom uv_obj <- rbind(c(0, 1), c(1, 1), c(1, 0), c(0, 0)) mesh <- flip_uvs(list(vertices = mesh$vertices, triangles = mesh$triangles, uv = uv_obj)) img <- render_mesh(mesh$vertices, mesh$triangles, uv = mesh$uv, texture = tex) ``` **C++ example:** `examples/cpp/spot_cow/` renders Keenan Crane's textured cow from an OBJ file plus a PNG texture, including the UV conversion. ### Mesh Utilities #### Bounding Box Compute or visualise the axis-aligned bounding box: ```r mesh <- generate_torus(c(0, 0, 0), 1.5, 0.4, 32, 16) bbox <- mesh_bbox(mesh) bbox$min # c(xmin, ymin, zmin) bbox$max # c(xmax, ymax, zmax) # Generate a wireframe bounding box mesh for rendering bbox_mesh <- generate_bbox(mesh, color = c(0, 0, 0, 1), radius = 0.02) ``` #### Axis Arrows Generate coloured XYZ axis arrows (red X, green Y, blue Z): ```r axes_mesh <- generate_axes(center = c(0, 0, 0), size = 2) # Render together with your mesh: img <- render_scene(list(mesh, axes_mesh), cam) ``` ## Lower-Level Rendering Beyond `render_mesh()` and `render_scene()`, scimesh provides lower-level functions for special use cases. ### Raw Triangles (no index buffer) Render geometry where positions and colors are flat arrays with 3 vertices per triangle — useful for dynamically generated geometry: ```r positions <- matrix(c(0,0,0, 1,0,0, 0.5,1,0, 0,0,1, 1,0,1, 0.5,1,1), ncol = 3, byrow = TRUE) colors <- matrix(c(1,0,0,1, 0,1,0,1, 0,0,1,1, 1,1,0,1, 0,1,1,1, 1,0,1,1), ncol = 4, byrow = TRUE) cam <- camera_auto(positions) img <- render_triangles(positions, colors, cam) ``` ### Points Render point cloud data with depth-tested circular markers: ```r pts <- matrix(rnorm(300), ncol = 3) cols <- matrix(c(1, 0.5, 0, 1), nrow = nrow(pts), ncol = 4, byrow = TRUE) img <- render_points(pts, cols, radius = 4) ``` ### Lines (as Cylinders) Render line segments as thin cylinders: ```r from <- matrix(c(0,0,0, 0,0,0), ncol = 3, byrow = TRUE) to <- matrix(c(1,0,0, 0,1,0), ncol = 3, byrow = TRUE) cols <- matrix(c(1,0,0,1, 0,1,0,1), ncol = 4, byrow = TRUE) img <- render_lines(from, to, radii = 0.05, colors = cols, cam) ``` ## Image Utilities ### Colour Bars scimesh can produce horizontal or vertical colour bars in pure R (no X11), ready to be composed alongside rendered images: ```r cbar <- colorbar_horizontal(viridis_colormap, n_colors = 256, width = 600, height = 80, ticks = c(0, 0.5, 1), tick_labels = c("min", "mid", "max"), title = "Value") ``` Both `colorbar_horizontal()` and `colorbar_vertical()` accept any colormap, specified either as: - **A function** returning hex colors (e.g., `viridis_colormap`, `diverging_colormap`, `grDevices::hcl.colors`, or `viridisLite::viridis`) - **A vector of color strings** (e.g., `c("red", "white", "blue")`) Built-in colormaps (no extra packages required): ```r # Viridis (perceptually uniform, colourblind-friendly) viridis_colormap(256) # Blue-white-red diverging (for signed data like Z-scores) diverging_colormap(256) # Any base R palette via wrapper functions my_cmap <- function(n) grDevices::hcl.colors(n, palette = "inferno") cbar <- colorbar_horizontal(my_cmap) # Or pass colors directly cbar <- colorbar_horizontal(c("darkblue", "cyan", "yellow", "red")) ``` If you have `viridisLite` installed, you can use its full colormap family (magma, inferno, plasma, cividis) directly: ```r library(viridisLite) cbar <- colorbar_horizontal(viridis) # viridisLite function ``` See `examples/R/colormaps/run.R` for a complete demonstration. ### Compositing Multiple Images `compose_layout()` arranges rendered images in a grid with optional per-row/per-column cropping to eliminate wasted whitespace: ```r img1 <- render_mesh(...) img2 <- render_mesh(...) result <- compose_layout(list(img1, img2), nrow = 1L, crop = TRUE) tmp_file <- tempfile(fileext = ".png") write_png(result, tmp_file) printf("Composed image written to: %s\n", tmp_file) ``` ### Image Stacking Quick helpers for combining images: ```r # Horizontal stack (side by side) result <- stack_horizontal(img1, img2, img3) # Vertical stack (one below another) result <- stack_vertical(img1, img2) # With a colorbar result <- stack_horizontal(img1, img2, colorbar = colorbar_horizontal(viridis_colormap)) ``` ## Interoperability with Other Packages ### rgl / tmesh3d scimesh and [rgl](https://cran.r-project.org/package=rgl) mesh formats can be converted in both directions, and scimesh render functions accept rgl meshes **transparently** — no manual conversion needed. #### Rendering an rgl mesh directly Pass an rgl `tmesh3d` object (or any list with `vb` and `it` components) directly to `render_mesh()` or include it in a scene list for `render_scene()` — the conversion happens automatically: ```r if (requireNamespace("rgl", quietly = TRUE)) { rgl_mesh <- rgl::tetrahedron3d() img <- render_mesh(rgl_mesh) # transparent conversion } ``` This also works when mixing scimesh and rgl meshes in a scene: ```r cube <- generate_cuboid(c(-1, 0, 0), c(0.5, 0.5, 0.5)) img <- render_scene(list(cube, rgl_mesh), cam) # mixed formats ``` #### rgl → scimesh (explicit) If you prefer explicit conversion, use `mesh_from_rgl()`. This also works **without the rgl package** installed — any list with `vb` and `it` components is accepted: ```r tmesh <- list( vb = rbind(c(-1, -1, 1, 1), c(-1, 1, 1, -1), c(0, 0, 0, 0), c(1, 1, 1, 1)), it = rbind(c(1L, 1L), c(2L, 3L), c(3L, 4L))) mesh <- mesh_from_rgl(tmesh) img <- render_mesh(mesh) ``` #### scimesh → rgl (inverse) Convert a scimesh mesh back to rgl's `tmesh3d` format with `mesh_to_rgl()`. Vertex colors are forwarded automatically if present in the mesh: ```r mesh <- generate_cuboid(c(0, 0, 0), c(1, 1, 1), color = c(1, 0, 0, 1)) rgl_mesh <- mesh_to_rgl(mesh) if (requireNamespace("rgl", quietly = TRUE)) { rgl::shade3d(rgl::tmesh3d( vertices = rgl_mesh$vb, indices = rgl_mesh$it), col = "red") } ``` The returned list has `vb` (4×N homogeneous coordinates), `it` (3×M index matrix), and optionally `normals` and `mat` (material/colors). #### Matching rgl's Default View When rendering an rgl mesh with scimesh, you may want the output to visually match what you'd see in an rgl window. Pass `rgl_compat = TRUE` to `camera_auto()` to get rgl's default 30° FOV, 15° elevation, and bounding-sphere distance: ```r if (requireNamespace("rgl", quietly = TRUE)) { rgl_mesh <- rgl::cube3d() # rgl primitive cam <- camera_auto(rgl_mesh, rgl_compat = TRUE) img <- render_mesh(rgl_mesh, camera = cam) write_png(img, "cube_rgl_view.png") } ``` The `rgl_compat` camera uses the same algorithm rgl uses internally: it computes the bounding sphere (half-diagonal of the AABB) and places the eye at `distance = sphere_radius / sin(FOV / 2)` along a direction tilted 15° above the −Z axis. #### Rvcg, Morpho, fsbrain, and other packages The rgl `mesh3d`/`tmesh3d` format is the lingua franca for 3D meshes in R. Packages like [Rvcg](https://cran.r-project.org/package=Rvcg) (mesh processing via VCGLIB), [Morpho](https://cran.r-project.org/package=Morpho) (geometric morphometrics), and [fsbrain](https://cran.r-project.org/package=fsbrain) (neuroimaging) all produce and consume rgl-format meshes. Since scimesh transparently accepts rgl meshes, you can process a mesh with any of these packages and render it with scimesh **without any conversion step**: ```r if (requireNamespace("Rvcg", quietly = TRUE)) { mesh <- Rvcg::vcgSphere() # returns rgl-format mesh img <- render_mesh(mesh) # scimesh renders it directly } ``` ## Examples The repository includes runnable R example scripts in `examples/R/`: | Example | What it shows | |---------|--------------| | `spot_cow/run.R` | Textured OBJ mesh with multi-light setup and SSAO | | `dragon/run.R` | Stanford Dragon with specular highlights and 4x AA | | `primitives/run.R` | All procedural primitives in shaded + wireframe gallery | | `transparency/run.R` | Semi-transparent pial overlay on white matter | | `whole_brain_sulc/run.R` | Whole-brain sulcal depth with cortex masking | | `video_frames_orbit/run.R` | Turntable orbit frame sequence | | `colormaps/run.R` | Custom colormaps with colorbars | Run all examples at once: ```sh cd examples/R bash run_all_R.sh ``` ## FAQ **How fast is it?** A typical cortical surface (~300k triangles) at 1200x900 with 2x SSAA renders in 1--3 seconds on a modern CPU. Smaller meshes at lower resolution can render in 200 ms or less, depending on the CPU and number of lights. **Why is there no interactive 3D window in which I can rotate the mesh?** scimesh is a headless (off-screen) renderer, also known as a software rasterizer. It produces images in roughly seconds, on just the CPU. For interactive rotation or real-time animation, you need to render at least 30 images *per second*. This is only achievable with a hardware renderer, i.e., a graphics card and the full software stack required to make use of it. If you have a graphics card, the typical solution in R is to use rgl/OpenGL for interactive visualization. **What about volume data, like 3D MRI scans in neuroimaging?** scimesh renders 3D surface meshes. Volume slice visualisation (e.g., `volvis.lb.with.surface()` in fsbrain) is done entirely in R/magick without 3D rendering --- it works independently of the renderer backend. Technically it's just a 2D image, no renderer needed. This means you can use the functions in fsbrain for volume visualisation even if you do not have rgl/OpenGL, and are using scimesh for surface (mesh) rendering. **My images look pixelated / jagged --- how do I fix this?** Render at higher resolution, e.g. `(render_options(width = 2560, height = 1440))`, and enable anti-aliasing: `render_options(aa_samples = 2L)`, or even higher like `4L`. **How do I get a transparent background?** This is the default (`background_color = c(0, 0, 0, 0)`). See the Background and Transparency section under Rendering Features. **How do I control the number of CPU threads?** Set the `threads` option in `render_options()`. The default 0 automatically uses all available cores. Set to 1 for single-threaded rendering, or any positive integer to cap the thread count. **What affects render performance?** The main factors are triangle count, output resolution (pixels), anti-aliasing level (`aa_samples`), number of lights, SSAO, and whether transparency sorting is needed. For largest meshes, the most effective optimisations are reducing `aa_samples` and output resolution. SSAO and multi-light setups each add roughly constant overhead per pixel.