|
|
|
|
|
|
|
Papers related to Virtual Light Fields and Global Illumination in general
Virtual Light Fields, Light Field rendering and
Lumigraphs
|
Slater, M., "Tutorial on Light Field Rendering", presented at VRST'2000, October 2000. |
|
Abstract These notes introduce the concept and some aspects of the implementation of light fields for computer graphics rendering. All computer graphics rendering may be thought of as solutions to a integral equation which expresses the radiance at any point on a surface in a scene in any direction. Light fields may be thought of as a kind of brute force solution to this equation, made possible today by the vast increases in memory and processing power that have occurred over the past few years. We therefore first introduce this radiance equation, and provide a framework for discussion of the various approaches to rendering in this context, and prior to this we present some of the background mathematics for completeness. Then we introduce the traditional approach to light field representation and rendering. Finally we argue that the light field approach provides a different paradigm for computer graphics rendering of virtual scenes, and which can include global illumination. We call this a virtual light field, and give an illustration as to how the goal might be achieved. |
|
|
|
Schirmacher, H., Heidrich, W. and Seidel, H-P., "High-Quality Interactive Lumigraph Rendering Through Warping", Graphics Interface 2000, 2000. |
|
Abstract We introduce an algorithm for high-quality, interactive light field rendering from only a small number of input images with dense depth information. The algorithm bridges the gap between image warping and interpolation from image databases, which represent the two major approaches in image based rendering. By warping and blending only the necessary parts of each reference image, we are able to generate a single view-corrected texture for every output frame at interactive rates. In contrast to previous light field rendering approaches, our warping-based algorithm is able to fully exploit per-pixel depth information in order to depth-correct the light field samples with maximum accuracy. The complexity of the proposed algorithm is nearly independent of the number of stored reference images and of the final screen resolution. It performs with only small overhead and very few visible artifacts. We demonstrate the visual fidelity as well as the performance of our method through various examples. |
|
|
|
Chai, J., Tong, X., Chan, S. and Shum, H., "Plenoptic Sampling", SIGGRAPH 2000, 2000. |
|
Abstract This paper studies the problem of plenoptic sampling in image-based rendering (IBR). From a spectral analysis of light field signals and using the sampling theorem, we mathematically derive the analytical functions to determine the minimum sampling rate for light field rendering. The spectral support of a light field signal is bounded by the minimum and maximum depths only, no matter how complicated the spectral support might be because of depth variations in the scene. The minimum sampling rate for light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval. Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering. Plenoptic sampling goes beyond the minimum number of images needed for anti-aliased light field rendering. More significantly, it utilizes the scene depth information to determine the minimum sampling curve in the joint image and geometry space. The minimum sampling curve quantitatively describes the relationship among three key elements in IBR systems: scene complexity (geometrical and textural information), the number of image samples, and the output resolution. Therefore, plenoptic sampling bridges the gap between image-based rendering and traditional geometry-based rendering. Experimental results demonstrate the effectiveness of our approach. |
|
|
|
Slater, M., "A Note on Virtual Light Fields", University College London, 2000. |
|
Abstract This short paper describes an attempt to construct a light field for virtual scenes with global illumination. The light field is constructed directly without the necessity for intermediate images produced by another rendering program. The light field is represented as a set of parallel subfields, where each subfield is a set of rectangularly organized parallel rays in some orientation. It is shown that rendering an object into the 4-dimensional ray space can be decomposed into a set of 2-dimen-sional rasterisations. Radiance is propagated through the light field starting from the emitters, using a method similar to progressive refinement radiosity. Once the light field is illuminated in this way, sets of parallel rays can be projected onto a lens and accumulate radiance onto an image plane. Thus constant time rendering walkthrough can be achieved. The light field so generated is approximate, and the resulting images are poor. However, this note suggests a line of research which may prove useful in the longer term. |
|
|
|
Wood, D. N., Azuma, D. I., Aldinger, K., Curless, B., Duchamp, T., Salesin, D. H. and Stuetzle, W., "Surface Light Fields for 3D Photography", SIGGRAPH 2000, 2000. |
|
Abstract A surface light field is a function that assigns a color to each ray originating on a surface. Surface light fields are well suited to constructing virtual images of shiny objects under complex lighting conditions. This paper presents a framework for construction, compression, interactive rendering, and rudimentary editing of surface light fields of real objects. Generalizations of vector quantization and principal component analysis are used to construct a compressed representation of an object’s surface light field from photographs and range scans. A new rendering algorithm achieves interactive rendering of images from the compressed representation, incorporating view-dependent geometric level-of-detail control. The surface light field representation can also be directly edited to yield plausible surface light fields for small changes in surface geometry and reflectance properties. |
|
|
|
Isaksen, A., McMillan, L. and Gortler, S. J., "Dynamically Reparameterized Light Fields", SIGGRAPH 2000, 2000. |
|
Abstract This research further develops the light field and lumigraph image-based rendering methods and extends their utility. We present alternate parameterizations that permit 1) interactive rendering of moderately sampled light fields of scenes with significant, unknown depth variation and 2) low-cost, passive autostereoscopic viewing. Using a dynamic reparameterization, these techniques can be used to interactively render photographic effects such as variable focus and depth-of-field within a light field. The dynamic parameterization is independent of scene geometry and does not require actual or approximate geometry of the scene. We explore the frequency domain and ray-space aspects of dynamic reparameterization, and present an interactive rendering technique that takes advantage of today’s commodity rendering hardware. |
|
|
|
Yu, Y., Debevec, P., Malik, J. and Hawkins, T., "Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs", SIGGRAPH 1999, 1999. |
|
Abstract In this paper we present a method for recovering the reflectance properties of all surfaces in a real scene from a sparse set of photographs, taking into account both direct and indirect illumination. The result is a lighting-independent model of the scene’s geometry and reflectance properties, which can be rendered with arbitrary modifications to structure and lighting via traditional rendering methods. Our technique models reflectance with a low-parameter reflectance model, and allows diffuse albedo to vary arbitrarily over surfaces while assuming that non-diffuse characteristics remain constant across particular regions. The method’s input is a geometric model of the scene and a set of calibrated high dynamic range photographs taken with known direct illumination. The algorithm hierarchically partitions the scene into a polygonal mesh, and uses image-based rendering to construct estimates of both the radiance and irradiance of each patch from the photographic data. The algorithm computes the expected location of specular high-lights, and then analyzes the highlight areas in the images by running a novel iterative optimization procedure to recover the diffuse and specular reflectance parameters for each region. Lastly, these parameters are used in constructing high-resolution diffuse albedo maps for each surface. The algorithm has been applied to both real and synthetic data, including a synthetic cubical room and a real meeting room. Re-renderings are produced using a global illumination system under both original and novel lighting, and with the addition of synthetic objects. Side-by-side comparisons show success at predicting the appearance of the scene under novel lighting conditions. |
|
|
|
Schirmacher, H., Heidrich, W. and Seidel, H-P., "Adaptive Acquisition of Lumigraphs from Synthetic Scenes" , EUROGRAPHICS 1999 vol. 18, no. 3, 1999. |
|
Abstract Light fields and Lumigraphs are capable of rendering scenes of arbitrary geometrical or illumination complexity in real time. They are thus interesting ways of interacting with both recorded real-world and high-quality synthetic scenes. Unfortunately, both light fields and Lumigraph rely on a dense sampling of the illumination to provide a good rendering quality. This induces high costs both in terms of storage requirements and computational resources for the image acquisition. Techniques for acquiring adaptive light field and Lumigraph representations are thus mandatory for practical applications. In this paper we present a method for the adaptive acquisition of images for Lumigraphs from synthetic scenes. Using image warping to predict the potential improvement in image quality when adding a certain view, we decide which new views of the scene should be rendered and added to the light field. This a-priori error estimator accounts for both visibility problems and illumination effects such as specular highlights. |
|
|
|
Camahort, E. and Fussell, D., "A Geometric Study of Light Field Representations", Technical Report TR99-35, 1999. |
|
Abstract A simple light-field model represents radiance through each point in 3D space along each direction in 2D directional space. Due to their spatial complexity, light-field models in Computer Graphics restrict the support of the light-field function to 4D oriented line space. This paper studies 4D light-field representations in both the continuous and the discrete line spaces. We first give a theoretical introduction to light fields and their continuous representations. Then we define the concept of uniform line-space parameterization and discuss its relationship to light-field rendering. The second part of this paper surveys current implementations of discrete light-field models. We describe the features of each implementation and their rendering artifacts. Then we define three geometric error measures for light-field models and use them to evaluate the implementations. Finally, we study the measures of lines represented by different configurations of the above light-field representations. |
|
|
|
Heidrich, W., Lensch, H., Cohen, M. F. and Seidel, H-P., "Light Field Techniques for Reflections and Refractions", Eurographics Rendering Workshop 1999, June 1999. |
|
Abstract Reflections and refractions are important visual effects that have long been considered too costly for interactive applications. Although most contemporary graphics hardware supports reflections off curved surfaces in the form of environment maps, refractions in thick, solid objects cannot be handled with this approach, and the simplifying assumptions of environment maps also produce visible artifacts for reflections. Only recently have researchers developed techniques for the interactive rendering of true reflections and refractions in curved objects. This paper introduces a new, light field based approach to achieving this goal. The method is based on a strict decoupling of geometry and illumination. Hardware support for all stages of the technique is possible through existing extensions of the OpenGL rendering pipeline. In addition, we also discuss storage issues and introduce methods for handling vector-quantized data with graphics hardware. |
|
|
|
Lalonde, P. and Fournier, A., "interactive Rendering of Wavelet Projected Light Fields", Graphics Interface 1999, June 1999. |
|
Abstract Light field techniques allow the rendering of objects in time complexity unrelated to their geometric complexity. The technique discretely samples the space of light rays exiting the boundary around an object and then reconstructs a requested view from these data. In order to generate high quality images a dense sampling of the space is required which leads to large data sets. These data sets exhibit a high degree of coherence and should be compressed in order to make their size manageable. We present a wavelet-based method for storing light fields over planar domains. The parameterization is based on the Nusselt embedding, which leads to simplifications in shading computations when the light fields are used illumination sources. The wavelet transform exploits the coherence in the data to reduce the size of the data sets by factors of 20 times or more without objectionable deterioration in the rendered images. The wavelet representation also allows a hierarchical representation in which detail can be added incrementally, and in which each coarser view is an appropriately filtered version of the finer detail. The wavelet coefficients are compressed by thresholding the coefficients and storing them in a sparse hexadecary tree. The tree encoding allows random access over the compressed wavelet coefficients which is essential for extracting slices and point samples from the light field. |
|
|
|
Muller, H. and Hinkenjann, A., "Linear Line Space Meshing For Light Fields", Research Report No. 716, June 1999. |
|
Abstract A general mesh type for representing the light field of a three-dimensional scene of (directional) diffusely and specularly reflecting surfaces, and anisotropically diffuse light sources is presented. The light field is represented by a piecewise linear function. The mesh emerges as a product of surface cells, but is more adaptive than the usual regular product meshes. Adaptivity is achieved by allowing hanging nodes on which the radiance is interpolated without cracks where continuity is required, and by hierarchization. |
|
|
|
Azuma, D., "Interactive Rendering of Surface Light Fields", Technical Report UW-CSE-2000-04-01, May 1999. |
|
Abstract Many interactive 3D applications involve capturing, representing and reproducing the appearance of real objects. The surface light field represents the appearance of a solid object under static lighting by sampling the radiance along rays parametrized by surface position and direction. An implementation can represent this data as high-resolution surface geometry, coupled with multiple texture maps, each associated with a direction. The representation can be rendered by a multipass algorithm utilizing common graphics hardware. Through careful optimization, it is possible to achieve interactive frame rates using existing implementations of OpenGL. Surface light fields tend to produce higher resolution renderings than similar methods such as the lumigraph, and do not exhibit occlusion ghosts and other artifacts associated with the lumigraph at low sampling rates. Construction and optimization of surface light field data present a number of challenges. |
|
|
|
Vogelgsang, C. and Greiner. G., "Hardware Accelerated Light Field Rendering", Technical Report 11/1999, Uni Erlangen, 1999. |
|
Abstract Light field rendering has become one of the major topics in image based rendering. To extend the wide-spread use of these techniques it is crucial to have fast and efficient rendering algorithms. In this paper we will present methods for accelerating the display of traditional two-plane-parametrized light fields and lumigraphs. The proposed algorithms use standard graphics hardware. First we analyse the well-known approaches to light field rendering. Then we improve them by introducing techniques for faster geometry setup and texture handling. Additionally some extensions to the common rendering methods are described and finally we show some results by comparing the known approaches with our new techniques. |
|
|
|
Camahort, E., Lerios, A. and Fussel, D., "Uniformly Sampled Light Fields", Proc. of the 9th EUROGRAPHICS Workshop on Rendering, 1998. |
|
Abstract Image-based or light field rendering has received much recent attention as an alternative to traditional geometric methods for modeling and rendering complex objects. A light field represents the radiance flowing through all the points in a scene in all possible directions. We explore two new techniques for efficiently acquiring, storing, and reconstructing light fields in a (nearly) uniform fashion. Both techniques sample the light field by sampling the set of lines that intersect a sphere tightly fit around a given object. Our first approach relies on uniformly subdividing the sphere and representing this subdivision in a compact data structure which allows efficient mapping of image pixels or rays to sphere points and then to subdivision elements. We sample a light field by joining pairs of subdivision elements and store the resulting samples in a multi-resolution, highly compressed fashion that allows efficient rendering. Our second method allows a uniform sampling of all five dimensions of the light field, using hierarchical subdivision for directional space and uniform grid sampling for positional space. Light field models are acquired using parallel projections along a set of uniform directions. Depth information can also be stored for high-quality image rendering. The system can provide bounds on key sources of error in the representation and can be generalized to arbitrary scenes comprising multiple complex objects. |
|
|
|
Heidrich, W., Kautz, J., Slusallek, P and Seidel, H-P., "Canned Lightsources", Proceedings of the 9th Eurographics Workshop on Rendering, 1998. |
|
Abstract Complex luminaries and lamp geometries can greatly increase the realism of synthetic images. Unfortunately, the correct rendering of illumination from complex lamps requires costly global illumination algorithms to simulate the indirect illumination reflected or refracted by parts of the lamp. Currently, this simulation has to be repeated for every scene in which a lamp is to be used, and even for multiple instances of a lamp within a single scene. In this paper, we separate the global illumination simulation of the interior lamp geometry from the actual scene rendering. The lightfield produced by a given lamp is computed using any of the known global illumination algorithms. Afterwards, a discretized version of this lightfield is stored away for later use as a lightsource. We describe how this data can be efficiently utilized to illuminate a given scene using a number of different rendering algorithms, such as ray-tracing and hardware-based rendering. |
|
|
|
Ihm, I., Park, S. and Lee, R. K., "Rendering of Spherical Light Fields", Proceedings of the 5th Pacific Conference on Computer Graphics and Applications, 1997. |
|
Abstract A plenoptic function is a parameterized function describing the flow of light in space, and has served as a key idea in building some of the recent image-based rendering systems. This paper presents a new representation scheme, called a spherical light field, of the plenoptic function, that is based on spheres. While methods using spherical coordinates are thought to require substantially more computation than those using planar or cylindrical coordinates, we show that spheres can also be used efficiently in representing and resampling the flow of light. Our image-based rendering algorithm is different from the previous systems, the light field and lumigraph, in that it is an "object-space" algorithm that can be easily embedded into the traditional polygonal rendering system. Our method is easily accelerated by 3D graphics boards that support the primitive functionality, such as viewing and smooth shading. In addition, we introduce an encoding scheme based on wavelets for compression of the huge data resulting from sampling of the spherical light field. The proposed technique can be easily adapted to compress the light field and lumigraph data, and offers as high compression ratios as the previous methods. Furthermore, it naturally creates a multi-resolutional representation of the light flow that can be exploited effectively in the future applications. We show how to access the compressed data efficiently using a modified significance map and an incremental decoding technique, and report experimental results on several test data sets. |
|
|
|
Gu, X., Gortler, S. J. and Cohen, M. F., "Polyhedral Geometry and the Two-Plane Parameterization", Eighth Eurographics Workshop on Rendering, 1997. |
|
Abstract Recently the light-field and lumigraph systems have been proposed as general methods of representing the visual information present in a scene. These methods represent this information as a 4D function of light over the domain of directed lines. These systems use the intersection points of the lines on two planes to parameterize the lines in space. This paper explores the structure of the two-plane parameterization in detail. In particular we analyze the association between the geometry of the scene and subsets of the 4D data. The answers to these questions are essential to understanding the relationship between a lumigraph, and the geometry that it attempts to represent. This knowledge is potentially important for a variety of applications such as extracting shape from lumigraph data, and lumigraph compression. |
|
|
|
Sloan, P., Cohen, M. F., and Gortler, S. J., "Time Critical Lumigraph Rendering", 1997 Symposium on Interactive 3D Graphics, April 1997. |
|
Abstract It was illustrated in 1996 that the light leaving the convex hull of an object (or entering a convex region of empty space) Can be fully characterized by a 4D function over the space of rays crowing a surface surrounding the object (or surrounding the empty apace) [10, 8]. Methods to represent this function and quickly render individual images from this representation given an arbitrary cameras were also described. This paper extends the work outlined by Gortler et al [8] by demonstrating a taxonomy of methods to accelerate the rendering process by trading off quality for time. Given the specific limitation of a given hardware configuration, we discuss methods to tailor a critical time rendering strategy using these methods. |
|
|
|
Gortler, S. J., Grzeszczuk, R., Szeliski, R. and Cohen M. F., "The Lumigraph", SIGGRAPH 1996, 1996. |
|
Abstract This paper discusses a new method for capturing the complete appearance of both synthetic and real world objects and scenes, representing this information, and then using this representation to render images of the object from new camera positions. Unlike the shape capture process traditionally used in computer vision and the rendering process traditionally used in computer graphics, our approach does not rely on geometric representations. Instead we sample and reconstruct a 4D function, which we call a Lumigraph. The Lumigraph is a subset of the complete plenoptic function that describes the flow of light at all positions in all directions. With the Lumigraph, new images of the object can be generated very quickly, independent of the geometric or illumination complexity of the scene or object. The paper discusses a complete working system including the capture of samples, the construction of the Lumigraph, and the subsequent rendering of images from this new representation. |
|
|
|
Levoy, M. and Hanrahan, P., "Light Field Rendering", SIGGRAPH 1996, 1996. |
|
Abstract A number of techniques have been proposed for flying through scenes by redisplaying previously rendered or digitized views. Techniques have also been proposed for interpolating between views by warping input images, using depth information or correspondences between multiple images. In this paper, we describe a simple and robust method for generating new views from arbitrary camera positions without depth information or feature matching, simply by combining and resampling the available images. The key to this technique lies in interpreting the input images as 2D slices of a 4D function - the light field. This function completely characterizes the flow of light through unobstructed space in a static scene with fixed illumination. We describe a sampled representation for light fields that allows for both efficient creation and display of inward and outward looking views. We have created light fields from large arrays of both rendered and digitized images. The latter are acquired using a video camera mounted on a computer-controlled gantry. Once a light field has been created, new views may be constructed in real time by extracting slices in appropriate directions. Since the success of the method depends on having a high sample rate, we describe a compression system that is able to compress the light fields we have generated by more than a factor of 100:1 with very little loss of fidelity. We also address the issues of antialiasing during creation, and resampling during slice extraction. |
|
|
|
Ramamoorthi, R. and Hanrahan, P., "An Efficient Representation for Irradiance Environment Maps", SIGGRAPH2001, 2001. |
|
Abstract We consider the rendering of diffuse objects under distant illumination, as specified by an environment map. Using an analytic expression for the irradiance in terms of spherical harmonic coefficients of the lighting, we show that one needs to compute and use only 9 coefficients, corresponding to the lowest-frequency modes of the illumination, in order to achieve average errors of only 1%. In other words, the irradiance is insensitive to high frequencies in the lighting, and is well approximated using only 9 parameters. In fact, we show that the irradiance can be procedurally represented simply as a quadratic polynomial in the cartesian components of the surface normal, and give explicit formulae. These observations lead to a simple and efficient procedural rendering algorithm amenable to hardware implementation, a prefiltering method up to three orders of magnitude faster than previous techniques, and new representations for lighting design and image-based rendering. |
|
|
|
Gibson, S. and Murta, A., "Interactive Rendering with Real-World Illumination", 11th Eurographics Workshop on Rendering, June 2000. |
|
Abstract We propose solutions for seamlessly integrating synthetic objects into background photographs at interactive rates. Recently developed image-based methods are used to capture real-world illumination, and sphere-mapping is used illuminate and render the synthetic objects. We present a new procedure for approximating shadows cast by the real-world illumination using standard hardware-based shadow mapping, and a novel image composition algorithm that uses frame-buffer hardware to correctly overlay the synthetic objects and their shadows onto the background image. We show results of an OpenGL implementation of the algorithm that is capable of rendering complex synthetic objects and their shadows at rates of up to 10 frames per second on an SGI Onyx2. |
|
|
|
Kang, S. B., Szeliski, R. and Anandan, P., "The geometry-image representation tradeoff for rendering", International Conference on Image Processing, September 2000. |
|
Abstract It is generally recognized that 3-D models are compact representations for rendering. While pure image-based rendering techniques are capable of producing highly photorealistic outputs, the size of the input "model" is usually very large. The important issues in trading off geometry versus images include compactness of representation, photorealism of reconstructed views, and speed of rendering. In this paper, we describe our past work in modeling and rendering, and articulate lessons learnt. We then delineate our vision of an ideal rendering system. |
|
|
|
Chang, C-F., Bishop, G. and Lastra, A., "LDI Tree: A Hierarchical Representation for Image-Based Rendering", SIGGRAPH 1999, August 1999. |
|
Abstract Using multiple reference images in 3D image warping has been a challenging problem. Recently, the Layered Depth Image (LDI) was proposed by Shade et al. to merge multiple reference images under a single center of projection, while maintaining the simplicity of warping a single reference image. However it does not consider the issue of sampling rate. We present the LDI tree, which combines a hierarchical space partitioning scheme with the concept of the LDI. It preserves the sampling rates of the reference images by adaptively selecting an LDI in the LDI tree for each pixel. While rendering from the LDI tree, we only have to traverse the LDI tree to the levels that are comparable to the sampling rate of the output image. We also present a progressive refinement feature and a "gap filling" algorithm implemented by pre-filtering the LDI tree. We show that the amount of memory required has the same order of growth as the 2D reference images. This also bounds the complexity of rendering time to be less than directly rendering from all reference images. |
|
|
|
Zhang, Z., "Image-Based Geometrically-Correct Photorealistic Scene/Object Modeling (IBPhM): A Review", Proceedings of the Asian Conference on Computer Vision, January 1998. |
|
Abstract There are emerging interests from both computer vision and computer graphics communities in obtaining photorealistic modeling of a scene or an object from real images. This paper presents a tentative review of the computer vision techniques used in such modeling which guarantee the generated views to be geometrically correct. The topics covered include mosaicking for building environment maps, CAD-like modeling for building 3D geometric models together with texture maps extracted from real images, image-based rendering for synthesizing new views from uncalibrated images, and techniques for modeling the appearance variation of a scene or an object under different illumination conditions. Major issues and difficulties are addressed. |
|
|
|
Slusallek, P., Heidrich, W., Vogelgsang, C., Ott, M. and Seidel, H-P., "Radiance Maps: An Image-Based Approach to Global Illumination", research paper, 1998. |
|
Abstract We present a new image-based approach to global illumination that is designed to explicitly make use of coherence in the radiance field of a scene. The approach builds on recently developed ideas from image-based rendering in order to reuse previously computed geometric and illumination information as far as possible. The algorithm consists of a hierarchy of three methods, each taking advantage of a different coherence pattern in the radiance field. At the top level, we sample primary layered-depth radiance maps of the environment, thereby abstracting from the exact geometry of the scene. In the intermediate level, coherence in ray-space is exploited in order to derive new radiance maps by reprojection from primary maps. At the lowest level, gradient information is used to spatially interpolate smooth irradiance contributions, handling non-smooth contributions separately. Missing or uncertain information in lower levels is handled by resampling the environment, and feeding these samples back to adjacent top-level maps. This adaptively improves the layered-depth images and thus the results of later reprojection operations. The essence of our algorithm is an image-based approach to quickly provide accurate radiance maps anywhere in a scene. In this paper, we use the radiance maps to compute irradiance in diffuse environments. However, the comprised information can be used for a much wider range of applications. |
|
|
|
Shade, J., Gortler, S. J., He, L-W. and Szeliski, R., "Layered Depth Images", SIGGRAPH 98, 1998. |
|
Abstract In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan’s warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alpha-compositing can be done efficiently without depth sorting. This makes splatting an efficient solution to the resampling problem. |
|
|
|
Seitz, S. M. and Kutulakos, K. N., "Plenoptic Image Editing", Proc. ICCV 1998, 1998. |
|
Abstract This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene’s generalized plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element in realizing these operations is a new volumetric decomposition technique for reconstructing an scene’s plenoptic function from an incomplete set of camera viewpoints. |
|
|
|
Lischinski, D. and Rappoport, A., "Image-Based Rendering for Non-Diffuse Synthetic Scenes", Eurographics Rendering Workshop, June 1998. |
|
Abstract Most current image-based rendering methods operate under the assumption that all of the visible surfaces in the scene are opaque ideal diffuse (Lambertian) reflectors. This paper is concerned with image-based rendering of non-diffuse synthetic scenes. We introduce a new family of image-based scene representations and describe corresponding image-based rendering algorithms that are capable of handling general synthetic scenes containing not only diffuse reflectors, but also specular and glossy objects. Our image-based representation is based on layered depth images. It represents simultaneously and separately both view-independent scene information and view-dependent appearance information. The view-dependent information may be either extracted directly from our data-structures, or evaluated procedurally using an image-based analogue of ray tracing. We describe image-based rendering algorithms that recombine the two components together in a manner that produces a good approximation to the correct image from any viewing position. In addition to extending image-based rendering to non-diffuse synthetic scenes, our paper has an important methodological contribution: it places image-based rendering, light field rendering, and volume graphics in a common framework of discrete raster-based scene representations. |
|
|
|
Mark, W. R., McMillan, L. and Bishop, G., "Post-Rendering 3D Warping", Proceedings of 1997 Symposium on Interactive 3D Graphics, April 1997. |
|
Abstract A pair of rendered images and their Z-buffers contain almost all of the information necessary to re-render from nearby viewpoints. For the small changes in viewpoint that occur in a fraction of a second, this information is sufficient for high quality re-rendering with cost independent of scene complexity. Re-rendering from previously computed views allows an order-of-magnitude increase in apparent frame rate over that provided by conventional rendering alone. It can also compensate for system latency in local or remote display. We use McMillan and Bishop's image warping algorithm to re-render, allowing us to compensate for viewpoint translation as well as rotation. We avoid occlusion-related artifacts by warping two different reference images and compositing the results. This paper explains the basic design of our system and provides details of our reconstruction and multi-image compositing algorithms. We present our method for selecting reference image locations and the heuristic we use for any portions of the scene which happen to be occluded in both reference images. We also discuss properties of our technique which make it suitable for real-time implementation, and briefly describe our simpler real-time remote display system. |
|
|
|
Seitz, S. M. and Dyer, C. R., "View Morphing", SIGGRAPH1996, 1996. |
|
Abstract Image morphing techniques can generate compelling 2D transitions between images. However, differences in object pose or viewpoint often cause unnatural distortions in image morphs that are difficult to correct manually. Using basic principles of projective geometry, this paper introduces a simple extension to image morphing that correctly handles 3D projective camera and scene transformations. The technique, called view morphing, works by prewarping two images prior to computing a morph and then postwarping the interpolated images. Because no knowledge of 3D shape is required, the technique may be applied to photographs and drawings, as well as rendered scenes. The ability to synthesize changes both in viewpoint and image structure affords a wide variety of interesting 3D effects via simple image transformations. |
|
|
|
McMillan, L. and Bishop, G., "Plenoptic Modeling: An Image-Based Rendering System", SIGGRAPH 1995, 1995. |
|
Abstract Image-based rendering is a powerful new approach for generating real-time photorealistic computer graphics. It can provide convincing animations without an explicit geometric representation. We use the "plenoptic function" of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation. The plenoptic function is a parameterized function for describing everything that is visible from a given point in space. We present an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function. In addition, we introduce a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equivalent to the epipolar constraint defined for planar projections. |
|
|
|
Chen, S. E., "QuickTime® VR – An Image-Based Approach to Virtual Environment Navigation", SIGGRAPH 1995, 1995. |
|
Abstract Traditionally, virtual reality systems use 3D computer graphics to model and render virtual environments in real-time. This approach usually requires laborious modeling and expensive special purpose rendering hardware. The rendering quality and scene complexity are often limited because of the real-time constraint. This paper presents a new approach which uses 360-degree cylindrical panoramic images to compose a virtual environment. The panoramic image is digitally warped on-the-fly to simulate camera panning and zooming. The panoramic images can be created with computer rendering, specialized panoramic cameras or by "stitching" together overlapping photographs taken with a regular camera. Walking in a space is currently accomplished by "hopping" to different panoramic points. The image-based approach has been used in the commercial product QuickTime VR, a virtual reality extension to Apple Computer's QuickTime digital multimedia framework. The paper describes the architecture, the file format, the authoring process and the interactive players of the VR system. In addition to panoramic viewing, the system includes viewing of an object from different directions and hit-testing through orientation-independent hot spots. |
|
|
|
Chen, S. E. and Williams, L., "View Interpolation for Image Synthesis", SIGGRAPH 1993, 1993. |
|
Abstract Image-space simplifications have been used to accelerate the calculation of computer graphic images since the dawn of visual simulation. Texture mapping has been used to provide a means by which images may themselves be used as display primitives. The work reported by this paper endeavors to carry this concept to its logical extreme by using interpolated images to portray three-dimensional scenes. The special-effects technique of morphing, which combines interpolation of texture maps and their shape, is applied to computing arbitrary intermediate frames from an array of prestored images. If the images are a structured set of views of a 3D object or scene, intermediate frames derived by morphing can be used to approximate intermediate 3D transformations of the object or scene. Using the view interpolation approach to synthesize 3D scenes has two main advantages. First, the 3D representation of the scene may be replaced with images. Second, the image synthesis time is independent of the scene complexity. The correspondence between images, required for the morphing method, can be pre-determined automatically using the range data associated with the images. The method is further accelerated by a quadtree decomposition and a view-independent visible priority. Our experiments have shown that the morphing can be performed at interactive rates on today's high-end personal computers. Potential applications of the method include virtual holograms, a walkthrough in a virtual environment, image-based primitives and incremental rendering. The method also can be used to greatly accelerate the computation of motion blur and soft shadows cast by area light sources. |
|
|
|
Drettakis. G. and François X. Sillion, F. X., "Interactive update of global illumination using a line-space hierarchy", SIGGRAPH 1997, 1997. |
|
Abstract Interactively manipulating the geometry of complex, globally illuminated scenes has to date proven an elusive goal. Previous attempts have failed to provide interactive updates of global illumination and have not been able to offer well-adapted algorithms controlling the frame rate. The need for such interactive updates of global illumination is becoming increasingly important as the field of application of radiosity algorithms widens. To address this need, we present a novel algorithm which provides interactive update rates of global illumination for complex scenes with moving objects. In the context of clustering for hierarchical radiosity, we introduce the idea of an implicit line-space hierarchy. This hierarchy is realized by augmenting the links between hierarchical elements (clusters or surfaces) with shafts, representing the set of lines passing through the two linked elements. We show how line-space traversal allows rapid identification of modified links, and simultaneous cleanup of subdivision no longer required after a geometry move. The traversal of line-space also limits the amount of work required to update and solve the new hierarchical system after a move, by identifying the modified paths in the scene hierarchy. The implementation of our new algorithm allows interactive updates of illumination after object motion for scenes containing several thousand polygons, including global illumination effects. Finally, the line-space hierarchy traversal provides a natural control mechanism allowing the regulation of the tradeoff between image quality and frame rate. |
|
|
|
Arnaldi, B., Priol, T., Renambot, L. and Pueyo, X., "Visibility Masks for Solving Complex Radiosity Computations on Multiprocessors", Technical Report No 3008, October 1996. |
|
Abstract In this paper, we present a modified version of the virtual wall concept we introduced in a previously published paper. The goal of our work is to design a strategy to handle very complex scenes (more than 1 million of patches) for radiosity computation. Comparing to other radiosity algorithms, our solution focuses on the ability to compute the radiosity on local environments instead of solving the problem for the whole environment. By splitting the problem into subproblems, using Virtual Interface and Visibility Masks, our technique is able to achieve better data locality than other standard solutions. This property is capital when using either a modern sequential computer to reduce data movement in the memory hierarchy or a multiprocessors to keep as low as possible communication between processors whatever the communication paradigm is: either message passing or shared variable. In the paper, we present an implementation of visibility masks on a distributed memory parallel computer (Intel Paragon XP/S). |
|
|
|
Baranoski, G. V. G., Bramley, R. and Shirley, P., "Iterative Methods For Fast Radiosity Solutions", Technical Report No 429, Indiana University, April 1995. |
|
Abstract In applications involving the radiosity method such as computer animation, time is a crucial factor. This report shows that iterative methods which converge in a smaller number of iterations do not necessarily solve the radiosity system of linear equations in a faster way. For high average reflectance environments is introduced a method that converges faster than any standard method presented in the computer graphics literature so far. |
|
|
|
Smits, B., Arvo, J. and Greenberg, D., "A Clustering Algorithm for Radiosity in Complex Environments", SIGGRAPH 1994, 1994. |
|
Abstract We present an approach for accelerating hierarchical radiosity by clustering objects. Previous approaches constructed effective hierarchies by subdividing surfaces, but could not exploit a hierarchical grouping on existing surfaces. This limitation resulted in an excessive number of initial links in complex environments. Initial linking is potentially the most expensive portion of hierarchical radiosity algorithms, and constrains the complexity of the environments that can be simulated. The clustering algorithm presented here operates by estimating energy transfers between collections of objects while maintaining reliable error bounds on each transfer. Two methods of bounding the transfers are employed with different tradeoffs between accuracy and time. In contrast with the O(s2) time and space complexity of the initial linking in previous hierarchical radiosity algorithms, the new methods have complexities of O(s log s) and O(s) for both time and space. Using these methods we have obtained speedups of two orders of magnitude for environments of moderate complexity while maintaining comparable accuracy. |
|
|
|
Hanrahan, P., Salzman, D. and Aupperle, L., "A Rapid Hierarchical Radiosity Method", SIGGRAPH 1991, 1991. |
|
Abstract This paper presents a rapid hierarchical radiosity algorithm for illuminating scenes containing large polygonal patches. The algorithm constructs a hierarchical representation of the form factor matrix by adaptively subdividing patches into subpatches according to a user-supplied error bound. The algorithm guarantees that all form factors are calculated to the same precision, removing many common image artifacts due to inaccurate form factors. More importantly, the algorithm decomposes the form factor matrix into at most O(n) blocks (where n is the number of elements). Previous radiosity algorithms represented the element-to-element transport interactions with n^2 form factors. Visibility algorithms are given that work well with this approach. Standard techniques for shooting and gathering can be used with the hierarchical representation to solve for equilibrium radiosities, but we also discuss using a brightness-weighted error criteria, in conjunction with multigridding, to even more rapidly progressively refine the image. |
|
|
|
Chen, S. E., "Incremental Radiosity: An Extension of Progressive Radiosity to an Interactive Image Synthesis System", SIGGRAPH 1990, 1990. |
|
Abstract Traditional radiosity methods can compute the illumination for a scene independent of the view position. However, if any part of the scene geometry is changed, the radiosity process will need to be repeated from scratch. Since the radiosity methods are generally expensive computationally, the traditional methods do not lend themselves to interactive uses where the geometry is constantly changing. This paper presents a new radiosity algorithm to incrementally render scenes with changing geometry and surface attributes, In other words, the question to be asked is "What is the minimum recomputation I need to do if l turn off a light source, change the color of a surface, add or move an object?" Because a modeling change generally exhibits some coherence and affects only parts of an image, the proposed method may drastically reduce the rendering time and therefore allow interactive manipulation. In addition, since the method is conducted incrementally and view-independently, the rendering process can start before the modeling process is completed. The traditional paradigm of modeling-then-rendering is changed to rendering-while-modeling. This approach not only gives the user better visual feedback but also effectively utilizes CPU time otherwise wasted in the modeling process. |
|
|
|
Jensen, H. W., "A Practical Guide to Global Illumination using Photon Maps", Siggraph 2000 Course 8, July 2000. |
|
Abstract This course serves as a practical guide to photon maps. Any reader who can implement a ray tracer should be able to add an efficient implementation of photon maps to his or her ray tracer after attending this course and reading the course notes. There are many reasons to augment a ray tracer with photon maps. Photon maps makes it possible to efficiently compute global illumination including caustics, diffuse color bleeding, and participating media. Photon maps can be used in scenes containing many complex objects of general type (e.g. the method is not restricted to tessellated models). The method is capable of handling advanced material descriptions based on a mixture of specular, diffuse, and non-diffuse components. Furthermore, the method is easy to implement and experiment with. This course is structured as a two hour tutorial. We will therefore assume that the participants have knowledge of global illumination algorithms (in particular ray tracing), material models, and radiometric terms such as radiance and flux. We will discuss in detail photon tracing, the photon map data structure, the photon map radiance estimate, and rendering techniques based on photon maps. We will emphasize the techniques for efficient computation throughout the presentation. Finally, we will present some examples of scenes rendered with photon maps and explain the important aspects that we considered when rendering each scene. |
|
|
|
Jensen, H. W. and Christensen, P. H., "Efficient Simulation of Light Transport in Scenes with Participating Media using Photon Maps", SIGGRAPH 1998, 1998. |
|
Abstract This paper presents a new method for computing global illumination in scenes with participating media. The method is based on bi-directional Monte Carlo ray tracing and uses photon maps to increase efficiency and reduce noise. We remove previous restrictions limiting the photon map method to surfaces by introducing a volume photon map containing photons in participating media. We also derive a new radiance estimate for photons in the volume photon map. The method is fast and simple, but also general enough to handle non-homogeneous media and anisotropic scattering. It can efficiently simulate effects such as multiple volume scattering, color bleeding between volumes and surfaces, and volume caustics (light reflected from or transmitted through specular surfaces and then scattered by a medium). The photon map is decoupled from the geometric representation of the scene, making the method capable of simulating global illumination in scenes containing complex objects. These objects do not need to be tessellated; they can be instanced, or even represented by an implicit function. Since the method is based on a bi-directional simulation, it automatically adapts to illumination and view. Furthermore, because the use of photon maps reduces noise and aliasing, the method is suitable for rendering of animations. |
|
|
|
Jensen, H. W., "Global Illumination Using Photon Maps", Rendering Techniques '96, 1996. |
|
Abstract This paper presents a two pass global illumination method based on the concept of photon maps. It represents a significant improvement of a previously described approach both with respect to speed, accuracy and versatility. In the first pass two photon maps are created by emitting packets of energy (photons) from the light sources and storing these as they hit surfaces within the scene. We use one high resolution caustics photon map to render caustics that are visualized directly and one low resolution photon map that is used during the rendering step. The scene is rendered using a distribution ray tracing algorithm optimized by using the information in the photon maps. Shadow photons are used to render shadows more efficiently and the directional information in the photon map is used to generate more optimal sampling directions and to limit the recursion in the distribution ray tracer by providing an estimate of the radiance on all surfaces with the exception of specular and highly glossy surfaces. Noise and blur at discontinuities in the radiance estimate is reduced by using a cone filter. The results presented demonstrate global illumination in scenes containing procedural objects and surfaces with diffuse and glossy reflection models. The implementation is also compared with the Radiance program. |
|
|
|
Heirich, A. and Arvo, J., "Scalable Photorealistic Rendering of Complex Scenes", Proc. of the First Eurographics Workshop on Parallel Graphics and Visualization, August 1996. |
|
Abstract Photorealistic rendering of complex scenes poses computational demands as great as those of any large scale scientific or engineering calculation. Just as scientific calculations have benefited from access to scalable computing systems so too can photorealistic rendering. This paper describes an application of scalable parallel processors to photorealistic rendering of complex scenes by Monte Carlo path tracing. The application uses scalable implementation methods in order to achieve good performance on large numbers of computers and on models which require large amounts of data. The implementation is a message driven concurrent pipeline which employs a diffusion algorithm for dynamic load balancing. The application can be extended to partition extremely large models across physically distributed memory as well as to perform out-of-core calculations. |
|
|
|
Jensen, H. W., "Photon Maps in Bidirectional Monte Carlo Ray Tracing of Complex Objects", Computers & Graphics vol. 19 (2), pp. 215-224, March 1995. |
|
Abstract This paper describes a bidirectional Monte Carlo ray tracing method simulating global illumination in models containing complex objects that do not have to be tessellated. The two pass method combines a first pass light ray tracing (ray casting) with a second pass optimized Monte Carlo ray tracing. In the first pass, the light emitted from the light sources hit objects in the scene and may be reflected or transmitted - a kind of backward path tracing. This step handles all kinds of reflections and not only the specular to diffuse reflections. This turns out to be a valuable optimization. At every object-interaction, energy is stored on the surface of the object. For simple objects an illumination map is used. For complex objects e.g. procedurally based objects like fractals, energy is stored in a photon map. This new concept makes it possible to treat caustics upon such objects without having to parameterize the surface of the objects. The second pass, Monte Carlo ray tracing from the eye, visualizes the scene based upon the result from the first pass. We use the irradiance gradient method to model diffuse reflections seen directly from the eye. All secondary reflections are taken from the photon maps or the illumination maps. Only the caustic part of the ray casting step is visualized directly. |
|
|
|
Cook, R. L., "Stochastic Sampling in Computer Graphics", ACM Transactions on Graphics, vol. 5, no. 1, pp. 51-72, January 1986. |
|
Abstract Ray tracing, ray casting, and other forms of point sampling are important techniques in computer graphics, but their usefulness has been undermined by aliasing artifacts. In this paper it is shown that these artifacts are not an inherent part of point sampling, but a consequence of using regularly spaced samples. If the samples occur at appropriate nonuniformly spaced locations, frequencies above the Nyquist limit do not alias, but instead appear as noise of the correct average intensity. This noise is much less objectionable to our visual system than aliasing. In ray tracing, the rays can be stochastically distributed to perform a Monte Carlo evaluation of integrals in the rendering equation. This is called distributed ray tracing and can be used to simulate motion blur, depth of field, penumbrae, gloss, and translucency. |
|
|
|
Cook, R. L., Porter, T. and Carpenter, L., "Distributed Ray Tracing", SIGGRAPH 1984, July 1984. |
|
Abstract Ray tracing is one of the most elegant techniques in computer graphics. Many phenomena that are difficult or impossible with other techniques are simple with ray tracing, including shadows, reflections, and refracted light. Ray directions, however, have been determined precisely, and this has limited the capabilities of ray tracing. By distributing the directions of the rays according to the analytic function they sample, ray tracing can incorporate fuzzy phenomena. This provides correct and easy solutions to some previously unsolved or partially solved problems, including motion blur, depth of field, penumbras, translucency, and fuzzy reflections. Motion blur and depth of field calculations can be integrated with the visible surface calculations, avoiding the problems found in previous methods. |
|
|
|
Granier, X., Drettakis, G. and Walter, B., "Fast Global Illumination Including Specular Effects", Proc. of the Eleventh Eurographics Workshop on Rendering, 2000. |
|
Abstract Rapidly simulating global illumination, including diffuse and glossy light transport is a very difficult problem. Finite element or radiosity approaches can achieve interactive simulations for some classes of diffuse-only scenes, but more general methods are currently too slow and too noisy for interactive use. We present a new method which seamlessly integrates particle tracing (for non-diffuse transport) into the gather step of hierarchical radiosity (for diffuse transport) to efficiently handle all types of light transport chains. Our integrated approach results in rapid, good visual quality solutions. This is achieved using a radiosity algorithm producing smooth, noise free simulation of diffuse light transfers, and an integrated particle trace for rapid, high quality specular reflections such as caustics. Using our system, users can interactively visualize and manipulate small environments with global illumination including specular effects. Such general lighting effects can also be simulated for larger environments, albeit at a higher computational cost. Our system can also treat scenes which are lit mainly by indirect lighting, which is very hard using previous methods. With our method, smooth transition from fast, low quality to slower high quality solutions is possible. |
|
|
|
Walter, B., Hubbard, P. M., Shirley, P. and Greenberg, D. P., "Global Illumination Using Local Linear Density Estimation", ACM Transactions on Graphics, 16(3), 1997. |
|
Abstract This article presents the density estimation framework for generating view-independent global illumination solutions. It works by probabilistically simulating the light flow in an environment with light particles that trace random walks originating at luminaires and then using statistical density estimation techniques to reconstruct the lighting on each surface. By splitting the computation into separate transport and reconstruction stages, we gain many advantages including reduced memory usage, the ability to simulate nondiffuse transport, and natural parallelism. Solutions to several theoretical and practical difficulties in implementing this framework are also described. Light sources that vary spectrally and directionally are integrated into a spectral particle tracer using nonuniform rejection. A new local linear density estimation technique eliminates boundary bias and extends to arbitrary polygons. A mesh decimation algorithm with perceptual calibration is introduced to simplify the Gouraud-shaded representation of the solution for interactive display. |
|
|
|
Bekaert, P. and Willems, Y. D., "A Progressive Importance-Driven Rendering Algorithm", 10th Spring School on Computer Graphics and its Applications, June 1994. |
|
Abstract |
|
|
|
Aupperle, L. and Hanrahan, P., "A Hierarchical Illumination Algorithm for Surfaces with Glossy Reflection", SIGGRAPH 1993, 1993. |
|
Abstract |
|
|
|
Aupperle, L. and Hanrahan, P., "Importance and Discrete Three Point Transport", Eurographics Rendering Workshop, 1993. |
|
Abstract Global illumination within nondiffuse environments is ideally suited for computation under importance and radiance driven hierarchical refinement: a transport interaction is significant to the view from the eye only if it lies within paths of directional reflection of both radiance originating at a light source, and importance originating at the eye. We have recently developed a hierarchical algorithm for global illumination based on a radiance formulation for three point transport, and have implemented this algorithm over environments containing surfaces exhibiting glossy reflection. In this paper we derive the adjoint to our original transport formulation, and describe the preliminary results of application of this adjoint in the form of an importance driven version of our glossy implementation. These results show a significant reduction in transport computation, and indicate that an importance and radiance driven hierarchical technique possesses great promise for the development of efficient global illumination algorithms for general reflection. |
|
|
|
Magnor, M., Endmann, A. and Girod, B., "Progressive Compression and Rendering of Light Fields", Proc. Vision, Modeling, and Visualization (VMV-2000), pp. 199-203, November 2000. |
|
Abstract The paper presents a progressive light-field coding scheme based on four-dimensional wavelet decomposition. At 100:1 compression, reconstructed light-field images are visually indistinguishable from the original images, enabling even large light-field data sets to fit into local memory. Progressive decoding allows continuous adjustment of rendering quality to available computational resources. The proposed scheme is validated using two well-known light fields. Rendered image quality remains acceptable even if less than one thousandth of the original light-field information is decoded. |
|
|
|
Magnor, M., Eisert, P. and Girod, B., "MODEL-AIDED CODING OF MULTI-VIEWPOINT IMAGE DATA", Proc. of the IEEE International Conference on Image Processing (ICIP2000), pp. 919-922, September 2000. |
|
Abstract The paper presents a new coding technique for images taken from arbitrary recording positions around a static scene, based on reconstructed object geometry. Such data structures occur in image-based rendering applications where many hundreds to thousands of images need to be stored and transmitted. Approximate scene geometry enables disparity compensation as well as occlusion detection, leading to improved image prediction. Images are coded in hierarchical order to ensure efficient exploitation of inter-image similarities. The 3-D geometry model allows rendering new views by warping recorded images. The presented algorithm is validated using real-world image data sets, achieving better than 1000:1 compression at acceptable reconstruction quality. |
|
|
|
Magnor, M. and Girod, B., "Data Compression for Light Field Rendering", IEEE Trans. Circuits and Systems for Video Technology, vol. 10, no. 3, pp. 338-343, April 2000. |
|
Abstract Two light-field compression schemes are presented. The codecs are compared with regard to compression efficiency and rendering performance. The first proposed coder is based on video-compression techniques that have been modified to code the four-dimensional light-field data structure efficiently. The second coder relies entirely on disparity-compensated image prediction, establishing a hierarchical structure among the light-field images. Coding performance of both schemes is evaluated using publicly available light-fields of synthetic as well as real-world scenes. Compression ratios vary between 100:1 and 2000:1, depending on reconstruction quality and light-field scene characteristics. |
|
|
|
Magnor, M. and Girod, B., "Hierarchical Coding of Light Fields with Disparity Maps", Proc. of the IEEE: International Conference on Image Processing (ICIP-99), pp. 334-338, October 1999. |
|
Abstract A coder for light fields is presented. Due to the large amount of data needed to represent a complete light field, a hierarchical decomposition is employed. The full light field is built up by recursively predicting intermediate images from a small subset of light-field images. Intermediate images are predicted by disparity-compensating multiple surrounding images. The predicted images are refined using DCT coding. Rate-distortion measurements for two standard light fields verify the efficiency of the proposed scheme. Compression ratios of 1000:1 are achieved at acceptable quality of the rendered views. |
|
|
|
Magnor, M. and Girod, B., "Adaptive Block-based Light Field Coding", Proc. 3rd International Workshop on Synthetic and Natural Hybrid Coding and 3-D Imaging (IWSNHC3DI'99), pp. 140-143, September 1999. |
|
Abstract A compression scheme is presented that has been designed to efficiently code light fields. Based on video compression techniques, the light field image array is divided into image blocks. Different block-coding modes have been devised to exploit redundancy between images over a wide range of target bit rates. Prediction from multiple reference images further enhance coding efficiency. Rate-constrained mode selection is accomplished by employing Lagrangian optimization. The coder's operational rate-distortion performance is evaluated using different light fields. Depending on light-field scene characteristics and reconstruction quality, compression ratios up to 1000:1 are achieved. |
|
|
|
Cosman, P. C., Oehler, K. L., Riskin, E. A. and Gray, R. M., "Using Vector Quantization for Image Processing", Proc. of the IEEE, 81(9):1326-1341, September 1993. |
|
Abstract Image compression is the process of reducing the number of bits required to represent an image. Vector quantization, the mapping of pixel intensity vectors into binary vectors indexing a limited number of possible reproductions, is a popular image compression algorithm. Compression has traditionally been done with little regard for image processing operations that may precede or follow the compression step. Recent work has used vector quantization both to simplify image processing tasks -- such as enhancement, classification, halftoning, and edge detection -- and to reduce the computational complexity by performing them simultaneously with the compression. After briefly reviewing the fundamental ideas of vector quantization, we present a survey of vector quantization algorithms that perform image processing. |
|
|
Error estimation and miscellaneous
|
Kutulakos, K. N., "Shape from the Light Field Boundary", Proc. CVPR, 1997. |
|
Abstract Ray-based representations of shape have received little attention in computer vision. In this paper we show that the problem of recovering shape from silhouettes becomes considerably simplified if it is formulated as a reconstruction problem in the space of oriented rays that intersect the object. The method can be used with both calibrated and uncalibrated cameras, does not rely on point correspondences to compute shape, and does not impose restrictions on object topology or smoothness. |
|
|
|
Hubbard, P. M., "Approximating Polyhedra with Spheres for Time-Critical Collision Detection", ACM Transactions on Graphics, 15(3), July 1996. |
|
Abstract This paper presents a method for approximating polyhedral objects to support a time-critical collision-detection algorithm. The approximations are hierarchies of spheres, and they allow the time-critical algorithm to progressively refine the accuracy of its detection, stopping as needed to maintain the real-time performance essential for interactive applications. The key to this approach is a preprocess that automatically builds tightly fitting hierarchies for rigid and articulated objects. The preprocess uses medial-axis surfaces, which are skeletal representations of objects. These skeletons guide an optimization technique that gives the hierarchies accuracy properties appropriate for collision detection. In a sample application, hierarchies built this way allow the time-critical collision-detection algorithm to have acceptable accuracy, improving significantly on that possible with hierarchies built by previous techniques. The performance of the time-critical algorithm in this application is consistently 10 to 100 times better than a previous collision-detection algorithm, maintaining low latency and a nearly-constant frame rate of 10 frames per second on a conventional graphics workstation. The time-critical algorithm maintains its real-time performance as objects become more complicated, even as they exceed previously reported complexity levels by a factor of more than 10. |
|
|
|
Ashdown, I., "Near-Field Photometry: Measuring and Modeling Complex 3-D Light Sources", SIGGRAPH 1995, August 1995. |
|
Abstract This paper presents a method that, based on a field-theoretic approach to photometry, accurately and efficiently measures and models arbitrarily complex three-dimensional light sources. It allows the prediction of direct illuminance at any point on any surface anywhere within the surrounding 3-D space without requiring any knowledge of the geometry of the source or its distance from the surface being illuminated. The method is easily integrated with ray tracing techniques and radiosity methods, including multiprocessor implementations. |
|
|
|
Dorsey, J. O., Arvo, J. and Greenberg, D. P., "Interactive Design of Complex Time-Dependent Lighting", IEEE Computer Graphics and Applications, March 1995. |
|
Abstract We describe new tools for the interactive design of complex, time-dependent lighting in scenes with fixed geometry. The work is motivated by the difficulty of visualizing complicated lighting sequences during the design of large-scale theatrical productions. Fast interaction is achieved regardless of scene and lighting complexity, even when used in conjunction with costly rendering techniques such as radiosity and ray tracing. Time-variant lighting is simulated using linear combinations of static images, each depicting the scene under different lighting conditions. Multiple levels of accuracy facilitate interactive design by trading image quality for fast feedback in a controlled way. Coarse approximations are used in the preliminary design stage to allow for instantaneous feedback, then the sequence is progressively refined by computing basis solutions in order of increasing overall contribution. When changes to the design are required, existing global solutions are re-used to the greatest extent possible. The techniques are demonstrated with complex models based on actual stage sets. |
|
|
|
Arvo, J., Torrance, K. and Smits, B., "A Framework for the Analysis of Error in Global Illumination Algorithms", SIGGRAPH 1994, July 1994. |
|
Abstract In this paper we identify sources of error in global illumination algorithms and derive bounds for each distinct category. Errors arise from three sources: inaccuracies in the boundary data, discretization, and computation. Boundary data consist of surface geometry, reflectance functions, and emission functions, all of which may be perturbed by errors in measurement or simulation, or by simplifications made for computational efficiency. Discretization error is introduced by replacing the continuous radiative transfer equation with a finite-dimensional linear system, usually by means of boundary elements and a corresponding projection method. Finally, computational errors perturb the finite-dimensional linear system through imprecise form factors, inner products, visibility, etc., as well as by halting iterative solvers after a finite number of steps. Using the error taxonomy introduced in the paper we examine existing global illumination algorithms and suggest new avenues of research. |
|
|
|
Simoncelli, E. P., Freeman, W. T., Adelson, E. H. and Heeger, D. J., "Shiftable Multi-scale Transforms", IEEE trans. Information Theory vol. 38(2), March 1992. |
|
Abstract Orthogonal wavelet transforms have recently become a popular representation for multi-scale signal and image analysis. One of the major drawbacks of these representations is their lack of translation invariance: the content of wavelet subbands is unstable under translations of the input signal. Wavelet transforms are also unstable with respect to dilations of the input signal, and in two dimensions, rotations of the input signal. We formalize these problems by defining a type of translation invariance that we call "shiftability". In the spatial domain, shiftability corresponds to a lack of aliasing; thus, the conditions under which the property holds are specified by the sampling theorem. Shiftability may also be considered in the context of other domains, particularly orientation and scale. We explore "jointly shiftable" transforms that are simultaneously shiftable in more than one domain. Two examples of jointly shiftable transforms are designed and implemented: a one-dimensional transform that is jointly shiftable in position and scale, and a two-dimensional transform that is jointly shiftable in position and orientation. We demonstrate the usefulness of these image representations for scale-space analysis, stereo disparity measurement, and image enhancement. |
|
|
|
Adelson, E. H. and Bergen, J. R., "The Plenoptic Function and the Elements of Early Vision", Computational Models of Visual Processing, MIT Press 1991. |
|
Abstract What are the elements of early vision? This question might be taken to mean, What are the fundamental atoms of vision?—and might be variously answered in terms of such candidate structures as edges, peaks, corners, and so on. In this chapter we adopt a rather different point of view and ask the question, What are the fundamental substances of vision? This distinction is important because we wish to focus on the first steps in extraction of visual information. At this level it is premature to talk about discrete objects, even such simple ones as edges and corners. There is general agreement that early vision involves measurements of a number of basic image properties including orientation, color, motion, and so on. Figure l.l shows a caricature (in the style of Neisser, 1976), of the sort of architecture that has become quite popular as a model for both human and machine vision. The first stage of processing involves a set of parallel pathways, each devoted to one particular-visual property. We propose that the measurements of these basic properties be considered as the elements of early vision. We think of early vision as measuring the amounts of various kinds of visual "substances" present in the image (e.g., redness or rightward motion energy). In other words, we are interested in how early vision measures "stuff" rather than in how it labels "things." |
|
|
