From bc64db2c9fa46dcf3f83192e508eebdf8146ece7 Mon Sep 17 00:00:00 2001 From: TimothyLottes Date: Mon, 21 Nov 2016 22:07:16 -0500 Subject: [PATCH] Add files via upload --- 20161117.html | 64 +++++++++++ 20161121.html | 302 ++++++++++++++++++++++++++++++++++++++++++++++++++ index.html | 13 ++- 3 files changed, 375 insertions(+), 4 deletions(-) create mode 100644 20161117.html create mode 100644 20161121.html diff --git a/20161117.html b/20161117.html new file mode 100644 index 0000000..9401236 --- /dev/null +++ b/20161117.html @@ -0,0 +1,64 @@ +
+

20161117 - Mechanics of Vulkan SPIR-V Specialization

+
+ +References +
+SPIR-V | +Vulkan +
+
+ +SPIR-V Background +
+ +
+Human readable SPIR-V example. +
+
+
+%1 = OpTypeInt 32 1       ; declare a signed 32-bit type
+OpDecorate %2 SpecId 3    ; set 'constantID = 3' 
+%2 = OpSpecConstant %1 -8 ; some signed 32-bit integer constant
+
+
+Vulkan Background +
+Showing below the related structures which show how to set specialization constants. +
+
+
+typedef struct VkSpecializationMapEntry {
+  uint32_t constantID; // SpecId in SPIR-V source.
+  uint32_t offset;     // Byte offset in VkSpecializationInfo.pData for value.
+  size_t   size;
+} VkSpecializationMapEntry;
+
+// Passed into  VkPipelineShaderStageCreateInfo
+typedef struct VkSpecializationInfo {
+  uint32_t                        mapEntryCount;
+  const VkSpecializationMapEntry* pMapEntries;
+  size_t                          dataSize;
+  const void*                     pData;
+} VkSpecializationInfo;
+
+
+ + + + + + +
+ + diff --git a/20161121.html b/20161121.html new file mode 100644 index 0000000..34f5697 --- /dev/null +++ b/20161121.html @@ -0,0 +1,302 @@ +
+

20161121 - Moving Beyond Rendering With Triangles

+
+
This post describes practical non-triangle rendering working backwards from the back-buffer.
+
+

Deferring Filtering to Image Reconstruction

+Samples are shaded on fixed positions on objects instead of being interpolated to variable positions on a screen grid. +Filtering and interpolation are deferred until the end of the pipeline. +This means that shading rate is decoupled from resolution and frame rate, +filtering quality is decoupled from shading rate and coverage or Z test rate, +and shading is temporally stable. + +
+
+
+Reconstruction Happens in Post-Warped Space +
+This is a non-optional critical component for image quality and performance! +This applies if a classic non-VR game is simulating a physical camera lens +or alternatively if a VR game is outputing to an HMD with a warped view. +The reconstruction filtering must be applied in post-warped space to look right. +
+
+When a rectangular projection is bilinear fetched to implement a lens warp, +the output pixels on average are somewhere inbetween 4 texels, +and thus the output is that of a poor variable strength low-pass filter. +With MSAA, those 4 texels are typically the result of a second low-pass filter. +Part of the aim to have the app render 2x the pixels on average is to attempt to have enough aliasing +to counteract the low-pass filter(s). +This can never produce a high quality image. + +
+
+Below is an image I recently re-found from years back using this reconstruction technique in colored monochrome +with a warped projection and all sub-pixel sized geometry with no temporal filtering (lower quality spatial reconstruction only). +It looks much better in video. +
+
+
+
+ +
+Reconstruction Source Image +
+Presenting a good starting place to begin to construct a 64-bit version of what I've done in the past. +Yes this is another good time to point out that some PC platforms still don't expose 64-bit global atomics. +Working on fixing that ... +
+
+The source for the reconstruction pass is in the post-warped space (ie exactly what gets scanned out) +and consists of a single 64-bit value per texel. +The resolution of this source image can be different than the output back-buffer size. +For instance it is possible to have 2x the area to have higher quality, +or 0.5x of the area to have up-sampling. +
+
+The 64-bit texel is a packed data structure. +Possible example of packing below. +Ordering requires that depth be at MSB bits, +but in theory other values could be moved around. +Bits per component and pre-packing transforms can be adjusted for various quality trade-offs. +This pass is expected to be ALU bound, so optimizing pack and un-pack could be important (possibly using LDS, or accept 8-bit for all non-depth components dithering really well before quantization, using SDWA to unpack, etc). +
+
+
+OFFSET  BITS  MEANING
+======  ====  =======
+48      16    Logarithmic depth, reversed (65535=near, 0=far), at MSB
+44      4     Weight (after depth so highest weight wins same depth collision)
+32      12    Green
+20      12    Red
+8       12    Blue
+4       4     Sub-pixel X offset
+0       4     Sub-pixel Y offset
+
+
+In a pass before reconstruction, shaded samples are atomically scattered into this image using atomicMax64(). +So the nearest depth will win a collision (then highest weight in my example, and so on). +The following is critical for image quality. +Collisions are expected, and transformed into film grain. +Right before scatter, each sample is given a spatial temporal stochastic blue-noise offset +/- one pixel in range. +The sub-pixel offset in the packed texel +provides the correct non-offseted original position to sub-pixel precision. +Sub-pixel X offset can be visualized as the following. +
+
+
+0 1 2 3 4 5 6 7 8 9 a b c d e f
+= = = = = = = = = = = = = = = =
+       +---------------+
+       |               |
+       .     pixel     .
+
+
+Weight is the sub-pixel area of the shaded sample, probably best to decode by converting to {0.0 to 1.0} and squaring. +LOD fade-in and fade-out can be done stochastically and can be improved by reducing sub-pixel area to make more transparent. +
+
+For depth encoding, Maximizing Depth Buffer Range and Precision on the Outerra Blog +is a great reference. +Use log(depth + 1.0) or perhaps the modified near field linearization talked about on Outerra. +This maximizes the depth precision in the scene. +
+
+The color should be decoded from a non-linear to linear space before filtering. +Perhaps something as easy as converting to {0.0 to 1.0} and squaring or cubing. +
+
+
+Reconstruction Filtering +
+Only have time to cover the basic spatial component of the reconstruction filter. +Incorporating a temporal component can be done for greatly increased quality. +
+
+Spatial filter is easy to do, +given an output pixel work with a fixed circular pattern of the nearest N texels in the 64-bit/texel image source. +Example of a N=12 pattern below +which would work well when source image size is not the same as output image size. +
+
+
+  A B  
+C D E F  
+G H I J  
+  K L  
+
+
+For each N texels, take the weighted average, sum the following for each sample, +color * (weight * f(subPixelPrecisionDistanceFromSampleToResolvedPixelCenter)), divide by the sum of weights, (weight * f(...)). +The f() is the filter kernel which could be a Gaussian (which is exp2(-scale*dst*dst)), +or for something sharper a circular filter with a soft fall-off (similar to DOF filter), or something better. +
+
+
+Grain +
+After reconstruction filtering, add a linear blue-noise film grain for the final analog distraction-free output. +
+
+
+Anti-Aliased Composite of Transparency +
+The best place to composite/apply transparency is before packing and scattering the 64-bit packed samples. +This way the sample can work with it's discrete depth value, interpolating from the likely lower resolution transparency layer. +Ultimately transparency then maintains the same high quality filtering as everything else. +
+
+Showing two stills below from a similar technique in which a spatial temporal filter is used to filter 1-shaded sample/pixel lit fog, +which is then applied to opaque pixels which are traced in a stocastic pattern, +then finally being reconstructed with a spatial temporal filter similar to the one described in this post +(using N nearest samples, taking a weighted average). +
+
+
+
+
+ +
+
+
+Image Pyramid Reconstruction +
+What about hole filling and variable sized points? +
+Will get to data expansion in the later section, this is another option/tool which can be used for creative solutions. +The idea is simple, take the projected size of the sample before 64-bit atomic scattering, +and use it to choose a level in an image pyramid +(the pyramid is unpacked into a single texture layer). +Before reconstruction, run from smallest layer up, +splitting the larger samples into interpolated smaller samples on each pass and merging with the next finer level. +When interpolating samples that are near to each other, remember to also do a weight based average +and generate a new sub-pixel offset. +Ideally weight should be adjusted so that finer samples override similar depth coarser samples. +
+
+Another trick which I've employed in the past +is to use the coarse level data to compute depth to get a more accurate estimate of reprojection to use to help fill in the holes. +
+
+There are a collection of methods to manage reconstruction depending on engine details and desired quality/perf trade-offs, +and typical reconstruction costs are similar to temporal filtering with classic triangle rendering. +
+
+
+ +

Managing a Point Based Rendering Pipeline

+Lets continue with the elephant in the room, is it practical on current hardware? +From what I've seen the answer is a definite yes, +and the gains in distraction free rendering are substantial. +The engine I showed at my prior GTC talk, shot below, was driving 1080p at over 120 Hz. +That engine had per pixel costs over an order of magnitude from a simple point renderer, it was sphere traced. +It is effectively an engine optimized to solve the problem of skinning each pixel hundreds of times +(the sphere tracing distance estimator is a set of nested transforms). +
+
+
+ +
+
+
+Rendering Traditional Game Assets +
+Keep the standard sheet of mipmaped textures for surface properties. +Break the texture layers into SIMD friendly 8x8 tiles of texels. +Each tile contains some amount of per-tile meta-data (which is shared across the wave and can be fetched with SMEM) such as the following. +
+
+ +
+Add an extra compressed texture sheet for object-space position in the tile local coordinate space. +Now geometry is fully represented in textures, +and only compute is needed to render. +
+
+Rendering becomes a process of running a compute shader to fetch tile data, +transform, optionaly skin, and shade the collection of texels in the tile. +Pack the 64-bit sample values, +and output using atomicMax64() into the post-warped space for reconstruction later. +Note sample data is loaded once, processed, and written only as a minimal 64-bit value. +Quite minimal in comparison with traditional fixed function which expands VS output to maximum size and routes around chip. +Also the tile has good final projected spatial locality, there is enough work to hide the atomic scatter cost, and atomic operations are fire-and-forget. +
+
+There are many other interesting variants, such as caching shading results in a tile cache. +Could keep updating the most important shading until some safe point before v-sync, +then switch to late-frame point atomic scatter for the 64-bit packed sample values. +
+
+
+Managing Data Expansion +
+The other elephant in the room, how to manage the need for hierarchical data expansion? +As in, this tile might project to a box of 8 texels, or fill a large chunk of the screen. +Or this tile might have high amounts of perspective. Etc ... +
+
+My advice is to start with a temporally coherent listing of tiles, +similar to, or effectively exactly how one manages a virtual texture. +Except where typical virtual texturing would work in 128x128 granularity, +this is driven in 8x8 granularity. +Instanced objects would be given some local space in the virtual texture similar to how lightmaps have local space for instanced geometry. +This has the great side-effect of providing a place to cache texture space shading results. +
+
+It has been possible for a while now to fully manage a virtual texture on the GPU. +Part of the tile shading process will be to run HZB occlusion on the title, +check if it is visible, or percentage of visibility estimate which can be multiplied by projected size. +This drives a LOD expand/prune priority for the tile in the virtual texture. +Adaptively manage a per-frame low watermark of visibility which is a cut-off where tiles self-prune +to get recycled to attach to a new part of the virtual texture which needs to get expanded. +
+
+Costs are thus bounded by the size of the physical tile cache. +
+
+Virtual texture cache has limits to how much expansion can be done in a given frame, +typically one level/frame of expansion. +Fast motion can demand a lot more than that when projected texels are effectively kept close to pixel sized. +Some amount of expansion can be soked up with the "Image Pyramid Reconstruction" concept mentioned above. +The second tool is to allow the tile itself loop and walk it's own data at different scales based on the needs of visibility or projected texel density. +
+
+
+Tile Expansion - Variable Resolution Shading +
+Tile starts at full size. Tile keeps track of a scale and {x,y} offset inside the the tile (these are low-cost dynamically uniform SGPRs). +Transforms (including skinning) to view-space, projects to post-warped space, and drives a waveMin and waveMax to gather projected bounds. +This projected bounds can be used to select which HZB layer to sample from for occlusion, and which level of the image pyramid to atomic scatter into. +If the output sample density isn't fine enough, +the tile can cut scale in half, and then quad sub-divide using {x,y} to track state as it walks across the sub-tile. +Any sub-tile which doesn't hit texel density targets can be further sub-divided at run-time in the same shader +(by dropping scale, walking the sub-tile quad, and then moving to the prior scale backing out). +
+
+Note this is compute. It is ok for tiles to have variable run-time, as long as they are bounded. Leverage the "Image Pyramid Reconstruction" for worst case expansion (bounded cost), +and limit the amount of hierarchical sub-division. Fast motion and scaling will often mask the ability for anyone to focus on details in a single frame anyway. +
+
+There are many variations which could be quite interesting here. +For example, the shader can still do classic procedural texturing for enhanced detail, +except in this case the shader defined procedural detail can be fully 3D displacement mapped. +
+
+
+

Until Next Time

+The intent of this post was to begin to paint a picture of the high-level of the major components which enable a non-triangle rendering path for traditional game content, +running backwards compatible hardware, +to get to pixel level detail with perfect anti-aliasing, +with flexibibility to hit any resolution, shading rate, and frame rate trade-off. +There are many other aspects of the solution space to dive into in the future. +
+
+ +
+ + diff --git a/index.html b/index.html index 91278a7..a41428c 100644 --- a/index.html +++ b/index.html @@ -25,7 +25,7 @@ O OOO ||..||OO||||||||||OOo || ||||||||||||| ||||OO|| |o| ||||||||OO||OOO .. .. . .// . ... / 

 

-
20161117 - Index - RSS

+
20161121 - Index - RSS

 

 

 

@@ -36,10 +36,10 @@ O  OOO  ||..||OO||||||||||OOo  ||  |||||||||||||  ||||OO||  |o|  ||||||||OO||OOO
 

 Recently Changed
 

+20161121 - Moving Beyond Rendering With Triangles

+20161117 - Mechanics of Vulkan SPIR-V Specialization

 20161114 - Believing in an Image

 20161113 - Vintage Programming 2

-20161030 - Demo Tube Mega List

-20161112 - Plasma Displays

 

 Updates
 

@@ -54,11 +54,12 @@ and attempting to restore what I can from prior lost images.
 

 
 

 2016

+20161121 - Moving Beyond Rendering With Triangles

+20161117 - Mechanics of Vulkan SPIR-V Specialization

 20161114 - Believing in an Image

 20161113 - Vintage Programming 2

 20161112 - Plasma Displays

@@ -330,6 +331,10 @@ and attempting to restore what I can from prior lost images.
 20080628 - Micro Polygons

 

 2007

+20071126 - Optimization and More

+20071121 - Deferred Shading III

+20071116 - Deferred Shading II

+

 20071026 - Transform Feedback

 20071025 - Motion Cards

 20071024 - Geometry Shader Woes