HOME
CV
Pascal Guehl
PPTBF
Model Parameters Estimation Appearance Space Transitions
APPLICATIONS
Color Transfer Texture Extrapolation Texturing & Modeling Environment Interaction
COMPARISONS
Noise by-example Texture Synthesis
DATA
BDD Exemplars
PERFORMANCES
Timings

Semi-Procedural Textures using Point Process Texture Basis Functions

EGSR 2020 (Eurographics Symposium on Rendering)

CFG (Computer Graphics Forum)

Submission ID: 1018

Implementation & Timings

This is a supplementary material associated to the article Procedural Texture Extrapolation using Point Process Texture Basis Functions. This is intended for developpers who would like to integrate it in industry for production, researchers that would like to use it as starting SDK, etc...

We rely on 3 types of machines:

Graphics Card GPU architecture Nb Cores Memory
NVIDIA GeForce 1060 GTXPascal12806 Go
NVIDIA GeForce 1080 GTXPascal25608 Go
NVIDIA Quadro P5000Pascal256016 Go

PPTBF Model

We rely on the following cross-plateform technologies:

Technology Description
C++programming langage
CMakecross-plateform project building
OpenGL3D graphics library
gladOpenGL loader
glm3D maths library
stbimage load/save
GLFWwindow/graphics context management
ImGuigraphical user interface

Mixing Graphics and Compute

We use OpenGL compute shaders for all algorithms and traditional graphics shaders for rendering.

PPTBF Synthesis Timing for a Mega-Kernel Approach

We use one unique megakernel based on OpenGL compute shader. The PPTBF is stored in a 1D texture of 1-channel floating point (R32F). This approach requires no additional GPU memory allocation. But requires a lot of registers, giving rise to register pressure, latency... Complex code => branching...

For OpenGL compute kernels, we use blocks of 8x8 threads. Memory is limited by graphics card. For each output image size, we test without and with repulsive forces (5 iterations in Lloyd algorithm). These are the results for Voronoi window and 1 to 5 Gabor kernels in Feature function.

We show speed according to image size. Target device: NVIDIA GeForce 1060 GTX 6Go.

Output Size (pixels) Time (ms) Add repulsion forces (Lloyd)
256 x 25612.587...
512 x 51246.802...
1024 x 1024177.103...
2048 x 2048754.369...

PPTBF Synthesis Timing for a Wavefront Approach

Using a wavefront approach split algorithm into multi-passes. [Laine et al. 2013] It follows node-based tools way of thinking. User can then choose and customize components. The deformation component D could be fBm (fractional brownian motion); turbulence, etc... The point process component PP could be normal or add forces such as repulsions. The wavefront approach requires to store results at each step. We rely on imageStore to write into textures, and bindless texture extensions to be able to access a lot of textures/images in read or write mode. Bindless textures requires an GPU device at least Kepler (but some old Kepler are not working).

For OpenGL compute kernels, we use blocks of 8x8 threads. Memory is limited by graphics card. For each output image size, we test without and with repulsive forces (5 iterations in Lloyd algorithm). These are the results for Voronoi window and 1 to 5 Gabor kernels in Feature function.

We show speed according to image size. Target device: NVIDIA GeForce 1060 GTX 6Go.

Output Size (pixels) Total Time (ms) Model & Deformation (ms) Point Process (ms) Window Function (ms) Feature Function (ms) Compositing (ms)
256 x 25611.4860.1227.6633.6800.021...
512 x 51246.4830.45831.46114.5000.065...
1024 x 1024254.953.1031.798156.46096.4650.230...
2048 x 20481067.5727.162684.703410.8410.866...

PPTBF Rendering

Rendering relies on a classical fullscreen triangle with a fragment shader reading the PPTBF texture.

For OpenGL compute kernels, we use blocks of 8x8 threads. Memory is limited by graphics card. For each output image size, we test without and with repulsive forces (5 iterations in Lloyd algorithm). These are the results for Voronoi window and 1 to 5 Gabor kernels in Feature function.

We show speed according to image size. Target device: NVIDIA GeForce 1060 GTX 6Go.

Database Generation

I. PPTBF Database

We empirically sampled our PPTBF parameter space based on our real-time PPTBF visualizer/designer. One cannot uniformly sample due to too much parameter. We try to keep parameters impacting appearance, and separate data according to 98 classes based on point process tiling types, window function types and feature function types. We finally obtain >= 45 millions of PPTBF images (continous, real values).

Nb Images Size Storage Time Machines Used
>= 45 millions (45446112) 400x400 pixels <= 4 To 2 weeks 12 GTX 1060, 1 GTX 1080, 2 Quadro P500

II. Descriptor Database

We used 3 standard computer vision descriptors plus one comming from deep learning. None of them seemed to give better results than the others in all cases.

NOTE: we reduced the size of the PPTBF database using one of the descriptor, thresholding them in 3 categoris (20%, 50% and 80%), binarizing them, computing descriptors, comparing descriptors with L2-distance given a threshold based on our observations (in terms of perceptual appearance, trying to remove similar candidates).

Threshold Nb Images
20%171702
50%140743
80%195112

We used 4 descriptors:

Appearance Space Navigation

We rely on the following cross-plateform technologies:

Technology Description
Anacondacross-plateform package framework
Pythonscientific programming
Numpyscientific computing framework
GPyGaussian Process framework
FLANNnearest neighbor search library

We rely on the FLANN library for fast nearest neighbor search to determine similar PPTBF candidates. Our target machine used 8 cores.

Nb Feature Vectors Generation Time Request Time (150 nearest neighbors)
5182524.977 s695 ms

We rely on the GPy python Gaussian Process framework.

Nb Elements (nearest neighbors) Nb Elements (after MCMC refinement) GPR (regression) GPLVM (latent variable model) Similarity Map
201503 s2 s0.3 s