Foundations
Colour model and notation
Pixels arrive as 8-bit sRGB with an optional alpha channel. Throughout this page a pixel is
written p = (R, G, B, A) with each component in [0, 255], and the normalised form
x = v/255 in [0, 1] is used wherever the maths is scale-free.
The sRGB transfer function
Stored sRGB values are not proportional to light. The encoding applies a roughly 1/2.2 power
curve, so decoding to linear light is:
sRGB to linear ⎧ x / 12.92 if x ≤ 0.04045
L(x) = ⎨
⎩ ((x + 0.055) / 1.055)^2.4 otherwise
This matters because operations that are physically "adding light" - exposure, blurring,
alpha blending - are only strictly correct in linear space, while operations defined
perceptually - contrast, most stylistic filters - are conventionally done in the encoded space.
Image Lab follows the same convention as the CSS filter specification and the canvas 2D
compositor: pixel operations run on the encoded sRGB values. The trade is a small loss of
physical accuracy in exchange for results that match what the browser itself would produce, and
for a preview that agrees exactly with the committed render.
Luminance
Wherever a single brightness value is needed, the Rec. 709 luma coefficients are used:
LumaY = 0.2126·R + 0.7152·G + 0.0722·B
The weights are wildly unequal because human cone response is: green carries most of the
perceived brightness, blue almost none. Using a flat (R+G+B)/3 average instead is the classic
reason naive grayscale conversions look wrong.
Adjust panel
The tone curve is one composed function
Exposure, levels, brightness, contrast, shadow/highlight shaping, gamma and white-balance shift
are all point operations: the output for a pixel depends only on that pixel's own
value. Composing them gives a single function T: [0,255] → [0,255] per channel.
Composed per-channel tone operatorx₁ = x₀ · 2^EV exposure, in stops
x₂ = clamp( (255·x₁ − B) / (W − B), 0, 1 ) levels, black B and white W
x₃ = x₂^(1/γ_mid) midtone gamma
x₄ = x₃ · β brightness β
x₅ = (x₄ − ½)·κ + ½ contrast κ about mid grey
x₆ = x₅ + s·(1 − x₅)³·½ shadow lift, s ∈ [−1, 1]
x₇ = x₆ + h·(x₆)³·½ highlight lift, h ∈ [−1, 1]
x₈ = clamp(x₇, 0, 1)^(1/γ) output gamma
x₉ = x₈ + δ_c white balance offset, per channel
T(v) = round( 255 · clamp(x₉, 0, 1) )
Why the cubic weights
The shadow term is weighted by (1 − x)³ and the highlight term by x³. Both are unity at their
own end of the range and fall off fast, so a shadow lift leaves highlights untouched and vice
versa. A linear weight would bleed across the whole range and simply read as brightness.
Why it is a lookup table
Because T only depends on the input value, it is evaluated 256 times per channel and
cached. Applying it is then one array index per subpixel:
CostNaive: W·H·3 evaluations of the full chain
Tabled: 3·256 evaluations + W·H·3 array lookups
For a 24 MP image that is 768 evaluations instead of ~72 million.
This is the single reason the sliders can preview at frame rate: the expensive part is
independent of image size, and every additional slider is free because it folds into the same
table.
White balance
Temperature moves the red and blue channels in opposition; tint moves green against both. The
automatic version uses the grey-world assumption - that a scene averages to neutral - and
solves for per-channel gains:
Grey-world gainsμ_c = (1/N) Σ p_c for c ∈ {R, G, B}
μ̄ = (μ_R + μ_G + μ_B) / 3
g_c = μ̄ / μ_c then p_c ← clamp(g_c · p_c)
It fails predictably: a photo that genuinely is mostly one colour (a forest, a red wall) will be
pushed towards grey, because the assumption it rests on is false for that image.
Adjust panel
Saturation, vibrance and hue
These are not point operations on a single channel - they mix channels - so they run as a
second pass after the tone tables.
Saturation
Linear interpolation between the pixel and its own luminance, extrapolating past 1 to oversaturate:
Saturation, factor σp_c ← Y + σ·(p_c − Y) where Y is the luma of p
σ = 0 → fully desaturated (p_c = Y for all c)
σ = 1 → identity
σ > 1 → pushed away from the neutral axis
Vibrance
Vibrance is saturation with a per-pixel gain that shrinks as the pixel gets more saturated, so
already-vivid areas stop moving while muted ones keep going. Using HSV-style saturation as the
measure:
Vibrance, amount νS(p) = (max(R,G,B) − min(R,G,B)) / max(R,G,B) 0 when neutral, 1 when pure
σ_effective = σ + ν · (1 − S(p))
p_c ← Y + σ_effective·(p_c − Y)
The (1 − S) factor is the whole trick, and it is why vibrance is the safer control
for photos of people: skin sits at low-to-mid saturation, so it receives a much smaller boost
than a saturated background does.
Hue rotation
Hue rotation is a rotation about the neutral axis of RGB space, expressed as a 3x3 matrix. Image
Lab uses the matrix from the filter effects specification, which is what the browser's own
hue-rotate() uses:
Hue rotation by θ, with (l_R, l_G, l_B) = (0.213, 0.715, 0.072) ⎡ l_R ⎤ ⎡ 1−l_R −l_G −l_B ⎤ ⎡ −l_R −l_G 1−l_B ⎤
M(θ) = ⎢ l_R ⎥·1 + cos θ·⎢ −l_R 1−l_G −l_B ⎥ + sin θ·⎢ 0.143 0.140 −0.283 ⎥
⎣ l_R ⎦ ⎣ −l_R −l_G 1−l_B ⎦ ⎣ −(1−l_R) l_G l_B ⎦
(each row of the first term is (l_R, l_G, l_B))
[R' G' B']ᵀ = M(θ) · [R G B]ᵀ
Being a rotation about the grey axis, it leaves neutral colours neutral - grey stays grey at any
angle - and approximately preserves luminance, which is why it does not visibly brighten or
darken the image as you sweep it.
Filters panel
Convolution
Sharpen, emboss, edge detect, contour and smooth are all discrete 2D convolutions. For a kernel
K of odd side n = 2r+1:
Discrete 2D convolution 1 r r
(I ∗ K)(x, y) = ─── Σ Σ K(i, j) · I(x − i, y − j) + bias
d i=−r j=−r
d = divisor (usually ΣK, or 1 when ΣK = 0)
The divisor normalises gain. When ΣK = 1 the filter preserves average brightness; when
ΣK = 0 the result is centred on zero, which is why edge and emboss kernels are given a
bias of 128 to push the mid-point back into visible range.
The kernels in use
Sharpen - a discrete Laplacian added back to the identity. ΣK = 1.
0 −1 0
−1 5 −1
0 −1 0
Edge detect - the Laplacian alone. ΣK = 0, bias 128.
−1 −1 −1
−1 8 −1
−1 −1 −1
Emboss - an asymmetric directional derivative, lighting the image from the top-left.
−2 −1 0
−1 1 1
0 1 2
Smooth - a weighted box average, d = 13.
1 1 1
1 5 1
1 1 1
Boundary handling
Sampling outside the image is resolved by clamping coordinates to the edge (equivalent to
extending the border pixels outward). The alternatives - zeroing, wrapping, or mirroring -
produce a dark rim, a wrapped rim, and a reflected rim respectively. Clamping is chosen because
its artefact is the least visible.
Separability
A kernel is separable when it factors as an outer product K = uvᵀ,
which drops the cost from O(n²) to O(2n) per pixel. A Gaussian is
separable; the sharpen and emboss kernels above are not. This is exactly why Gaussian blur is
delegated to the compositor - a GPU-side separable pass beats anything done per-pixel in
JavaScript by a wide margin.
2D Gaussian, and its separation 1 −(i² + j²) / 2σ²
G(i, j) = ───── · e
2πσ²
1 −i²/2σ² 1 −j²/2σ²
= ──── e · ──── e = g(i) · g(j)
√(2π)σ √(2π)σ
Filters panel
Unsharp masking
The name is historical: the technique came from darkroom practice, where a blurred (unsharp)
positive was used as a mask against the negative. Digitally it is high-frequency boosting -
subtract a low-passed copy to isolate detail, then add a multiple of it back.
Unsharp maskD(x, y) = I(x, y) − (I ∗ G_σ)(x, y) the detail (high-pass) layer
⎧ I(x, y) + a · D(x, y) if |D(x, y)| ≥ t
O(x, y) = ⎨
⎩ I(x, y) otherwise
a = amount, σ = radius, t = threshold
What each control actually does
- Amount aGain on the detail layer. a = 1 doubles local contrast at the edges; beyond about 1.5 the overshoot becomes a visible halo.
- Radius σSets which spatial frequencies count as "detail". Small σ sharpens fine texture; large σ becomes local contrast enhancement rather than sharpening.
- Threshold tA deadband. Where the local difference is below t, the pixel is left alone, which stops the filter from amplifying sensor noise in flat sky or skin.
Note the relationship to the sharpen kernel above: with σ small enough that the Gaussian
collapses to a 3x3 neighbourhood, unsharp masking and the Laplacian sharpen kernel converge on
the same operator. Unsharp masking just exposes the radius and threshold as parameters instead
of hard-coding them.
Filters panel
The median filter is not a convolution
Median denoise selects the middle value of the sorted neighbourhood rather than a weighted sum.
That makes it a rank-order filter and, crucially, non-linear:
Median of a (2r+1)² windowO(x, y) = median{ I(x+i, y+j) : −r ≤ i, j ≤ r }
Non-linear: median(A + B) ≠ median(A) + median(B)
The consequence is the one property that makes it worth having: a median filter removes
impulse noise (single hot or dead pixels, salt-and-pepper speckle) essentially perfectly, because
an outlier can never be selected as the middle of the sorted set. A Gaussian blur, being an
average, instead smears the outlier over its whole neighbourhood.
The cost is O(WH · n² log n) with a naive sort per window,
which is why the window is kept small and the preview runs at display resolution.
Advanced panel
Histogram equalization
The intent is to redistribute intensities so the output histogram is as close to uniform as a
discrete remapping allows. The mechanism is the cumulative distribution function.
Global equalizationh(i) = #{ pixels with luma = i }, i ∈ [0, 255]
i
cdf(i) = Σ h(k)
k=0
⎡ cdf(i) − cdf_min ⎤
T(i) = round ⎢ ───────────────── · 255 ⎥
⎣ N − cdf_min ⎦
N = total pixels, cdf_min = smallest non-zero cdf value
The theory: if a variable is transformed by its own CDF, the result is uniformly distributed.
Quantisation to 256 integer levels means real images only approximate this.
Preserving colour
Equalizing R, G and B independently wrecks colour, because each channel gets a different
mapping and their ratios change. Image Lab equalizes luminance only and rescales chroma to
follow:
Chroma-preserving remapY = luma(p)
Y' = T(Y)
p_c ← clamp( p_c · (Y' / Y) ) with the Y = 0 case handled separately
Ratios between channels are preserved, so hue survives. The failure mode of global equalization
is different and unavoidable: it is driven by one histogram for the entire frame, so a large
uniform region (a sky) dominates the CDF and the interesting parts of the image get compressed.
That is the problem CLAHE exists to solve.
Advanced panel
CLAHE
Contrast Limited Adaptive Histogram Equalization computes a separate mapping per tile, clips
each histogram to bound the amount of amplification, and interpolates between tile mappings so
no tile boundaries appear. Three ideas, each fixing a failure of the previous one.
1. Adaptive: equalize per tile
The image is divided into a t x t grid; each tile gets its own histogram and
CDF. This solves the "sky dominates the frame" problem because each region is normalised
against its own statistics. On its own it introduces two new problems.
2. Contrast limited: clip and redistribute
In a nearly flat tile, the histogram is a tall spike, so the CDF has an enormous slope there and
equalization amplifies whatever noise exists into visible mush. Bounding the histogram height
bounds the slope, and therefore the amplification:
Clipping with redistributionlimit = ⌊ clip · n_tile / 256 ⌋ clip is the user's clip limit
excess = Σ max(0, h(i) − limit)
h(i) ← min(h(i), limit)
h(i) ← h(i) + ⌊excess / 256⌋ excess handed back uniformly
(remainder spread one per bin)
Redistribution matters: simply discarding the excess would change the total and distort the
CDF's endpoint. Handing it back uniformly keeps Σh = ntile.
3. Interpolated: blend the mappings
Applying each tile's mapping to its own pixels produces visible blocking at tile edges. Instead,
every pixel is mapped by bilinearly blending the four mappings whose tile centres
surround it:
Bilinear blend of neighbouring tile mappingsLet (u, v) be the pixel's position between the four surrounding
tile centres, u, v ∈ [0, 1), and T₀₀ … T₁₁ their mappings.
T(i) = (1−v)·[ (1−u)·T₀₀(i) + u·T₁₀(i) ]
+ v ·[ (1−u)·T₀₁(i) + u·T₁₁(i) ]
Note this interpolates the mapping functions, not the resulting pixel values. That
distinction is what makes the transition continuous across the whole image.
Choosing the parameters
- Clip limit1.0 is nearly a no-op. 2-3 is the usual working range. Above ~5 noise amplification dominates.
- Tile gridFewer, larger tiles approach global equalization. More, smaller tiles increase local detail but exaggerate noise and cost more.
Filters panel
Floyd-Steinberg error diffusion
Quantising to few levels produces banding, because a smooth gradient collapses into flat plateaus
with hard steps. Error diffusion fixes this by refusing to throw the quantisation error away:
each pixel's error is pushed onto neighbours that have not been processed yet, so the error
averages out spatially and the eye integrates it back into the original tone.
Quantisation with L levels per channelstep = 255 / (L − 1)
Q(v) = round(v / step) · step
err = v − Q(v)
The Floyd-Steinberg distribution kernel ⎡ · # 7/16 ⎤
⎢ 3/16 5/16 1/16 ⎥
⎣ ⎦
# = current pixel, · = already processed
Weights sum to 1, so no energy is created or lost.
I(x+1, y ) += err · 7/16
I(x−1, y+1) += err · 3/16
I(x , y+1) += err · 5/16
I(x+1, y+1) += err · 1/16
The scan order (left to right, top to bottom) is what makes this work: every target is a pixel
the loop has not reached yet, so the accumulated error is always available when that pixel is
finally quantised. Accumulation is done in floating point rather than in the 8-bit buffer,
because rounding the error at each step would defeat the entire purpose.
Two levels per channel gives the classic 8-colour newsprint look. Four to six gives a
recognisably retro image that still reads correctly at a distance. This is also, in essence,
what the GIF encoder does when reducing to a 256-colour palette.
Basic panel
Resampling
Resizing means reconstructing a continuous signal from discrete samples and re-sampling it on a
new grid. The reconstruction filter chosen is the entire quality question.
Lanczos kernel, order a (a = 3 typical) ⎧ sinc(x) · sinc(x / a) if |x| < a
Lₐ(x) = ⎨
⎩ 0 otherwise
where sinc(x) = sin(πx) / (πx), and sinc(0) = 1
Lanczos is a windowed sinc. The ideal reconstruction filter is sinc, but it has
infinite support; the second sinc term is a window that truncates it to ±a while
keeping most of its frequency response. Its negative lobes are what produce visibly crisper
edges than bilinear - and also what produce slight ringing overshoot near hard edges.
Aliasing when downscaling
Shrinking an image lowers the sampling rate. By the Nyquist criterion, any content above half
the new sampling frequency will alias - fold down into false low-frequency patterns, the moiré
you see on brick walls and fine fabric in a badly resized photo.
Nyquist limitf_max < f_s / 2
Downscaling by factor k reduces f_s by k, so everything above
the new f_s/2 must be filtered out BEFORE resampling, not after.
Browsers apply a single fixed-width filter step, which is inadequate for large reductions. So
for any downscale beyond 2x, Image Lab repeatedly halves the image first:
Progressive halvingwhile (current_width / 2 > target_width):
draw current into a canvas of half its size
then perform the final resize to the exact target
Each halving step is a low-pass filter followed by 2x decimation - which is precisely the
anti-aliasing that Nyquist demands. Skipping it is the single most common cause of a
"why does my thumbnail look terrible" bug.
The four options
- NearestZero-order hold. Blocky, but the only correct choice for pixel art and for any image where exact original values must survive.
- BilinearFirst-order. Cheap, slightly soft, no ringing.
- BicubicThird-order. Sharper than bilinear with mild overshoot.
- LanczosWindowed sinc. Sharpest detail retention; the usual default for photographs.
Advanced panel
Seam carving
Content-aware resizing changes an image's dimensions by removing paths of low-importance pixels
instead of scaling uniformly. Avidan and Shamir's formulation, and the one genuinely expensive
operation in the app.
Energy
The importance of a pixel is the magnitude of the image gradient at that pixel:
Gradient magnitude energye(x, y) = | ∂I/∂x | + | ∂I/∂y |
computed with a Sobel operator:
⎡ −1 0 +1 ⎤ ⎡ −1 −2 −1 ⎤
Gₓ = ⎢ −2 0 +2 ⎥ G_y = ⎢ 0 0 0 ⎥
⎣ −1 0 +1 ⎦ ⎣ +1 +2 +1 ⎦
Flat regions score near zero; edges and textured detail score high.
The optimal seam
A vertical seam is a connected top-to-bottom path with exactly one pixel per row, where
consecutive rows shift by at most one column. Finding the minimum-energy seam by brute force
would be O(3H). Dynamic programming reduces it to a single pass:
Cumulative minimum energyM(x, y) = e(x, y) + min{ M(x−1, y−1),
M(x, y−1),
M(x+1, y−1) }
with M(x, 0) = e(x, 0) on the first row.
After filling M, the smallest value in the bottom row is the end of the cheapest seam;
backtracking the argmin choices recovers the path. Remove it, and repeat once per column of
width change.
CostOne seam: O(W · H)
k seams: O(k · W · H) energy must be recomputed after each removal
Reducing a 4000x3000 image by 500 px ≈ 6 × 10⁹ operations.
That total is why this runs on the server and why it is the one operation with a visible
progress indicator. It also explains the failure mode: seam carving assumes low-gradient means
unimportant, so it will happily carve straight through a smooth face or a clear sky horizon
while carefully preserving a textured background.
Basic panel
Alpha compositing
Flattening transparency, padding, and JPEG export all composite the image over an opaque
background using the Porter-Duff over operator:
Source over destinationα_o = α_s + α_d·(1 − α_s)
C_s·α_s + C_d·α_d·(1 − α_s)
C_o = ───────────────────────────────
α_o
With an opaque destination (α_d = 1, so α_o = 1) this reduces to:
C_o = C_s·α_s + C_d·(1 − α_s)
The division by αo is the un-premultiply step. Canvas 2D stores premultiplied alpha
internally, and the round trip through 8-bit premultiplied storage is lossy for low alpha values
- a pixel at α = 1/255 has only one bit of colour precision left. This is why repeatedly
applying and undoing transparency operations slowly degrades edge quality, and why the source
blob is kept intact rather than being re-derived from the canvas on every step.
JPEG has no alpha channel at all, so exporting to JPEG composites over white first. That is a
one-way operation, which is why the Flatten control exists: it makes the step explicit and lets
you choose the background colour rather than silently getting white.
Reference
Complexity summary
W, H are image dimensions, n the kernel side, t the CLAHE
tile count, k the number of seams.
Every client-side preview runs against a display-resolution copy - typically under a megapixel -
rather than the source image. Applying re-runs the identical code at full resolution. That is
the whole performance strategy, and it is why an O(WH·n² log n) median filter is still usable
interactively on a 24 megapixel photograph.
