Paintings are experienced visually. Argira asks whether their structure can also be experienced through sound.
Access to visual art remains a barrier for blind and low-vision people. Verbal descriptions give only partial access. Sonification offers an alternative that preserves structure.
The output WAV spectrogram is analysed to extract the spectral fractal dimension (Ds) via box-counting on log-scaled STFT.
Hann 1024-seg · overlap 768 · FFT 2048 · 80th percentile · scales 4–64
Monet's Poppies → high pitch, complex timbre (802 Hz, dual oscillator)
Van Gogh's Starry Night → dense texture, low frequency (272 Hz, high granular modulation)
Malevich → simple, quiet tone
X — hue_std: chromatic variability. Left = monochromatic, Right = full spectrum.
Y — Ds: spectral fractal dimension of the generated sound. Up = more complex.
Each point's colour encodes its position on X: blue → cyan → green → amber.
Ds ~ hue_std + Dv + tempo
β(hue_std) = 0.3265 · t = 12.981
β(Dv) = 0.1208 · t = 3.236
β(tempo) = −0.0002 · t = −0.721
R² = 0.8417 · p < 0.001
| Correlation | r | N | Type |
|---|---|---|---|
| hue_std → Ds | 0.887 | 45 | Designed |
| hue_std → Ds (replic.) | 0.923 | 74 | Replication |
| hue_std → Roughness | 0.687 | 30 | Emergent |
| hue_std → Rough. (no outlier) | 0.790 | 29 | Emergent |
| local_contrast → freq_hz | 0.949 | 74 | Structural |
| edge_density → effort | 0.759 | 74 | Structural |
Argira captures painting structure that is mathematically measurable and perceptually meaningful, without explicit perceptual rules.
The Dv suppression effect reveals that visual fractal complexity contributes to the auditory one independently of colour — mirroring findings in crossmodal correspondences.
An independent pipeline (band-limited sawtooth, no fractal metrics) confirms that hue_std predicts emergent spectral roughness: r=0.687 (r=0.790 excl. Malevich). A physical consequence of harmonic superposition, not of the algorithm's design. Replicated across 4 waveform types.
Core correlation r=0.923 (R²=0.852, permutation p<0.001, N=1000). Fully consistent with the original result (r=0.887, N=45). Not an artifact of the initial selection.
Two images with identical hue_std=0.500 but opposite saturation produce completely different acoustic profiles (6 vs 13 emergent harmonics). Harmonic superposition is the critical generative mechanism. Pure sine waves reduce r from 0.887 to 0.44.
Layer A (verified): stereo position preserves chromatic centre of mass, error <0.05.
Layer B (strong): local_contrast→sonic_freq_hz r=0.949; edge_density→effort r=0.759 (p<10⁻¹⁴).
Layer C (limit): direct inversion not currently feasible — a design choice, not a theoretical impossibility.
Systematic matrix of associations between 8 visual descriptors and 7 acoustic dimensions across 117 paintings. For each pair we compute S = (|r|+|ρ|)/2 (association strength) and δ = |ρ|−|r| (non-linearity index).
4 main findings:
① Two independent families — chromatic (hue_std, hue_entropy_bits) → frequency; structural (edge_density, luminance_contrast) → texture and timing.
② hue_std and hue_entropy_bits are not redundant — they independently predict distinct acoustic targets.
③ fractal_D: an honest null result — minimal predictive power (S max ≈ 0.27). Reported explicitly.
④ Roughness concentrates non-linearity — interaction effects not captured by linear models.
Key distinction: structure imposed by the operator (S≈1.000 in hue_std→odd_bias and luminance_contrast→decay_s) vs emergent corpus properties. That distinction is the central methodological contribution.
Argira Perceptual Field (v11): a continuous, navigable perceptual field running in any browser. 74 works organised in a 2D space (chromatic variety × visual complexity) with a Gaussian kernel (σ=0.18, k=7). Designed from feedback by a blind user (Felix).
74 works arranged in a two-dimensional space. The sound evolves continuously with cursor or touch movement — no discrete categories, no abrupt transitions.
CC BY-NC 4.0 · Zenodo
10.5281/zenodo.18865019
Datasets, source code, statistical analyses and extended experiments publicly available.
[1] Ramachandran & Hubbard — Synaesthesia (2001)
[2] Iosifyan et al. — Paintings & Music (2022)
[3] Spence — Crossmodal correspondences (2011)
[7] Taylor et al. — Fractal analysis of Pollock (1999)