Paintings are usually experienced visually. Argira asks whether their structure can also be experienced through sound.
Access to visual art remains a significant barrier for blind and visually impaired (VI) individuals. Verbal descriptions offer only partial access to the visual structure.
Sonification — translating data into sound — offers an alternative that preserves structural properties.
Output WAV → spectrogram → Ds (spectral fractal dimension) via box-counting on log-scaled STFT
Hann 1024-seg · overlap 768 · FFT 2048 · 80th percentile · scales 4–64
Monet · Poppies → high pitch, complex timbre (802 Hz, OSC2 active)
Van Gogh · Starry Night → dense texture, low frequency (272 Hz)
Malevich · Black Square → simple tone, silence
R²=0.8417 · p<0.001 · Argira v3.5
| Correlation | r | N | Type |
|---|---|---|---|
| hue_std → Ds | 45 | Designed | |
| hue_std → Ds (replic.) | 74 | Replication | |
| hue_std → Roughness | 30 | Emergent | |
| hue_std → Rough. (−outlier) | 29 | Emergent | |
| local_contrast → freq_hz | 74 | Structural | |
| edge_density → effort | 74 | Structural |
Argira captures structure from paintings that is mathematically measurable and perceptually meaningful, without explicit perceptual rules.
The suppression effect of Dv reveals that visual fractal complexity contributes to auditory fractal complexity independently of colour — mirroring findings in crossmodal correspondences.
Independent pipeline (bandlimited sawtooth, no fractal metrics) confirms hue_std → emergent spectral roughness: r=0.687 (r=0.790 excl. Malevich). Physical consequence of harmonic superposition. Replicated across 4 waveform types. Pixel-shuffle control (EXP-2) confirms global histogram property.
Core correlation r=0.923 (R²=0.852, permutation p<0.001, N=1000). Fully consistent with the original result (r=0.887, N=45). Not an artifact of the initial corpus selection.
Two images with identical hue_std=0.500 but opposite saturation → completely different acoustic profiles (6 vs 13 emergent harmonics). Pure sine waves reduce r from 0.887 to 0.44.
Systematic matrix S=(|r|+|ρ|)/2 and δ=|ρ|−|r| across 8 visual descriptors and 7 acoustic dimensions. Chromatic family (hue_std, hue_entropy_bits) → frequency; structural family (edge_density, luminance_contrast) → texture and timing. hue_std and hue_entropy_bits are non-redundant. fractal_D: honest null result (max S ≈ 0.27). Roughness concentrates the non-linearity. Key distinction: operator-imposed structure (S≈1.000) vs emergent corpus properties.
Argira Perceptual Field (v11): continuous navigable field in any browser. 74 works in 2D space (chromatic variety × visual complexity) with Gaussian kernel (σ=0.18, k=7). Designed with feedback from a blind user (Felix).
CC BY-NC 4.0 · Zenodo
10.5281/zenodo.18865019
Datasets, source code, statistical analyses and extended experiments publicly available.
[1] Ramachandran & Hubbard — Synaesthesia (2001)
[2] Iosifyan et al. — Paintings & Music (2022)
[3] Spence — Crossmodal correspondences (2011)
[7] Taylor et al. — Fractal analysis of Pollock (1999)