Optical vortex states-higher optical modes with helical phase progression and carrying orbital angular momentum-have been explored to increase the flexibility and capacity of optical fibres employed for example in mode-division-multiplexing, optical trapping and multimode imaging. A common requirement in such systems is high fidelity transfer of signals between different frequency bands and modes, which for vortex modes is not so straightforward. Here we report intervortex conversion between backward-propagating circularly polarised vortex modes at one wavelength, using chiral flexural phonons excited by chiral forward stimulated Brillouin scattering at a different wavelength. The experiment is carried out using chiral photonic crystal fibre, which robustly preserves circular polarisation states. The chiral acoustic wave, which has the geometry of a spinning single-spiral corkscrew, provides the orbital angular momentum necessary to conserve angular momentum between the coupled optical vortex modes. The results open up new opportunities for interband optical frequency conversion and the manipulation of vortex states in both classical and quantum regimes.