Indeed, seems that VBO's make a diference
My vertices go into a static VBO at load time and not touched again.
A dynamic index buffer is used to build indices based on visible leaves - the bsp vis data is used as well as Irrlicht bbox tests on bsp tree bbox nodes.
I don't actually need 32bit indices as I could partition vertices based on material (as long as no material needs > 64K verts). This is still WIP.
And I haven't got round to adding objects yet - these can also be PVS tested against the bsp visdata based on which leaves the object is "in".