How can I use a fragment shader along with a geometry buffer array to create points in React-Three-Fiber

Question

To clarify I am using React-Three-Fiber

I would like to create a bunch of points on the canvas, beforehand I was using buffered arrays of colors and positions to plot the points. Here is my attempt at getting it work with buffered positions and using shaderMaterial along with a fragmentShader

    <points>
        <bufferGeometry attach="geometry">
            <bufferAttribute
                attach="attributes-position"
                count={positions.length / 3}
                array={positions}
                itemSize={3}
                usage={THREE.DynamicDrawUsage}
            />
        </bufferGeometry>
        <shaderMaterial fragmentShader={fragmentShader}/>
    </points>

I am able to use the shader with other geometries so I know the mistake is in the way I am trying to use points. Using pointsMaterial or meshStandardMaterial worked fine, could anyone help me understand how to go about using shaders with points in R3F?