Your question is a bit open-ended. It'd be great if you give us an idea about the data like the data size, the model which you tried, etc. as that would help in getting relatable answers from the community.
One of the most important problems we usually face is handling the imbalanced data. However, there are some basic approaches to handle this like Image Augmentation techniques to increase the number of samples.
But your results can also be biased because of the complexity of the model. It might overfit easily as you mentioned the data is less. So please try plotting the train-test accuracy and loss graphs to get an idea when it is leading to overfitting.
Another approach that might help is the size of the image. Having the usual size of 64x64 might be very small for the model to extract features. Try increasing the size, use color images instead of converting to grayscale, and definitely augment the images.