Firstly, even though you were able to train your model successfully with 170 train images, it is best if you have up to 1000 images for the 2 hand symbols at least to ensure your trained model can achieve considerable accuracy on most image it predicts.
Secondly, the speed of prediction will depend on the model type you trained (DenseNet is the slowest and most accurate, ResNet is moderately accurate and fast).
Lastly, what are the specifications of your hardware?