Areas of swarm intelligence and machine learning are constantly evolving, recently attracting even more researchers world-wide. This stems from the no free lunch which states that universal approach that could render satisfying results for all practical challenges does not exist. Therefore, in this research a novel multi-swarm firefly algorithm, that tries to address flaws of original firefly metaheuristics, is proposed. Devised algorithm is applied to interesting and important practical challenge of plants classification, as part of the hybrid framework between machine learning and optimization metaheuristics. For this purpose, a set of 1,000 random images of healthy leaves, from one public repository, is retrieved for the following plants: apple, cherry, pepper and tomato. Hybrid framework includes pre-processing, constructing bag of features and classification steps. After pre-processing, a bag of features is constructed by utilizing well-known scale-invariant feature transform algorithm, K-means-based vocabulary generation and histogram. Such images are then categorized with support vector machine classifier. However, to obtain satisfying results for a particular dataset, the support vector machines hyper-parameters' need to be tuned and in the proposed research multi-swarm firefly algorithm is employed to determine optimal (sub-optimal) hyper-parameters' values for this practical challenge. Comparative analysis with the basic firefly metaheuristics and other well-known swarm intelligence algorithms was conducted to assess the performance of the proposed method in terms of precision, recall, F-score for this multi-class classification challenge. Obtained results show significant performance improvements of devised method over the original firefly algorithm and also better metrics than other state-of-the-art techniques in the majority of cases.