Detection and prevention of Sybil attacks in Optimized Link State Routing protocol for Mobile Ad-Hoc Networks
Abstract
Mobile Ad Hoc Networks (MANETs) are decentralized and infrastructure-less wireless networks characterized by dynamic topology and limited resources, making them highly susceptible to security threats such as Sybil attacks. The Optimized Link State Routing (OLSR) protocol, widely used for its proactive routing and Multi-Point Relay (MPR) efficiency, is particularly vulnerable to such attacks, where malicious nodes assume multiple identities to disrupt routing operations. This study proposes an enhanced OLSR protocol integrated with an Improved Grey Wolf Optimizer (IGWO) for effective detection and mitigation of Sybil attacks. In the proposed model, the IGWO algorithm is applied to optimize route selection and detect malicious nodes by modeling network nodes as search agents and evaluating their trustworthiness based on routing behavior, neighborhood consistency, and message exchange patterns. Specifically, HELLO and Topology Control (TC) messages are analyzed to identify anomalies associated with Sybil identities, while the IGWO dynamically selects optimal routing paths by prioritizing trustworthy nodes. The model was implemented and simulated in MATLAB under varying node mobility and packet sizes. Performance evaluation was conducted using packet delivery ratio (PDR), throughput, and end-to-end delay. Simulation results show that the proposed IGWO-OLSR protocol significantly outperforms the conventional OLSR protocol, achieving higher throughput (up to 20,570 Kbps), improved packet delivery ratio (up to 0.967), and reduced end-to-end delay, particularly under high mobility conditions. These results demonstrate that the integration of the Grey Wolf Optimizer enhances routing efficiency, improves detection of Sybil attacks, and strengthens overall network security. The proposed approach provides a robust and adaptive framework for securing routing protocols in dynamic MANET environments.References
Abolhasan, M., Wysocki, T., and Dutkiewicz, E. (2009). A review of routing protocols for mobile ad hoc networks. Ad Hoc Networks, 2(1), 1–22.
Akyildiz, I. F., Wang, X., and Wang, W. (2020). Next-generation routing protocols for wireless ad hoc networks. IEEE Communications Magazine, 58(2), 110-119.
Aljohani, A. J. (2021). A comprehensive survey on security attacks and countermeasures in mobile ad hoc networks. IEEE Access, 9, 156878–
https://doi.org/10.1109/ACCESS.2021.3129635
Clausen, T., and Jacquet, P. (2003). Optimized Link State Routing Protocol (OLSR). RFC 3626. IETF.
Conti, M., and Giordano, S. (2014). Mobile ad hoc networking: Milestones, challenges, and new research directions. IEEE Communications Magazine, 52(1), 85–96.
Douceur, J. R. (2002). The Sybil attack. In Proceedings of the 1st International Workshop on Peer-to-Peer Systems, 251–260.
Haas, Z. J., Pearlman, M. R., and Samar, P. (2002). The zone routing protocol (ZRP) for ad hoc networks. Internet Draft.
Huhtonen, T. (2022). Security threats and trust management in mobile ad hoc networks. Journal of Network and Computer Applications, 198, 103274. https://doi.org/10.1016/j.jnca.2021.103274
Johnson, D. B., and Maltz, D. A. (1996). Dynamic source routing in ad hoc wireless networks. Mobile Computing, 353–367.
Mahdy, A.M.S.; Lotfy, K.; Hassan, W.; El-Bary, A.A. (2021). Analytical solution of magneto-photo thermal theory during variable thermal conductivity of a semiconductor material due to pulse heat flux and volumetric heat source. Waves Random Complex Media, 31, 2040– 2057.
Mirjalili, S., Mirjalili, S. M., and Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69, 46–61.
Neeraj, K. (2021). Cooperative Sybil attacks in OLSR: Threats and countermeasures. Journal of Network Security and Applications, 31(3), 112-125.
Newsome, J., Shi, E., Song, D., and Perrig, A. (2004). The Sybil attack in sensor networks: Analysis and defenses. In Proceedings of the 3rd International Symposium on Information Processing in Sensor Networks, 259–268.
Qayyum, A. (2019). Security challenges in OLSR protocol: A survey. Journal of Network and Computer Applications, 22(1), 85-95.
Sarkar, S. K., Basavaraju, T. G., and Puttamadappa, C. (2013). Ad hoc mobile wireless networks: Principles, protocols, and applications. Auerbach Publications.
Siau, J., and Shen, C. (2020). Performance optimization techniques for ad hoc networks. Journal of Computer Networks, 25(6), 152-165.
Vecchi, M. P. (2022). Simulated annealing and modern optimization techniques. Springer.
Wagner, D. (2021). Wireless ad hoc and sensor networks: Principles, architectures, and applications. Springer Nature.
Wang, P. (2021). Scalability and performance optimization in OLSR networks. Ad Hoc and Sensor Wireless Networks, 25(2), 89-101.
Xiaohui, L. (2020). Sybil attack mitigation techniques in MANETs.
International Journal of Computer Networks and Communications, 30(5), 78-90.
Yen, C., and Chou, T. (2021). Adaptive routing algorithms in dynamic mobile networks. International Journal of Computer Science and Applications, 19(1), 92-105.
Zhang, L. (2019). Optimized Link State Routing for wireless ad hoc networks. International Journal of Network Protocols, 15(4), 102-115.
Copyright (c) 2026 Cecilia Agunbiade, Funmilola Ipeayeda, Adeleye Falohun, OLadayo Olakanmi, John O. Ogunyode

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The authors hereby represent and warrant that the paper is original and that they are the authors of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. If in future any violation of any copyright come in notice, then the author will be responsible and not FUOYEJET.
The authors declare that:
- This paper has not been published in the same form elsewhere.
- It will not be submitted anywhere else for publication prior to acceptance/rejection by this Journal.
- A copyright permission is obtained for materials published elsewhere and which require this permission for reproduction.
Furthermore, the copyright after publication belongs to the Author(s) (for articles published in 2020 and beyond) and licensed under the creative commons license CC-BY-NC (http://creativecommons.org/licenses/by-nc/4.0). The copyright covers the right to reproduce and distribute the article, including reprints, translations, photographic reproductions, microform, electronic form (offline, online) or any other reproductions of similar nature.