A Graph Theoretical Approach to Pruning Deep Neural Networks

Bachelor Thesis from the year 2024 in the subject Computer Sciences - Artificial Intelligence, grade: 100/100, Baden-Wuerttemberg Cooperative State University (DHBW) (Economics), course: Business Information Systems - Data Science, language: English, abstract: Imagine a world where deep learning models, despite their immense power, are no longer constrained by computational limitations. This vision fuels the innovative research presented herein, a journey into the realm of efficient deep neural networks through the lens of graph theory. This work introduces MLP-Rank, a groundbreaking method for network pruning that leverages the principles of weighted PageRank to identify and strategically remove redundant connections within multilayer perceptrons (MLPs). By representing the neural network architecture as a graph, the algorithm meticulously assigns importance scores to each connection, allowing for the targeted elimination of less crucial pathways, drastically reducing computational overhead without sacrificing accuracy. The core of this research delves into the algorithm's theoretical underpinnings, exploring its structural adaptations and modifications to the standard PageRank to optimize performance within neural network topologies. Rigorous experimentation across diverse datasets, including MNIST, Fashion-MNIST, and CIFAR-10, and various MLP architectures validates the efficacy of the MLP-Rank method, demonstrating significant improvements in inference speed and model compression. This exploration extends to a critical analysis of the theoretical assumptions against empirical results, bridging the gap between predicted and observed performance, and paving the way for future advancements in deep learning optimization. Discover how the synergy of graph theory and network pruning unlocks a new era of efficient, streamlined deep learning models, poised to revolutionize applications in resource-constrained environments, making AI more accessible and practical than ever before. This research is essential reading for anyone interested in Deep Neural Networks, Network Pruning, Graph Theory, Weighted PageRank, Inference Optimization, Sparsity, Accuracy, Speedup, MLP-Rank, and Model Compression, offering valuable insights into the future of efficient AI. The proposed methodology not only promises faster inference times but also contributes to the development of more sustainable and environmentally friendly AI solutions by reducing the energy footprint of deep learning models.