Object recognition with competitive convolutional neural networks

Basit Öğe Kaydı
dc.authorid0000-0001-9033-8934
dc.contributor.advisorEskil, M. Taneren_US
dc.contributor.authorErkoç, Tuğbaen_US
dc.contributor.otherIşık Üniversitesi, Lisansüstü Eğitim Enstitüsü, Bilgisayar Mühendisliği Doktora Programıen_US
dc.date.accessioned2023-08-03T12:23:10Z
dc.date.available2023-08-03T12:23:10Z
dc.date.issued2023-06-12
dc.departmentIşık Üniversitesi, Lisansüstü Eğitim Enstitüsü, Bilgisayar Mühendisliği Doktora Programıen_US
dc.descriptionText in English ; Abstract: English and Turkishen_US
dc.descriptionIncludes bibliographical references (leaves 82-90)en_US
dc.descriptionxvii, 91 leavesen_US
dc.description.abstractIn recent years, Artificial Intelligence (AI) has achieved impressive results, often surpassing human capabilities in tasks involving language comprehension and visual recognition. Among these, computer vision has experienced remarkable progress, largely due to the introduction of Convolutional Neural Networks (CNNs). CNNs are inspired by the hierarchical structure of the visual cortex and are designed to detect patterns, objects, and complex relationships within visual data. One key advantage is their ability to learn directly from pixel values without the need for domain expertise, which has contributed to their popularity. These networks are trained using supervised backpropagation, a process that calculates gradients of the network’s parameters (weights and biases) with respect to the loss function. While backpropagation enables impressive performance with CNNs, it also presents certain drawbacks. One such drawback is the requirement for large amounts of labeled data. When the available data samples are limited, the gradients estimated from this limited information may not accurately capture the overall data behavior, leading to suboptimal parameter updates. However, obtaining a sufficient quantity of labeled data poses a challenge. Another drawback is the requirement of careful configuration of hyperparameters, including the number of neurons, learning rate, and network architecture. Finding optimal values for these hyperparameters can be a time-consuming process. Furthermore, as the complexity of the task increases, the network architecture becomes deeper and more complex. To effectively train the shallow layers of the network, one must increase the number of epochs and experiment with solutions to prevent vanishing gradients. Complex problems often require a greater number of epochs to learn the intricate patterns and features present in the data. It’s important to note that while CNNs aim to mimic the structure of the visual cortex, the brain’s learning mechanism does not necessarily involve back-propagation. Although CNNs incorporate the layered architecture of the visual cortex, the reliance on backpropagation introduces an artificial learning procedure that may not align with the brain’s actual learning process. Therefore, it is crucial to explore alternative learning paradigms that do not rely on backpropagation. In this dissertation study, a unique approach to unsupervised training for CNNs is explored, setting it apart from previous research. Unlike other unsupervised methods, the proposed approach eliminates the reliance on backpropagation for training the filters. Instead, we introduce a filter extraction algorithm capable of extracting dataset features by processing images only once, without requiring data labels or backward error updates. This approach operates on individual convolutional layers, gradually constructing them by discovering filters. To evaluate the effectiveness of this backpropagation-free algorithm, we design four distinct CNN architectures and conduct experiments. The results demonstrate the promising performance of training without backpropagation, achieving impressive classification accuracies on different datasets. Notably, these outcomes are attained using a single network setup without any data augmentation. Additionally, our study reveals that the proposed algorithm eliminates the need to predefine the number of filters per convolutional layer, as the algorithm automatically determines this value. Furthermore, we demonstrate that filter initialization from a random distribution is unnecessary when backpropagation is not employed during training.en_US
dc.description.abstractSon yıllarda Yapay Zeka (YZ) dili anlama ve görsel tanımayı içeren görevlerde genellikle insan yeteneklerini geride bırakarak etkileyici sonuçlar elde etti. Bunların arasında, bilgisayarla görme, büyük ölçüde Evrişimli Sinir Ağlarının (ESA) ortaya çıkması ile dikkate değer bir ilerleme kaydetti. ESAlar, görsel korteksin hiyerarşik yapısından ilham alarak görsel verilerdeki kalıpları, nesneleri ve karmaşık ilişkileri tespit etmek icin tasarlanmıştır. En önemli avantajlarından biri, popülerliklerine katkıda bulunan, bir uzmana ihtiya. Duymadan doğrudan piksel değerlerinden öğrenme yetenekleridir. Bu ağlar, kayıp fonksiyonuna göre ağ parametrelerinin (ağrılıklar ve eğilimler) gradyanlarını hesaplayan denetimli geri yayılım ile eğitilir. Geri yayılım, ESAlarda etkileyici bir performans sağlarken, bazı dezavantajlar da getirir. Bu dezavantajlardan biri büyük miktarlarda etiketlenmiş veri gereksinimidir. Mevcut veri örnekleri sınırlı olduğunda, bu sınırlı bilgiden hesaplanan gradyanlar , genel veri davranışını doğru bir şekilde yakalayamayabilir ve bu da yetersiz parameter güncellemelerine yol açar. Bununla birlikte, yeterli miktarda etiketlenmiş veri elde etmek bir zorluk teşkil etmektedir. Diğer nir dezavantaj nöron sayısı, öğrenme hızı ve ağ mimarisi dahil olmak üzere hiperparametrelerin dikkatli bir şekilde yapılandırılması gerekliliğidir. Bu hiperparametreler için en uygun değerleri bulmak zaman alıcı bir süreç olabilir. Ayrıca, görevin karmaşıklığı arttıkça ağ mimarisi daha derin ve karmaşık bir hale gelir. Ağın sığ katmanlarını etkili bir şekilde eğitmek için, epok sayısı artırılmalı ve kaybolan gradyanları önlemek için çözümler üretilmelidir. Karmaşık problemler, verilerde bulunan karmaşık kalıpları ve özellikleri öğrenmek için genellikle daha fazla sayıda epok gerektirir. ESAlar görsel korteksin yapısını taklit etmeyi amaçlasa da, beynin öğrenme mekanizmasının mutlaka geri yayılımı içermediğini not etmek önemlidir. ESAlar görsel korteksin katmanlı mimarisini içermelerine rağmen, geri yayılıma dayanan öğrenme, beyningerçek öğrenme süreciyle uyumlu olmayabilen yapay bir öğrenme prosedürü sunar. Bu nedenle, geri yayılıma dayanmayan alternatif öğrenme paradigmalarını keşfetmek önem teşkil etmektedir. Bu tez çalışmasında, önceki araştırmalardan farklı olarak ESAlar için denetimsiz eğitime yönelik benzersiz bir yaklaşım araştırılmaktadır. Önerilen yaklaşım diğer denetimsiz yöntemlerin aksine, filtrelerin eğitimi için geri yayılmaya olan bağlılığı kaldırır. Geri yayılım ile öğrenme yerine, veri etiketleri veya geriye dönük hata güncellemeleri gerektirmeden görüntüleri yalnızca bir kez işleyerek veri kümesi özelliklerini çıkarabilen bir filtre çıkarma algoritması sunuyoruz. Bu yaklaşım bireysel Evrişimli katmanlar üzerinde çalışır ve filtreleri eğitim örnekleri üzerinden keşfederek evrişim katmanının filtrelerini kademeli olarak oluşturur. Bu geri yayılımsız algoritmanın etkinliğini değerlendirmek için dört farklı ESA mimarisi tasarladık ve deneyler yaptık. Sonuçlar, farklı veri kümelerinde etkileyici sınıflandırma doğrulukları elde ederek, geri yayılım olmadan eğitimin mümkün olabileceğini göstermektedir. Özellikle, bu sonuçlara herhangi bir veri arttırımı olmadan vet ek bir ağ kullanılarak ulaşılmıştır. Ek olarak, çalışmamızda önerilen algoritma, evrişim katmanı başına filtre sayısını önceden belirleme ihtiyacını ortadan kaldırmaktadır çünkü algoritmamız bu değeri otomatik olarak belirlemektedir. Ayrıca, eğitim sırasında geri yayılım kullanılmadığından rastgele bir dağılımdan filtrelere ilkdeğer verilmesinin gereksiz olduğunu da bu çalışma ile gösterdik.en_US
dc.description.tableofcontentsConvolutional Neural Network Architectureen_US
dc.description.tableofcontentsConvolutional Layeren_US
dc.description.tableofcontentsActivation Functionen_US
dc.description.tableofcontentsConvolution Operationen_US
dc.description.tableofcontentsPooling Layeren_US
dc.description.tableofcontentsFully Connected Layeren_US
dc.description.tableofcontentsOutput Layeren_US
dc.description.tableofcontentsTraining of CNNen_US
dc.description.tableofcontentsLITERATURE SURVEYen_US
dc.description.tableofcontentsInitial Stepsen_US
dc.description.tableofcontentsBackpropagation Eraen_US
dc.description.tableofcontentsFundamental Deep Learning Problemen_US
dc.description.tableofcontentsRevival of the Neural Networks Researchen_US
dc.description.tableofcontentsGPU Eraen_US
dc.description.tableofcontentsUnsupervised Learning with Backpropagationen_US
dc.description.tableofcontentsNeocognitronen_US
dc.description.tableofcontentsConvolutional Filter Discoveryen_US
dc.description.tableofcontentsCenter of Gravity Based Candidate Filter Extractionen_US
dc.description.tableofcontentsUnsupervised Learning Algorithm for Convolutional Layers of CCNN Architectureen_US
dc.description.tableofcontentsMNISTen_US
dc.description.tableofcontentsEMNIST-Digitsen_US
dc.description.tableofcontentsKuzushiji-MNISTen_US
dc.description.tableofcontentsFashion-MNISTen_US
dc.description.tableofcontentsMNIST Experiment Resultsen_US
dc.description.tableofcontentsEMNIST-Digits Experiment Resultsen_US
dc.description.tableofcontentsKuzushiji-MNIST Experiment Resultsen_US
dc.description.tableofcontentsFashion-MNIST Experiment Resultsen_US
dc.description.tableofcontentsFilters Discovered via Proposed Unsupervised Processen_US
dc.description.tableofcontentsMNIST Dataseten_US
dc.description.tableofcontentsEMNIST-Digits Dataseten_US
dc.description.tableofcontentsKuzushiji-MNIST Dataseten_US
dc.description.tableofcontentsFashion-MNIST Dataseten_US
dc.description.tableofcontentsExtracted Filters in Subsequent Layersen_US
dc.description.tableofcontentsSamples with Incorrect Classificationen_US
dc.description.tableofcontentsIncorrectly Classified MNIST Samplesen_US
dc.description.tableofcontentsIncorrectly Classified EMNIST-Digits Samplesen_US
dc.description.tableofcontentsIncorrectly Classified Kuzushiji-MNIST Samplesen_US
dc.description.tableofcontentsIncorrectly Classified Fashion-MNIST Samplesen_US
dc.description.tableofcontentsComparison of Performance Against Other Studiesen_US
dc.description.tableofcontentsComparison of Performance Against Unsupervised Studiesen_US
dc.description.tableofcontentsComparison of Performance Against Mixed Studiesen_US
dc.description.tableofcontentsComparison of Performance Against Supervised Studiesen_US
dc.description.tableofcontentsProof of Linear Independence of the Extracted Filtersen_US
dc.description.tableofcontentsProof of Independence over the Order of Candidate Processing for Filter Extractionen_US
dc.description.tableofcontentsComparison to Low-Capacity CNNen_US
dc.description.tableofcontentsCCNN networks that are used in the experiments with various datasets. Convolutional layers either use 5 × 5 or 3 × 3 filters. Maxpooling is applied on the feature maps on 2 × 2 windows with strides of 2. The size of the convolutional filters is denoted with n while the maxpooling window size is shown with men_US
dc.description.tableofcontentsExtracted filter counts and the test accuracy of individual models on MNIST dataseten_US
dc.description.tableofcontentsThe confusion matrix represents the performance of Model A on the MNIST dataseten_US
dc.description.tableofcontentsPerformance metrics of Model type A for individual classes of MNIST dataseten_US
dc.description.tableofcontentsExtracted filter counts and the test accuracy of individual models on EMNIST-Digits dataseten_US
dc.description.tableofcontentsThe confusion matrix represents the performance of Model A on the EMNIST-Digits dataseten_US
dc.description.tableofcontentsPerformance metrics of Model type A for individual classes of EMNISTDigits dataseten_US
dc.description.tableofcontentsExtracted filter counts and the test accuracy of individual models on Kuzushiji-MNIST dataseten_US
dc.description.tableofcontentsThe confusion matrix represents the performance of Model B on the Kuzushiji-MNIST dataseten_US
dc.description.tableofcontentsPerformance metrics of Model type B for individual classes of KuzushijiMNIST dataseten_US
dc.description.tableofcontentsExtracted filter counts and the test accuracy of individual models on Fashion-MNIST dataseten_US
dc.description.tableofcontentsThe confusion matrix represents the performance of Model B on the Fashion-MNIST dataset. The classes are assigned to numbers ranging from 0 to 9. In order, the class labels correspond to Tshirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, and Ankle booten_US
dc.description.tableofcontentsPerformance metrics of Model type B for individual classes of FashionMNIST dataseten_US
dc.description.tableofcontentsComparison between previous works and our method for the number of epochs of training needed for convolutional filters, whether data augmentation and ensemble of networks are used. The legend of the table: ✓: applied, × : not applied, NA: no information availableen_US
dc.description.tableofcontentsComparison of the proposed method with other unsupervised studiesen_US
dc.description.tableofcontentsComparison of the proposed method with other mixed studiesen_US
dc.description.tableofcontentsComparison of the proposed method with other supervised studiesen_US
dc.description.tableofcontentsComparison of best performing model filter counts and test accuracy before and after addition of candidate shuffling. The median of the 50 runs of the experiments is also presenteden_US
dc.description.tableofcontentsA typical Convolutional Neural Networken_US
dc.description.tableofcontentsCNNs can recognize high level concepts like face by hierarchically building feature detectors starting from basic edge like shapes to complex features like eyesen_US
dc.description.tableofcontentsSigmoid, hyperbolic tangent and ReLU activation function curves shown. ReLU is most popular activation function in CNNsen_US
dc.description.tableofcontentsPReLU and Leaky ReLU activation function curves. PReLU and Leaky ReLU allows a small gradient for negative values whereas original ReLU strictly sets the negative values to zeroen_US
dc.description.tableofcontentsVisualization of convolution operation on with a 5 × 5 image and a 3 × 3 filter with a stride of 1 pixelen_US
dc.description.tableofcontentsVisualization of convolution operation on with a 5 × 5 image and a 3 × 3 filter with a stride of 1 pixel and zero padding of 1 pixel as per Equation 2.2. Green background is padding while the image data is shown with light blue backgrounden_US
dc.description.tableofcontentsConvolution layer applies convolution operation to the input images. The feature maps are then introduced to non-linearity with activation functionen_US
dc.description.tableofcontentsVisualization of max pooling operation on with a 4 × 4 image with 2 × 2 window size and strides of 2en_US
dc.description.tableofcontentsFully connected layers learn the relations between the high-level patternsen_US
dc.description.tableofcontentsSingle neuron in Fully Connected Layeren_US
dc.description.tableofcontentsNeurons dropped out with dropout regularization technique do not receive or transmit signal. Dropout with probability p = 0.5 is applied to the neuronsen_US
dc.description.tableofcontentsOutput Layer in a CNNen_US
dc.description.tableofcontentsThe proposed unsupervised backpropagationless filter extraction method. Images/feature maps are converted to candidates from which the filters are discovered without label information. Any filter candidate ���� can become a new filter for the current layer if the maximum similarity value is less than a preset threshold. If not, filter with the highest similarity’s weights is updateden_US
dc.description.tableofcontentsFirst layer filters of Model A trained with MNIST dataseten_US
dc.description.tableofcontentsFirst layer filters of Model B trained with Kuzushiji-MNIST dataseten_US
dc.description.tableofcontentsFirst layer filters of Model A trained with EMNIST-Digits dataseten_US
dc.description.tableofcontentsFirst layer filters of Model B trained with Fashion-MNIST dataseten_US
dc.description.tableofcontentsThe visualization illustrates the collection of 54 features obtained from the MNIST training images using Model type A in the second layeren_US
dc.description.tableofcontentsThe test images belonging to digit class 1 from the MNIST dataset are inaccurately classified by Model A. Among these images, the second, fourth, and fifth samples are mistakenly labeled as 6, potentially due to the presence of artifacts and curved elements within the imagesen_US
dc.description.tableofcontentsThe test images belonging to digit class 9 from the MNIST dataset are inaccurately labeled by Model Aen_US
dc.description.tableofcontentsMisclassified images from the digit class 6 in the EMNIST-Digits dataset, as predicted by Model Aen_US
dc.description.tableofcontentsThe test images belonging to digit class 8 from the EMNIST-Digits dataset are inaccurately labeled by Model Aen_US
dc.description.tableofcontentsMisclassified images from the class 3 in the Kuzushiji-MNIST dataset, as predicted by Model Ben_US
dc.description.tableofcontentsThe test images belonging to digit class 2 from the Kuzushiji-MNIST dataset are inaccurately labeled by Model Ben_US
dc.description.tableofcontentsThe test images belonging to the Bag class that were misclassifieden_US
dc.description.tableofcontentsThe mislabeled test images from the Shirt class, which were incorrectly classified as similar classes by Model Ben_US
dc.description.tableofcontentsBoxplot of the number of filters extracted from MNIST dataset for both layers of Model A with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the number of filters extracted from EMNIST-Digits dataset for both layers of Model A with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the number of filters extracted from Kuzushiji-MNIST dataset for both layers of Model B with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the number of filters extracted from Fashion MNIST dataset for both layers of Model B with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the test accuracy distribution of Model A over 50 runs on MNIST dataset with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the test accuracy distribution of Model A over 50 runs on EMNIST-Digits dataset with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the test accuracy distribution of Model B over 50 runs on Kuzushiji-MNIST dataset with candidate shufflingen_US
dc.description.tableofcontentsBoxplot of the test accuracy distribution of Model B over 50 runs on Fashion MNIST dataset with candidate shufflingen_US
dc.description.tableofcontentsCandidate Set Creation Processen_US
dc.description.tableofcontentsCCNN CoG Based Unsupervised Learning Algorithmen_US
dc.identifier.citationErkoç, T. (2023). Object recognition with competitive convolutional neural networks. İstanbul: Işık Üniversitesi Lisansüstü Eğitim Enstitüsü.en_US
dc.identifier.urihttps://hdl.handle.net/11729/5655
dc.institutionauthorErkoç, Tuğbaen_US
dc.institutionauthorid0000-0001-9033-8934
dc.language.isoenen_US
dc.publisherIşık Üniversitesien_US
dc.relation.publicationcategoryTezen_US
dc.rightsinfo:eu-repo/semantics/openAccessen_US
dc.rightsAttribution-NonCommercial-NoDerivs 3.0 United States*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/3.0/us/*
dc.subjectConvolutional neural networksen_US
dc.subjectUnsupervised learningen_US
dc.subjectFeature extractionen_US
dc.subjectEvrişimli sinir ağlarıen_US
dc.subjectDenetimsiz öğrenmeen_US
dc.subjectÖzellik çıkarmaen_US
dc.subject.lccQA76.87 .E75 O25 2023
dc.subject.lcshConvolutional neural networks.en_US
dc.subject.lcshUnsupervised learning.en_US
dc.subject.lcshFeature extraction.en_US
dc.titleObject recognition with competitive convolutional neural networksen_US
dc.title.alternativeRekabetçi evrişimli sinir ağları ile nesne tanımaen_US
dc.typeDoctoral Thesisen_US
dspace.entity.typePublication

Dosyalar

Orijinal paket
Listeleniyor 1 - 1 / 1
Küçük Resim
İsim:
Object_recognition_with_competitive_convolutional_neural_networks.pdf
Boyut:
3.44 MB
Biçim:
Adobe Portable Document Format
Açıklama:
DoctoralThesis
İndir
Lisans paketi
Listeleniyor 1 - 1 / 1
Küçük Resim Yok
İsim:
license.txt
Boyut:
1.44 KB
Biçim:
Item-specific license agreed upon to submission
Açıklama:
İndir

Koleksiyon

Lisansüstü Eğitim Enstitüsü Tez Koleksiyonu