Title:

A Generalized Learning Approach to Deep Neural Networks, Journal of Telecommunications and Information Technology, 2024, nr 3

Group publication title:

2024, nr 3, JTIT-artykuły

Creator:

Ponti, Francesca ; Frezza, Fabrizio ; Simeoni, Patrizio ; Parisi, Raffaele

Subject and Keywords:

deep neural networks ; machine learning ; optimization

Abstrakt:

Optimization of machine learning architectures is essential in determining the efficacy and the applicability of any neural architecture to real world problems. In this work a generalized Newton's method (GNM) is presented as a powerful approach to learning in deep neural networks (DNN). This technique was compared to two popular approaches, namely the stochastic gradient descent (SGD) and the Adam algorithm, in two popular classification tasks. The performance of the proposed approach confirmed it as an attractive alternative to state-of-the-art first order solutions. Due to the good results presented in the case of shallow DNN, in the last part of the article an hybrid optimization Method is presented. This method consists in combining two optimization algorithms, i.e. GNM and Adam or GNM and SGD, during the training phase within the layers of the neural network. This configuration aims to benefit from the strengths of both first- and second-order algorithms. In this case a convolutional neural network is considered and its parameters are updated with a different optimization algorithm. Also in this case, the hybrid approach returns the best performance with respect to the first order algorithms

Number:

3

Publisher:

National Institute of Telecommunications

Date:

2024, nr 3

Resource Type:

artykuł

Resource Identifier:

ISSN 1509-4553, on-line: ISSN 1899-8852

DOI:

10.26636/jtit.2024.3.1454

ISSN:

1509-4553

eISSN:

1899-8852

Source:

Journal of Telecommunications and Information Technology

Language:

ang

License:

CC BY 4.0

rights owner:

Instytut Łączności - Państwowy Instytut Badawczy