Phd Position F/m Reliable Deep Neural Network

Le descriptif de l’offre ci-dessous est en Anglais_

**Type de contrat **:CDD

**Niveau de diplôme exigé **:Bac + 5 ou équivalent

**Fonction **:Doctorant

**A propos du centre ou de la direction fonctionnelle**:
The Inria Rennes - Bretagne Atlantique Centre is one of Inria's eight centres and has more than thirty research teams. The Inria Center is a major and recognized player in the field of digital sciences. It is at the heart of a rich R&D and innovation ecosystem: highly innovative PMEs, large industrial groups, competitiveness clusters, research and higher education players, laboratories of excellence, technological research institute, etc.

**Contexte et atouts du poste**:
Context & background:

- increasing internet-connected IoTs, and also to alleviate the communication latency, especially for real-time safety-critical decisions, e.g., in autonomous driving.
References:

- [1] Y. LeCun, et al., “Deep learning,” Nature, vol. 521, no. 7553, pp. 436-444, May 2015, doi: 10.1038/nature14539.
[2] B. Moons, et al, “14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy
- frequency
- scalable Convolutional Neural Network processor in 28nm FDSOI,” in IEEE ISSCC, 2017.
[3] C. Torres-Huitzil and B. Girau, “Fault and Error Tolerance in Neural Networks: A Review,” IEEE Access, 2017. [4] A. Lotfi et al., “Resiliency of automotive object detection networks on GPU architectures,” in IEEE ITC, 2019 [5] A. Ruospo, et al., “Investigating data representation for efficient and reliable Convolutional Neural Networks,” in Microprocessors and Microsystems, 2020
[6] F. K. Dosilovic, et al, “Explainable artificial intelligence: A survey,” in MIPRO, 2018.
[7] N. Srivastava et al., “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, 2014.

**Mission confiée**:
Ph.D. thesis goal:

- cost selective fault-tolerance strategies; (ii) designing selective fault-tolerance approaches for DNN HW accelerators by using the analysis provided by the fault injection method. The reliability improvements obtained with the above-described methodology will be measured and a design space exploration will be carried out to obtain different DNN HW accelerator implementations providing different trade-offs between fault tolerance and energy efficiency.

**Principales activités**:
More in detail, the Ph.D. student will design and develop a methodology to perform large-scale fault analysis on state-of-the-art DNN hardware architectures. The fault analysis will determine the set of malignant HW faults that mostly impact the accuracy of classification (or other DNN objectives, such as image segmentation) during the inference phase.

DNN inference is known to be very complex, especially on large models and for large datasets. Moreover, in order to obtain statistically relevant metrics about the fault impacts, a large number of faults have to be injected. Since the complexity of fault injection grows linearly with DNN inference complexity and the number of injected faults, the biggest challenge of this task will be to reduce such complexity by proposing statistical or analytical methods to prune the fault space. A first approach is profiling DNN hyper-parameters (e.g. distributions of weight values, neuron activations, arithmetic kernel computations) to determine their sensitivity to the final DNN accuracy. In general, a divide and conquer approach will be designed to reduce the fault injection complexity. “Local” (i.e. at layer/kernel levels) fault injections will be performed, to avoid running the full inference. Then, solid and efficient approaches to link the local sensitivity the global accuracy will be designed and developed.

This will allow realizing an optimized accelerated fault injector framework, enabling large-scale fault simulations. In order to further push the performances and reduce injection time, direct execution of the network (or a portion) on FPGA accelerators can also be leveraged. The enhanced fault-injection framework will allow performing several reliability assessments, such as: (a) training a faulty network to find the fault density beyond which the learning capacity starts degrading; (b) performing inference on a faulty network to be able to identify the set of malignant faults; (c) fault injection during training for passive fault tolerance; (d) fault injection attacks during the inference to evaluate the security fault tolerance can offer.

Finally, error correction mechanisms will be designed, e.g. selective low-precision triplication using checkers/voters, most-significant bits reinforcement, standby sparing and correcting codes; alternatively, if a detection mechanism is available, the re-execution of the task of a faulty component or the dynamic rescheduling/mapping of the DNN to the HW-AI (bypassing faulty components) are viable options. A design space exploration will help