Artificial neural networks are dominating a vast majority of application scenarios to date, and will surely extend their lead in the near future. Especially, the superior performance of convolutional neural networks (CNNs) for image processing tasks presents a promising use case in innovative and cutting-edge domains. However, their dominance emerges from an ever-increasing memory intensity and computational complexity. In contrast to the increasing resource demand, real-world applications on embedded devices pose major challenges with regard to limited computing power, memory resources and available energy and/or latency budget for the deployment of CNNs in embedded settings. To counteract these challenges, this dissertation presents a tripartite hardware-software co-design paradigm for the efficient application of CNNs on embedded accelerators. This allows the traversal through the design space by either a top-down, meet-in-the-middle or a bottom-up approach. Moreover, six novel optimization methods, on the three levels of abstraction, are presented in this book, which further serve the illustration of the simplified design process. By means of successive exploration and refinement steps, it is shown how more powerful CNN-based applications can be created and make use of orthogonal optimization methods like pruning, quantization and Winograd convolution. Furthermore, the increase in data-level parallelism is achieved by quantized neural networks. In summary, we show that the optimization of CNNs for embedded applications, such as in the field of autonomous driving, can only be achieved through the interaction of the three abstraction levels (using expert knowledge) and synergies of different compression techniques to arrive at a fruitful HW-CNN co-design.