Advanced School for Computing and Imaging (ASCI)

ASCI office
Delft University of Technology
Building 28, room 04.E120
Van Mourik Broekmanweg 6
2628 XE – DELFT, The Netherlands

P: +31 15 27 88032

Visiting hours office
Monday, Tuesday, Thursday: 10:00 – 15:00


The ASCI office is located at the Delft University of Technology campus.  It is easily accessible by bicycle, public transport and car. The numbers of buildings can help you find your way around the campus. Make sure you remember the name and building number of your destination.

Contact us at +31 15 278 8032 or send us an email at

Inductive biases for pixel representation learning

Inductive biases for pixel representation learning

Author : Zenglin Shi
Promotor(s) : Prof.dr. C.G.M. Snoek / Dr. P.S.M. Mettens
University : University of Amsterdam
Year of publication : 2022
Link to repository :


This thesis explores inductive biases in the context of deep learning for pixel-level tasks. An inductive bias allows a learning algorithm to prioritize one solution over another, and to generalize beyond the training data. Inductive biases have been explored for deep learning, and this is what in part contributed to its success. Besides the well-known explicit inductive biases like L1 & L2 regularization, there are also many implicit inductive biases, which are introduced by implicit regularization such as optimization algorithms, dropout, attention mechanism, transfer learning. These implicit inductive biases have been showing remarkable ability to help generalization in deep learning. Different from explicit inductive biases, however, it is hard to explicitly identify and derive a formulation of implicit bias. Also, there is no direct way to control the strength of an implicit bias. As a result, it is difficult to apply an implicit bias in practice. This thesis strives to turn the implicit biases into an explicit form to uncover and exploit them for pixel representation learning. We have uncovered and exploited three implicit biases for different pixel-level tasks, including the spectral bias for the deep image prior, the salience bias for guided filtering, and the attentional bias for tiny object localization and counting. In addition, this thesis also seeks to develop new inductive biases by exploiting prior knowledge. We have developed three inductive biases for best-in-class object counting by discovering new knowledge.