Curriculum Learning
Publisher
Metadata
Show full item recordAbstract
Humans and animals learn much better when
the examples are not randomly presented but
organized in a meaningful order which illustrates
gradually more concepts, and gradually
more complex ones. Here, we formalize
such training strategies in the context
of machine learning, and call them “curriculum
learning”. In the context of recent research
studying the difficulty of training in
the presence of non-convex training criteria
(for deep deterministic and stochastic neural
networks), we explore curriculum learning
in various set-ups. The experiments show
that significant improvements in generalization
can be achieved. We hypothesize that
curriculum learning has both an effect on the
speed of convergence of the training process
to a minimum and, in the case of non-convex
criteria, on the quality of the local minima
obtained: curriculum learning can be seen
as a particular form of continuation method
(a general strategy for global optimization of
non-convex functions).