Today’s neural networks are even hungrier for data and power.
Training them requires carefully tuning the values of millions or even
billions of parameters that characterize these networks, representing
the strengths of the connections between artificial neurons. The goal is
to find nearly ideal values for them, a process known as optimization,
but training the networks to reach this point isn’t easy. “Training
could take days, weeks or even months,” said Petar Veličković, a staff research scientist at DeepMind in London.
That may soon change. Boris Knyazev
of the University of Guelph in Ontario and his colleagues have designed
and trained a “hypernetwork” — a kind of overlord of other neural
networks — that could speed up the training process. Given a new,
untrained deep neural network designed for some task, the hypernetwork
predicts the parameters for the new network in fractions of a second,
and in theory could make training unnecessary. Because the hypernetwork
learns the extremely complex patterns in the designs of deep neural
networks, the work may also have deeper theoretical implications.
https://www.quantamagazine.org/researchers-build-ai-that-builds-ai-20220125/
Neuroverkot ovat oppineet rakentamaan uusia neuroverkkoja.