PDF Publication Title:
Text from PDF Page: 072
5 The great success of neural networks is due in part to the simplicity of the backpropa- gation algorithm, which allows one to efficiently compute the gradient of any loss func- tion defined as a composition of differentiable functions. This simplicity has allowed re- searchers to search in the space of architectures for those that are both highly expressive and conducive to optimization; yielding, for example, convolutional neural networks in vision [LeC+98] and LSTMs for sequence data [HS97]. However, the backpropagation algorithm is only sufficient when the loss function is a deterministic, differentiable func- tion of the parameter vector. A rich class of problems arising throughout machine learning requires optimizing loss functions that involve an expectation over random variables. Two broad categories of these problems are (1) likelihood maximization in probabilistic models with latent vari- ables [Nea90; NH98], and (2) policy gradients in reinforcement learning [Gly90; Sut+99; Wil92]. Combining ideas from from those two perennial topics, recent models of atten- tion [Mni+14] and memory [ZS15] have used networks that involve a combination of stochastic and deterministic operations. In most of these problems, from probabilistic modeling to reinforcement learning, the loss functions and their gradients are intractable, as they involve either a sum over an exponential number of latent variable configurations, or high-dimensional integrals that have no analytic solution. Prior work (see Section 5.6) has provided problem-specific derivations of Monte-Carlo gradient estimators, however, to our knowledge, no previous work addresses the general case. Section 5.10 recalls several classic and recent techniques in variational inference [MG14; KW13; RMW14] and reinforcement learning [Sut+99; Wie+10; Mni+14], where the loss functions can be straightforwardly described using the formalism of stochastic compu- 64 STOCHASTIC COMPUTATION GRAPHS 5.1 overviewPDF Image | OPTIMIZING EXPECTATIONS: FROM DEEP REINFORCEMENT LEARNING TO STOCHASTIC COMPUTATION GRAPHS
PDF Search Title:
OPTIMIZING EXPECTATIONS: FROM DEEP REINFORCEMENT LEARNING TO STOCHASTIC COMPUTATION GRAPHSOriginal File Name Searched:
thesis-optimizing-deep-learning.pdfDIY PDF Search: Google It | Yahoo | Bing
Cruise Ship Reviews | Luxury Resort | Jet | Yacht | and Travel Tech More Info
Cruising Review Topics and Articles More Info
Software based on Filemaker for the travel industry More Info
The Burgenstock Resort: Reviews on CruisingReview website... More Info
Resort Reviews: World Class resorts... More Info
The Riffelalp Resort: Reviews on CruisingReview website... More Info
| CONTACT TEL: 608-238-6001 Email: greg@cruisingreview.com | RSS | AMP |