预览加载中，请您耐心等待几秒...

深度学习的随机矩阵理论模型v.pptx

预览
1/10
2/10
3/10
4/10
5/10
6/10
7/10
8/10
9/10
10/10

亲，该文档总共67页，到这已经超出免费预览范围，如果喜欢就直接下载吧～

下载提示 文本预览

如果您无法下载资料，请参考说明：

1、部分资料下载需要金币，请确保您的账户上有足够的金币

2、已购买过的文档，再次下载不重复扣费

3、资料包下载后请先用软件解压，在使用对应软件打开

深度学习的随机矩阵理论模型神经网络将许多单一的神经元连接在一起 一个神经元的输出作为另一个神经元的输入 多层神经网络模型可以理解为多个非线性函数“嵌套” 多层神经网络层数可以无限叠加 具有无限建模能力,可以拟合任意函数 Sigmoid Tanh Rectifiedlinearunits(ReLU) 层数逐年增加Featuresarelearnedratherthanhand-crafted Morelayerscapturemoreinvariances Moredatatotraindeepernetworks Morecomputing(GPUs) Betterregularization:Dropout Newnonlinearities Maxpooling,Rectifiedlinearunits(ReLU) TheoreticalunderstandingofdeepnetworksremainsshallowExperimentalNeuroscienceuncovered: NeuralarchitectureofRetina/LGN/V1/V2/V3/etc Existenceofneuronswithweightsandactivationfunctions(simplecells) Poolingneurons(complexcells) AllthesefeaturesaresomehowpresentinDeepLearningsystemsOlshausenandFielddemonstratedthatreceptivefieldslearnedfromimagepatches. OlshausenandFieldshowedthatoptimizationprocesscandrivelearningimagerepresentations. Olshausen-Fieldrepresentationsbearstrongresemblancetodefinedmathematicalobjectsfromharmonicanalysiswavelets,ridgelets,curvelets. Harmonicanalysis:longhistoryofdevelopingoptimalrepresentationsviaoptimization Researchin1990's:Waveletsetcareoptimalsparsifyingtransformsforcertainclassesofimages Classpredictionrulecanbeviewedasfunctionf(x)ofhigh-dimensionalargument CurseofDimensionality Traditionaltheoreticalobstacletohigh-dimensionalapproximation FunctionsofhighdimensionalxcanwiggleintoomanydimensionstobelearnedfromfinitedatasetsApproximationtheory Perceptronsandmultilayerfeedforwardnetworksareuniversalapproximators:Cybenko’89,Hornik’89,Hornik’91,Barron‘93 Optimizationtheory Nospuriouslocaloptimaforlinearnetworks:Baldi&Hornik’89 Stuckinlocalminima:Brady‘89 Stuckinlocalminima,butconvergenceguaranteesforlinearlyseparabledata:Gori&Tesi‘92 Manifoldofspuriouslocaloptima:Frasconi’97Invariance,stability,andlearningtheory Scatteringnetworks:Bruna’11,Bruna’13,Mallat’13 DeformationstabilityforLipschitznon-linearities:Wiatowski’15 Distanceandmargin-preservingembeddings:Giryes’15,Sokolik‘16 Geometry,generalizationboundsanddepthefficiency:Montufar’15,Neyshabur’15,Shashua’14’15’16 ……Optimi