Learning with huge memory
Recently a set of papers were published about "memorization" in neural networks. For example:Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts LayeralsoUnderstanding deep...
View ArticleWhen information is already lost
In speech recognition we frequently deal with noisy or simply corrupted recordings. For example, in call center recordings you still get error rates like 50% or 60% even with the best algorithms....
View ArticleGoodbye Google+
Dear friends, as you know Google+ is shutting down. I considered several alternatives: Facebook, Quora, Linkedin, my old blog, Reddit, Twitter, Telegram. Unfortunately there are things I dislike in all...
View ArticleThe theory of possibilities
I've got quite interested in the future prediction these days, one nice idea by Russian writer Sergey Borisovich Pereslegin is that we should build the future based on the theory of possibilities...
View ArticleThe masking problem - capsules, specaug, bert
An important issue with a modern neural networks is their vulnerability to the masked corruption, that is the random corruption of some small amount of samples in the image or sound. It is well known...
View ArticleInformation flows of the future
It is interesting how similar ideas raise here and there in seemingly unrelated context. The recent quote from Actionable Book Summary: The Inevitable by Kevin Kelly:And what’s next probably looks like...
View ArticleSelected Papers Interspeech 2019 Monday
Overall, it is going pretty good. Many very good papers, diarization joins with decoding, everything goes to the right direction.RadioTalk: a large-scale corpus of talk radio transcripts Doug Beeferman...
View ArticleSelected Papers Interspeech 2019 Tuesday
Spatial and Spectral Fingerprint in The Brain: Speaker Identification from Single Trial MEG Signals Oral; 1000–1020Debadatta Dash (The University of Texas at Dallas), Paul Ferrari (University of Texas...
View ArticleSelected Papers Interspeech 2019 Wednesday
A Highly Efficient Distributed Deep Learning System for Automatic Speech RecognitionWei Zhang, Xiaodong Cui, Ulrich Finkler, George Saon, Abdullah Kayi, Alper Buyuktosunoglu, Brian Kingsbury, David...
View ArticleSpectre and deep learning
I noticed a big slowdown in RELU layer performance recently, essentially the RELU operation can now take up to 10% in the total CPU count. This is with kernel 4.15. On older machines everything is just...
View Article