TensorFlow Blog

I took the recent fastai course that had two lessons about S4TF taught by Chris Lattner and walked away excited. It’s not ready for prime time yet but the roadmap looks great. Building auto-differentiation into the language and having an easy way for anyone to experiment with core architecture changes (without killing performance) will be a huge win for everyone. I expect lots of prototyping will be done on S4TF (because it will be easy for developers to keep things on the GPU with XLA and MLIR) and then the things that work will be back-ported to other ecosystems. https://medium.com/tensorflow/mlir-a-new-intermediate-representation-a…

Edward http://edwardlib.org/ An amazing Python library built atop Keras and Tensorflow for composing and training probabilistic models in just a few lines. If you're interested in machine learning and/or Bayesian statistics and modeling, you owe it to yourself to check this library out. The webpage has several different tutorials that are all super interesting. I don't know too much, but I believe this library will be superseded by Tensorflow Probability (Dustin Tran, who writes & maintains Edward is also on the TF Probability team). https://medium.com/tensorflow/introducing-tensorflow-probability-dca4c… Either one of these are super super valuable if you're looking to play around with and invent new ML algorithms, since you can do in 5 lines what would have taken you 1,000 lines and 4 days of debugging to do without these libraries.

For anyone wondering it is Electron app, that packs some heavy web app assets(300mb + 150mb runtime), but still loads rest of web app from https://client.getposturehealth.co, and then it uses google pretrained posenet tensorflowjs model[0][1]. (loads it from https://storage.googleapis.com/tfjs-models/savedmodel/posenet/mobilene… In perfect world one would made it offline, release it at fixed price and be done with it, in our imperfect world it is turned in infinite cash-cow that requires connectivity and bazillion metrics. [0] https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-b… [1] https://github.com/tensorflow/tfjs-models/blob/master/posenet/README.m…

There's a bunch of apps now sort of like this for golf: swing, posture, ball flight, etc. Given something like this old post about tensorflow[1], you'd think this isn't even the worst hobby project if you can get some footage that a model likes. I've never really thought to try running NBA clips through it... [1]https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-b…

Same thoughts here. I ran Thriller, Alligator by Of Monsters and Men, and In Hell I'll be in Good Company by The Dead South on the 2 / 5 / 4 stems, respectively. Impressive results. Definitely agree that some of the middle frequencies show some error. It would be really cool to create "music mappers"/life sounds tracks like what you can do with pictures & art styles (e.g. https://medium.com/tensorflow/neural-style-transfer-creating-art-with-…

Blog post - https://medium.com/tensorflow/tensorflow-2-0-is-now-available-57d706c2…

The memory constraint is certainly an issue, but I think they can be overcome with some software hacks (for a speed penalty, of course.) For example, gradient checkpointing and gradient accumulation might help: https://medium.com/tensorflow/fitting-larger-networks-into-memory-583e… https://medium.com/huggingface/training-larger-batches-practical-tips-…

I was really looking forward to probabilistic programming but then I saw this project is based on Julia. Not to keen on leaning Julia as I already know Python/C++/Java/Scala/R. Wish it was based on c++/python to at least get started with. Please let me know if there are other interesting probabilistic programming frameworks available or tutorials to get started with. Edit: https://medium.com/tensorflow/an-introduction-to-probabilistic-program… There is support in tensorflow for probabilistic programming. How is this any different?

What do you think of Keras in this space ? Because TF 2.0 is entirely keras based. https://medium.com/tensorflow/standardizing-on-keras-guidance-on-high-…

Check out federated learning. https://medium.com/tensorflow/introducing-tensorflow-federated-a4147aa…

Tensorflow support on AMD is supposedly not awful anymore. https://medium.com/tensorflow/amd-rocm-gpu-support-for-tensorflow-33c7… I wish someone who had >actual< practical experience in the domain would comment. It's an important question.

Also https://medium.com/tensorflow/test-drive-tensorflow-2-0-alpha-b6dd1e52…

If you're interested in upgrading your models, also make sure to check out this Medium post: https://medium.com/tensorflow/upgrading-your-code-to-tensorflow-2-0-f7…

Are the changes between ~1.13 and 2.0 finalized yet? Last I checked they were still in flux, outside of a few blog posts which state that config defaults changed: https://medium.com/tensorflow/whats-coming-in-tensorflow-2-0-d3663832e… Biggest change for me is it seems like there's more of an incentive to use tf.keras instead of external Keras, for simplicity.

This blog post gives a decent overview: https://medium.com/tensorflow/whats-coming-in-tensorflow-2-0-d3663832e…

So much bashing on static typing on deep learning:) Does any one from Google can explain the benefit since you guys are working on swift in tensorflow https://medium.com/tensorflow/introducing-swift-for-tensorflow-b75722c…

Not long time ago we got Swift for Tensorflow as first class citizen: https://medium.com/tensorflow/introducing-swift-for-tensorflow-b75722c…

Some more info in the TF DevSummit video: https://www.youtube.com/watch?v=vdG7uKQ2eKk and the blog post: https://medium.com/tensorflow/introducing-tensorflow-model-analysis-sc…

I think you're reading too much into it. It was announced at the Developer Summit.[0][1] Seems like a long ways to go for an April Fool's joke. [0]: https://medium.com/tensorflow/highlights-from-tensorflow-developer-sum… [1]: https://www.youtube.com/watch?v=Yze693W4MaU

Even from the paper, it's hard to tell what this library actually does: Section 5 in https://arxiv.org/pdf/1910.02054.pdf The paper talks about parameter partitioning and overlapped communication, but doesn't actually give many details on how those things happen. The library appears to be an implementation of some common algos for solving the 'pebble game,' as explained decently here: https://medium.com/tensorflow/fitting-larger-networks-into-memory-583e… The essential point is that: (1) model parallelism is hard to do and has historically been done manually to scale wide models across GPUs (2) inter-GPU I/O is expensive for vanilla data-parallel jobs (that typically use naive mirroring strategies) (3) researchers have figured out now how to 'compile' a deep model so that layers span GPUs and save on both memory usage and I/O (4) so scaling wide models is still hard, but now we have better tools for deep models Existing all-reduce-based data-parallel problems have already been well-studied (see e.g. https://people.eecs.berkeley.edu/~jfc/papers/14/Kylix.pdf ), so it's really nice to see gains through new techniques. Definitely like seeing this 'compilation' being wrapped up into a library. Just wish they did a better job of communicating key ideas.

> breakthrough innovation in programming which will enable programmers to deliver ambitious web/mobile apps at a very high speed. That's a pretty narrow framing. 10 years from now, a lot of programming won't look like traditional application development at all. For example, a self-driving car or other machine with a similar level of automation is a valuable product that requires a lot of programming but doesn't fit into the category of "web/moble app." Or consider the development of natural language interfaces: right now this is at the level of Alexa skills, but over the next 10 years we will see frameworks that allow programmers to specify not just a few phrases, but mini-languages for entire domains: pilots talking to airplanes, doctors talking to EMRs, etc. Use cases like this suggest that frameworks similar to PyTorch/Tensorflow will become increasingly important. These frameworks have features (particularly automatic differentiation, very sophisticated yet easy-to-use SGD optimizers like Adam, and built-in GPU support), which hugely increase productivity and lower the barrier to entry compared to the tools of 10 years ago yet they still have tons of rough edges. Its easy to imagine that 10 years from now they'll be even more powerful and easier to use; for example I'm pretty excited about the future possibilities of tools like [AutoGraph][AG]. [AG]: https://medium.com/tensorflow/autograph-converts-python-into-tensorflo…