SciML

It compares quite favorably. The nice thing is that most of Numpy and Octave functionality is already builtin to Julia (multidimensional matrices are first class as they are in Matlab/Octave). Julia performance is quite good and multiple dispatch is great for composing packages (once you get used to it). Check out SciML, Julia's answer to scipy (https://sciml.ai/) and Flux Julia's answer to PyTorch/Tensorflow (https://fluxml.ai/). Are these projects as mature as their Python counterparts? To be honest, no they're not, but they are making pretty rapid progress.

>some ODE support Best ODE/SDE etc support of any programming language. Including composing them with machine learning and AD, GPU, multithreading, DSLs and symbolic simplification etc https://sciml.ai/

MIT Julia Lab | julia.mit.edu | Boston (remote possible) | Devops and MLops The Julia Lab is the lab in MIT CSAIL that created Julia. Its work is on high-performance computation and scientific machine learning (https://sciml.ai/). We are looking for individuals to help the lab's development activities. We’re looking for a new devops and MLops engineer to work with the Julia Lab. You’d be in charge of keeping all of our systems running and creating new schemes to accelerate Julia development. Julia’s Base CI, the FluxML, SciML and GPU resources are managed by this position. Automated benchmarking tools 10, automating the installation of Julia in R and Python packages, helping improve Julia’s setup for compute clusters, building tools to make it easy to manage ML training in Git repositories, etc. are the kinds of problems that are encountered. If you’re also interested in technical computing and interested in splitting time between ops and work in this domain, that can also be arranged. This is more of an entry level position, great for students who have just graduated (or are graduating soon), and great for individuals looking to have a year devoted to getting a good PhD program application together. Can be remote, but working in Boston at MIT is preferred. Contact me via the Julia Discourse for more details.

As someone who uses TF/PyTorch/JAX and numba for their day-to-day DL work, I've been watching Julia's post-hype developments in this area intently. Julia has probably the most vibrant ML ecosystem of all the non-Python languages (remind me when someone has implemented a transformers library for Arraymancer [1]), and absolutely smokes Python in certain niches (e.g. neural ODEs and more under the SciML [2] umbrella, I'd argue Python is "lagging hard" there). Numba can work for speeding up simple operations, but is rife with limitations that prevent optimizing more idiomatic Python code and is an absolute pain to debug. Likewise, trying to decipher or even catch errors from deep in the C++ bowels of most modern DL frameworks is an exercise of frustration. Python may still be the best language for non-cutting edge deep learning, but we ought to consider whether we've overfit our algorithms to the limitations of the current system [3]. [1] https://github.com/mratsim/Arraymancer [2] https://sciml.ai/ [3] https://news.ycombinator.com/item?id=20301619

Martijn Visser and Julian Hofer explain how Deltares is using Julia and SciML[1] to develop Ribasim, the next-gen water resources software to be used in the Dutch National Hydrological Model. [1] https://sciml.ai

There are many native packages and wrapper scaffold libraries like Numpy. It will depend on your problem domain, but Julia often transparently supports its broadcast operator on most core data-structure math ("using LinearAlgebra" and the like). https://gist.github.com/AlexanderFabisch/6343090 (small feature comparison list, but ignore the incorrect opinions) https://sciml.ai/ https://fluxml.ai/Flux.jl/stable/gpu/ https://alan-turing-institute.github.io/MLJ.jl/dev/about_mlj/

To respond to both the parent question, and this comment: indeed, this is black-box optimal control in essence. However, this method is just one small aspect of the SciML [0] ecosystem now. The article is a little outdated in that sense. Once obtaining your NN control parameter, it's now possible to use Sparse Identification of Nonlinear Dynamics (SINDy) on that parameter to recover equations of motion governing it [1]. The real promise of these methods is to use the universal approximator power of NNs to get around the 'curse of dimensionality' & uncover presently unknown representations of motion within any system. Take a look at [2] for a more detailed description. [0]: https://sciml.ai/ [1]: https://datadriven.sciml.ai/dev/sparse_identification/sindy/ [2]: https://arxiv.org/abs/2001.04385

I think autograd and its suffusion into the modern tools for machine learning development is initiating a paradigm shift. If you haven't already, check out the SciML ecosystem in Julia. One can leverage it to build machine learning models that can slot right into any existing simulator, dynamical system, or any other method to represent systems to fit to data, and the gradients will be handled appropriately through the entire language natively. This video is from a talk at JuliaCon 2020 and is about the SciML ecosystem: https://www.youtube.com/watch?v=QwVO0Xh2Hbg For more info, check out https://sciml.ai/ I am not affiliated with them, but really admire their work and think it will become an invaluable tool in the coming years as the limits of neural networks become apparent and we stop running out of little tricks to squeeze one more SOTA paper out of them.

I'm studying in the intersection of physics and data science, and I think there's a number of places where physics can benefit from ML. From my current point of view though, most of these applications lie more on the experimental/computational sides of physics rather than the theoretical side. One of the current use cases is using ML to aid in the processing and analysis of data obtained from experiments. I would like to see more truly innovative work done on the theoretical side, but I don't think we'll see "AI" bridge the gap between QFT and GR any time soon. I think in order for something like that to happen we need a new approach, as the current approach of throwing deep learning models at it doesn't feel like the right answer. On a more general note, the SciML organization [1] has been quite successful in helping incorporating more ML into science. [1] https://sciml.ai/

Have you explored the SciML landscape at all (?): https://sciml.ai/ There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties. Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy. There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example). Long story short -- there's quite a lot of work going on. It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.

The unique property is the ability to just pick any code or library that is unaware of the differentiation library (unlike tf.function as it needs to specifically use tf methods) and get the gradient. In a language like Julia this is immediately useful as it has a massive ecosystem of numerical code that make sense to get gradients (like differential equations and the SciML project [1], or less conventional stuff like raytracers), but in a language like Swift (as there is no meaning to gradient of GUI libraries or frontend stuff) it is more of a "if you build they'll come" faith from Google. But regardless unique features, it's a have cake and eat it too type of interface. You don't need to learn a second language within the language like tensorflow's tf.* making it even more natural and flexiblethan pytorch, including all debug mechanisms of the host language itself, but you still get compile time graph creation like tensorflow, including all kinds of optimizations. It makes other approaches seem primitive by comparison, but creating it is much more complex, and the main audience is already more than used to using language within language solutions (like numpy) which can provide something almost as good even if less elegantly, so it's not easy to convince people as well (when it involves changing programming languages). [1] https://sciml.ai/

This is true, but even engineers see the advantages of Julia. My engineering school has went from almost pure Matlab usage to many key engineering courses switching to Julia due to its simplicity and friendliness. It's also SOTA for many engineering applications, particularly for acausal modelling and scientific machine learning (see https://sciml.ai), which has led to big companies like Pfizer adopting it [1]. And for engineers writing novel libraries, it clearly has a strong edge. See for example the work by NASA's JPL [2, 3], the FAA [4] or the CliMa project [5]. [1]: https://juliahub.com/case-studies/pfizer/ (see also https://info.juliahub.com/case-studies) [2]: https://exoplanets.nasa.gov/news/1669/seven-rocky-trappist-1-planets-m… [3]: https://ntrs.nasa.gov/citations/20170008266 [4]: https://youtu.be/19zm1Fn0S9M [5]: https://clima.caltech.edu/

AoT compiled binaries and libraries are fundamental building blocks for distributing software and running it quickly without latency. Julia itself calls AoT binaries, for example when calling Linear Algebra functions [1]. Chris Rackauckas's team had to leverage AoT compilation (i.e. system images) to reduce TTFX on DifferentialEquations.jl [2]. PyTorch and Tensorflow run without TTFX because they use AoT compiled binaries. What's more, most end users don't have to compile anything on their computer--they can simply download the binaries and libraries. For small scripts, yes you can simply distribute the package and have the user recompile on the fly. But imagine if your operating system had to recompile every time you turned the computer off. Or if an autonomous vehicle had to recompile an unseen patch of code while in motion. [1] https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#Standard-functi… [2] https://sciml.ai/news/2022/09/21/compile_time/