Cosma Shalizi

For information theory, a really recommend: Thomas M. Cover, Joy A. Thomas "Elements of Information Theory" - it starts elementarily, has direct examples, straightforward definitions and readable notation. (And first recommendation by Cosma Shalizi: http://vserver1.cscs.lsa.umich.edu/~crshalizi/notabene/information-the…

My biggest issue with Jeremey England is that he's basically rehashing Ilya Prigogine's ideas. Read Prigogine's books "order out of chaos", " Exploring complexity: An introduction" (best one), and less so "The End of Certainty". Prigogine won the Nobel prize for his "discovery" of dissipative structures, and he was a driving force in the ideas behind complexity theory. Wolfram's early work was full of references to Prigogine, but then Wolfram decided to write him out of history. As did England above. Progigine was derided for his ideas: http://vserver1.cscs.lsa.umich.edu/~crshalizi/notabene/prigogine.html I think the ideas are great, but we have some work to do.

The problem with covering claims of changing paradigms is: who is qualified to evaluate such ambitious claims? Realistically, not many folks. I think the best thing for a reporter to do is to connect the appropriate experts together to build a balanced story. Yes, spend time with Wolfram, take good notes etc.--but then find the right experts to respond to and evaluate what he said. To me, this story reads a bit too credulous to be really useful in understanding what Wolfram is claiming. I read Wolfram's blog post already, and this story seems about the same in tone and content. What I want to see is the equivalant of this [1] article: a cranky review by an expert in the field...with citations. [1] http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

I concur. His notebook is a wonder of great resources on many (many many) subjects, with often interesting comments. http://vserver1.cscs.lsa.umich.edu/~crshalizi/notabene/

If important means talking in confusing ways about simple concepts. Noam Chomsky said it best in his essay about Post Modernism: http://vserver1.cscs.lsa.umich.edu/~crshalizi/chomsky-on-postmodernism… This quote contains his criticism very clearly: "As for the "deconstruction" that is carried out (also mentioned in the debate), I can't comment, because most of it seems to me gibberish. But if this is just another sign of my incapacity to recognize profundities, the course to follow is clear: just restate the results to me in plain words that I can understand, and show why they are different from, or better than, what others had been doing long before and and have continued to do since without three-syllable words, incoherent sentences, inflated rhetoric that (to me, at least) is largely meaningless, etc. That will cure my deficiencies --- of course, if they are curable; maybe they aren't, a possibility to which I'll return."

tl;dr - a refutation of statistician Cosma Shalizi’s essay at http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/523.html in three parts: (1) "Shalizi’s first error is his assertion that cognitive tests correlate with each other because IQ test makers exclude tests that do not fit the positive [correlation matrix]. In fact, more or less the opposite is true. ... Cognitive tests correlate because all of them truly share one or more sources of variance." (2) "Shalizi’s second error is to disregard the large body of evidence that has been presented in support of g as a unidimensional scale of human psychological differences. The g factor is not just about the positive [correlation matrix]. A broad network of findings related to both social and biological variables indicates that people do in fact vary, both phenotypically and genetically, along this continuum that can be revealed by psychometric tests of intelligence and that has has widespread significance in human affairs." (3) "Shalizi’s third error is to think that were it shown that g is not a unitary variable neurobiologically, it would refute the concept of g. However, for most purposes, brain physiology is not the most relevant level of analysis of human intelligence. What matters is that g is a remarkably powerful and robust variable that has great explanatory force in understanding human behavior. Thus g exists at the behavioral level regardless of what its neurobiological underpinnings are like." Conclusion:…

I hate to be that guy, but I wish to share my opinion about closed source mathematical software. There is no doubt that what Wolfram Research has done with Mathematica is amazing and tempting. It is a very complete and uniform engine, and can be very useful for very different kinds of mathematics. However, Wolfram Research deliberately keeps their methods and source code closed. Note that this is more serious than just the "Stallman-esque" open-source-everything philosophy. Wolfram insists that users do not need to know implementation details of their methods. This is plainly in their documentation. You can see the uncompelling argument from Wolfram here [6]. The gist of the argument is that interfaces matter, not implementations. I strongly argue that users, especially mathematicians and engineers, should care about the internals of mathematical software, especially when it's being used, even in a utilitarian fashion, for research and engineering. Not only this, but Wolfram has litigated against his own employees for publishing mathematical proofs about cellular automata. Information about this lawsuit is sparse, but evidence of it can be seen in [0]. More information can be found here [1]. Unfortunately most responses to the above from users of Mathematica is "well I just use Mathematica as a calculator, nothing serious" or "I wouldn't look at the source code anyway, so what gives?" It's an unfortunate response, and I don't have a technical rebuttal, but a moral one,…

An important discovery, but equally important to keep it in proper context: "If you want to get a rough grasp of how the leopard might get its spots, then building a CA model (or something similar) can be very illuminating. It will not tell you whether that's actually how it works. This is an important example, because there is a classic theory of biological pattern formation, or morphogenesis, first formulated by Turing in the 1950s, which lends itself very easily to modeling in CAs, and with a little fine-tuning produces things which look like animal coats, butterfly wings, etc., etc. The problem is that there is absolutely no reason to think that's how those patterns actually form; no one has identified even a single pair of Turing morphogens, despite decades of searching. For a good account of the actual mechanisms of biological pattern formation, as they are being revealed by molecular developmental biology, see John Gerhart and Marc Kirschner, Cells, Embryos and Evolution: Toward a Cellular and Developmental Understanding of Phenotypic Variation and Evolutionary Adaptability (Oxford: Blackwell Scientific, 1997). This book provides an excellent discussion of the interactions between genetic control, self-organization, evolutionary forces and functional adpatations in development. (There is a nice review by Danny Yee.) — The failure of experiment to turn up any biological system working according to Turing's principles was remarked by the late, great John Maynard Smith…

For those curious for more details about "He scores high on the crackpot index, but not high enough to be completely discredited as a lunatic. His litigious nature and ability to wrangle the legal system in order to label everything "mine" is what keeps him in business.", see this review of "A New Kind of Science", which begins: > Attention conservation notice: Once, I was one of the authors of a paper on cellular automata. Lawyers for Wolfram Research Inc. threatened to sue me, my co-authors and our employer, because one of our citations referred to a certain mathematical proof, and they claimed the existence of this proof was a trade secret of Wolfram Research. I am sorry to say that our employer knuckled under, and so did we, and we replaced that version of the paper with another, without the offending citation. I think my judgments on Wolfram and his works are accurate, but they're not disinterested. > With that out of the way: it is my considered, professional opinion that A New Kind of Science shows that Wolfram has become a crank in the classic mold, which is a shame, since he's a really bright man, and once upon a time did some good math, even if he has always been arrogant. http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

Cosma Shalizi on this very good: http://vserver1.cscs.lsa.umich.edu/~crshalizi/bulletin/homo-reciprocan…

In other settings, a simple approximation of what Evolution does will balance your portfolio and find Nash Equilibriums for zero sum games much more quickly than linear programming (query: multiplicative update evolution). Having to wait for things to die can be restrictive (but genomes are declarative, what does that buy?). Roughly, with what amounts to the assumption that Lamarckian evolution is more or less not true, you can characterize evolution's learning power as almost as good as PAC (query: statistical query learning evolution). See alsos: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/601.html http://arxiv.org/abs/1211.4037: Natural selection. V. How to read the fundamental equations of evolutionary change in terms of information theory

I liked the beginning where DFW expresses his opinion on the purpose of literature, how it's fundamentally about becoming human and challenges presented by modern society. I also liked how he talked about the effect of television, which sort of jived with Amusing Ourselves to Death, which claims there are limits set by different mediums in the complexity of ideas pertaining to politics that can be presented and a decline in quality of public discourse. Then about 1/3 through the topic shifts to postmodernism, which for the reasons pointed out by Noam Chomsky I found difficult to take seriously [1]. There's even a part where DFW commends (perhaps in a tongue-in-cheek manner) the interviewer for having "a gift for lit-speak". [1] http://vserver1.cscs.lsa.umich.edu/~crshalizi/chomsky-on-postmodernism…

Here's something you might find interesting: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.html There have are also loads of replication projects, some currently ongoing, of both experimental and statistical papers, with generally dismal results.

A Rare Blend of Monster Raving Egomania and Utter Batshit Insanity: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/915.html By Cosma Shalizi

Wow, this is quite stunning to read. He actually baldly states that the only reason for experimenting is to confirm what you already believe, and that you should therefore just do as many experiments as it takes to 'prove' yourself correct, hide evidence of the failures and then stop - forever. This was linked from the article, BTW - and the responses they also linked to are dead on: http://mchankins.wordpress.com/2014/07/12/do-not-replicate http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/1102.html

1. It could be possible that Satoshi Nakaomoto does not exist or Bitcoin does not exist. 2. Therefore no one has invented Bitcoin or someone has invented Bitcoin. 3. Because each part of the proposition is equally likely we have to infer that we don't know whether Bitcoin was invented. Further reading, Betrand Russell's on denoting http://vserver1.cscs.lsa.umich.edu/~crshalizi/Russell/denoting/

perhaps : http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

While you're correct, the fact that someone can define an IQ test as the test that most strongly correlates with the g-factor (itself merely a composite factor from a number of different measures) doesn't mean that IQ, or the g-factor, or "general cognitive ability", corresponds to any physical causative factor: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/523.html

The details of the story are hard to find nowadays, probably because the lawyers told everyone to shut up about it. You can find some vague allusions to it here: " Wolfram quit Illinois, took the program private, and entered into complicated lawsuits with both his former employee and his co-authors (all since settled)." http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/ Btw, which side of his reputation does he live up to? To being a visionary genius or to being a raving crank?

> And 'A New Kind of Science' was a fascinating book. Indeed... A Rare Blend of Monster Raving Egomania and Utter Batshit Insanity: http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

Another good opportunity to re-post this critique of "A New Kind of Science" by Cosma Shalizi. It's not kind. http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

This whole field seems likely to be a case study supporting http://xkcd.com/882/. But the research does have a questionable assumption. On p 5 it says, "Such a result would not likely be due to differences in the methods used to generate g in the various datasets under comparison, since g is consistently measured by a wide variety of tests." This is true. However it is also true that, as http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/523.html points out, that all evidence we have is consistent with g being a correlation between different ways of weighting a large number of independent factors. If so, then it is easy to have two test measuring g that are fairly well correlated with each other, one of which picks up a particular factor and the other does not. If a gene affects that factor, then the test you use to measure to find these correlations will matter.

My impression is that Wolfram came to such conclusions because he is obsessed with cellular automata and with their mathematical universality in discrete domains. Of course, they are useless in continuous domains, so it would be ruinous to his world-view if the universe were continuous. So it's a bit like ancient metaphysics a la Plato and Co., where the properties of the universe are inferred from first principles (which come from... where, exactly?) rather than from any empirical observation. A review of Wolfram's book: http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/wolfram/

Good reading in addition to Pearl himself is via Cosma Shalizi. Ref list: http://vserver1.cscs.lsa.umich.edu/~crshalizi/notebooks/causality.html Chapters: http://www.stat.cmu.edu/~cshalizi/uADA/13/lectures/ch22.pdf http://www.stat.cmu.edu/~cshalizi/uADA/13/lectures/ch23.pdf http://www.stat.cmu.edu/~cshalizi/uADA/13/lectures/ch24.pdf Incidentally, Shalizi is a great source for going back to the basics. His course at CMU "Advanced Data Analysis from an Elementary Point of View" has course notes and exercises available gratis: http://www.stat.cmu.edu/~cshalizi/uADA/13/. And any time I need to get a reading list, the best place to start is usually his 'notebooks' page.

You may want to have a look at Ernest Gellner's "Nations and Nationalism" for more on your first point. http://books.google.com/books/about/Nations_And_Nationalism.html?id=jl… http://www.amazon.com/Nations-Nationalism-Second-Perspectives-Past/dp/… http://vserver1.cscs.lsa.umich.edu/~crshalizi/reviews/nations-and-nati…

Also from Shalizi, if you want a TL;DR http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/491.html

Shalizi is no slouch either - his notebooks are fascinating. http://vserver1.cscs.lsa.umich.edu/~crshalizi/notabene/

I was commenting on the OP. I started reading it but found just 'algorithms' with no indication of anything like any guarantees or qualities for them. Your i) sounds good. E.g., in stochastic dynamic programming with assumptions reasonably expected, e.g., essentially a Markov assumption, we can easily prove that the number crunching yields the decisions with the least possible expected cost from any 'measurable' means of manipulating the data. And once when I argued this, I also included a proof that additional data independent of what we already wouldn't help either. This result is, of course, 'obvious' although obvious is not a proof. The proof turned out to be just a light tap with Fubini's theorem. So, good: Some people actually argue that some of the algorithms actually have some desirable qualities. So, the world is not totally coming apart at the seams or being turned over to naive hackers who only know to watch the code run and then conclude qualitative that it seems 'nice'. On your ii), "some unknown independent stochastic process", that's not so clear. So, for some index set I and random variables X(i) for i in I, the set { X(i) | i in I } is independent? That's something of an unusual and, in practice, not very interesting stochastic process. Maybe some EEs would call that 'white noise'. On your iii), sorry, but essentially everything is 'stochastic'. Even a 'deterministic' process is stochastic, in practice usually is also Markov (past and…

Hmm, what should I get from that debate? I just re-read the transcript [1], and seems they just had a minor disagreement over the notion of justice. (And a couple other little "differences" which we could consider clarifications.) I currently don't see why it's so heavily cited; some others (not you) seem to consider it some legendary heavyweight fight in the history of philosophy. But it was just... an interview. As for his disagreements with certain philosophers... he makes a cogent point, that perhaps he simply doesn't understand it, but much of it seems to him written in a kind of gibberish, which can understandably lead to a lot of mistakes. [2] Imagine if I wrote hopefully useful code, but it was extremely hard to read and poorly documented... What properties would we expect that code to have? [1] http://www.chomsky.info/debates/1971xxxx.htm [2] http://vserver1.cscs.lsa.umich.edu/~crshalizi/chomsky-on-postmodernism…