R-bloggers

My daily fix: Run by biostatisticians Jeff Leek, Roger Peng, and Rafa Irizarry, Simply Statistics (http://simplystatistics.org/) it's more than stats - and it's a must read. R-bloggers (http://www.r-bloggers.com/) is the hub for everything R.

I follow http://www.r-bloggers.com/ but hopefully I'll be able to contribute something in the future.

Here is my opinion on Python and R. I used Pandas and switched to R. 1) Python 0 based as opposed to 1 based is big to me. I don't want to always have to minus on on columns to call them. 2) I much prefer Hadley Wickham's syntax to Pandas which closely follows it, but R piping with %>% is amazing with library magritr. Using dplyr and stringr, tidyr, rvest, httr, luberdate are awesome and consistant. 3) In R I can use dplyr whcih is fast or even use data.table for even more speed that uses a sql like syntax. In Python your choices are limited in comparison to R for tools. 4) HUGE and the reason I switched to R. I can output my graphics into a editable PowerPoint slide. My managers and company doesn't know that there are anything besides MS Office and I can give them slides that they can fiddle with for their own presentations. 5) RMarkDown and knitr for outputing reports in Word (UGH but ti works great), PDF or , LaTex or HTML is fantastic. Reproducable output is great. If I want I can also use Project Jupyter (IPython's new name in next version) with a R kernel. 6) Tools - I got RStudio installed on my work computers, and my personaly server has RStudio Server installed. This is what I have settled into using because it is the best statistic programing tool out there. Works better than Jupyter/IPython for 90% of the time, though I love Love LOVE Jupyter/IPython. 7) The community. R's community is fantastic [http://www.r-bloggers.com/] just look at all the daily…

I spent about a year blogging about R as I learned it (http://www.r-chart.com/). It is not a general purpose sort of language but is great for math / charting tasks. A couple of ways that software developer types can minimize time to get up and running: 1) Check out http://www.r-bloggers.com/. The site gives you a good sense of the state of R and Tal (who runs the site) has done a great job of promoting R and encouraging the community. 2) Pick one graphics package and stick with it. The standard functions are sufficient but ggplot2 has a more finished appearance. 3) Review available libraries. If you want to do something, someone else likely already has and has posted a package to CRAN. 4) One way for SQL developers to limit the need of learning the idiosyncrasies of the language is to use the sqldf package to manipulate data frames and use ggplot2 (which generally takes data frames as arguments) to display charts.

Time perhaps to post this classic from XKCD: https://www.r-bloggers.com/2018/09/xkcd-curve-fitting-in-r/ To anyone who has worked in research, the various strategies it shows would be tragically familiar.

https://en.wikipedia.org/wiki/Statistical_significance When the Higgs boson was declared "discovered," the particle was found to exist with a five-sigma certainty. This brief article does a great job of explaining what that means: https://www.r-bloggers.com/2012/07/the-higgs-boson-5-sigma-and-the-con… Physicists got to that point (very roughly) by constructing complex mathematical models and building large machines to test those models. In contrast, leading psychology models change with the seasons, and large-scale human experiments cannot be (ethically) conducted with the control as seen in physics. Because psychology experiments have substantial environmental variability, drawing robust conclusions in psychology is exceedingly difficult. This fact should mean that very-high-certainty findings in psychology studies must be subjected to the highest scrutiny.

I would recommend looking into Tol-palettes. Some of these are optimized for color-blindness. I use them regularly and have small utility functions to generate some palettes for n-different colors. That way you have a maximized contrast and I think the palettes look aesthetically pleasing. Also interesting: If you want to optimize text for readability on colored background, you should always choose black or white. Not an inverted color or anything. The threshold for choosing either is actually a bit dependent on the device the text is displayed on, but there are some rules that work decently for the general case. Here are some examples: https://www.r-bloggers.com/tol-color-schemes/ I think the 'sunset'-palette is a good choice for color blindness, but you should ask someone who is affected if that is a good choice.

I was thinking of the same article, and found it, I think [1]? There was a higher-rated link also posted around the same time with the same post content [2]. I also found a follow up post [3] while looking for [1]. [1] http://varianceexplained.org/r/trump-tweets/ [2] https://www.r-bloggers.com/text-analysis-of-trumps-tweets-confirms-he-… [3] http://varianceexplained.org/r/trump-followup/

I think this is a great resource https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-i… It doesn't cover deep learning at all, but contains material about many other ML techniques.

The Elements of Statistical Learning together with the online course (http://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in… makes for a great introduction. EDIT: Oops I should have said "An Introduction to Statistical Learning with Applications in R" rather than The Elements of Statistical Learning. The Elements book goes into way too much depth to be a good introduction to the subject.

> A major use-case for Excel integration is in finance, where a vast number of analysts - not newbies in any meaningful sense - work in Excel. R and Excel integration already exists: http://www.r-bloggers.com/a-million-ways-to-connect-r-and-excel/ Not that I think it's a good idea. Also, R is already a thing in Finance, quite a few job postings list it as a useful skill, and Linkedin has R/Finance groups. Anyhow, it's much easier to manipulate data with scripts than spreadsheets, especially with R packages like quantmod. Looking at giant spreadsheets makes my eyes bleed and my head hurt, plus I think R Studio makes more sense for newbies to R anyway (and R Studio Server is fantastic for businesses).

http://www.r-bloggers.com/ make sure you look at some older posts

There are few comments here complaining about R performance just like the parent comment. It's a shame that R does not have industrial strength industrial compiler based on this blog article unlike Python [1]. If you want to see the performance benchmark of R against major programming languages for data analysis please check this excellent keynote speech on the R compilation or more accurately on failures of R compilation effort [2]. As mentioned in [1], the main reason people do not focus on providing industrial strength compiler for R is probably because most of R program’s time are spent in the library codes that are written in a compiled language (e.g., C or Fortran). As you can see from [2], even though this is the case (computation intensive being delegated to proper compiled languages), the R programs running time still suffers due to the "impedance mismatch" because of the look up overheads, indirections, etc. Perhaps someone should try to compile, transpile and/or embed R inside D language similar to the efforts provided in [3][4]. D now supports C compiler internally, has DasBetterC and D interface to C++ (DPP) is second to none. Additionally, since D can also perform better than Fortran codes for numerical computing perhaps Fortran codes can eventually be replaced by existing high performance D library like Mir[5]. I can foresee this symbiotic relationship can be beneficial for R and D (R on D?). D can becomes very popular with extra humongous data analysis…

> R has nothing going for it except a rapidly dwindling number of packages that don't yet have a direct python equivalent. This is not a true statement. Here are the data that goes against this statement. 1. https://www.r-bloggers.com/on-the-growth-of-cran-packages/ 2. https://blog.revolutionanalytics.com/2017/01/cran-10000.html 3. https://www.r-bloggers.com/rs-remarkable-growth/ From 2015 to 2016: ~6,200 to More than 8,000 in April, 2016 From 2016 to 2017: CRAN now has 10,000 R packages. > Even then, the incoming generation of undergrads are increasingly proficient with programming and are shying away from R the same way that they shied away from Matlab after scipy matched it for 95% of their tasks. This is a generalization. So far you've made opinionated negative generalization with no data. Python is great because it learn from Matlab and took many great ideas and inspirations from Matlab. But I'm not going to make sweeping negative statements about Matlab or pretend to know how it going when I don't have enough data or experiences in it.

I use R a lot and I have to say some of these comments are weird. 1. R and Lisp are hardly alike even if it was inspire by it. It's like saying Erlang and Prolog is very similar. If you want learn FP do it in Erlang, Lisp, Haskell, etc.. Don't do it in R, it's half baked. 2. R syntax is ugly with warts. But built in datatype like dataframe, factor type, NA (missing value notion) value, make this language much better than many languages out there for dealing with data. Subsetting dataframe is a breeze even in base R. 3. There are many many advance statistical packages only in R. GLMnet was was in R for 4-5 years before someone decided to port it to Python. You can argue that there might be alternative package. But the statistician that created ridge, elastic, etc... method made GLMnet. There are many statistician out there that just implement their latest method in R. If you want to learn a subject in statistic there is probably a book out there and it'll have an R package and code to come along with it. Next to that will be SAS. There are very few stat book with python packages. You want to learn bayesian statistic? Social Network Analysis? There's a book for it with R code and a package to do that. Good luck finding one in Python for these subfield of statistic. There's a bayesian hierarchical analysis in Ecology and that book is in R. 4. ggplot2 is amazing for static graphic. R doesn't have good dynamic graphic out there and I kinda meh with Shiney. If you hate the…

It's been a few years since I really looked at R, but I don't think the problems with R are simply that people don't learn the language. Some languages are simply not as good as others. We can all learn more about the tools we use when programming, I know that I certainly could. But this doesn't make it our fault that a language is tricky or hard to debug or hard to understand. If we worked at it, I suppose we could all write more efficient programs by using assembler, but that doesn't mean that assembler is the best possible programming language for, say, statistical programming. Someone, Ross Ihaka, that knows a thing or two about R wrote a short post 6 years ago and said "simply start over and build something better". Take a look: http://www.r-bloggers.com/“simply-start-over-and-build-something-bette… My hope is that Julia will eventually be adopted as a basis for a future statistical programming language.

For the uninitiated http://www.r-bloggers.com/

You'll find a lot of them at http://www.r-bloggers.com/

This is a decade old, and its still on point https://www.r-bloggers.com/2016/09/the-simpsons-by-the-data/

Apologies, my bad, but I'm a bit too late to edit. The experts say that R qualifies as a lazy language [1, 2]. My impression was that R was mostly an eager language that somehow allowed for laziness. I will research this further and hopefully suss out why I got confused. [1] https://dl.acm.org/doi/10.1145/3360579 [2] https://www.r-bloggers.com/2018/07/about-lazy-evaluation/

> For example, in R, we try to avoid loops because they are very inefficient This was true before, but the performance of for loops has been improved a lot later years, and while vectorization is still faster, for loops are no longer a no-no See https://www.r-bloggers.com/2022/02/avoid-loops-in-r-really/

Mastodon won't survive this, the Fediverse will. Much as Twitter when it was a Rails app buckled under the load already we see things like https://github.com/mastodon/mastodon/issues/19505 and users of other instances also complain. It's an old crusty complex codebase. So yes you can certainly interact with Mastodon users and you will need to. https://www.r-bloggers.com/2022/10/getting-acquainted-with-mastodon-in… the largest Mastodon instance has less than a million users. Twitter has, what, 450 million?

It doesn't affect the chance of any one prisoner finding their number - that's still 50%. But the joint strategy massively reduces the independence of the individual prisoners' success, and hence the distribution of the number of successful prisoners. Rather than a normal distribution (sum of independent outcomes) with a big peak around 50% of the prisoners finding their number and a vanishingly small chance of them all (or none) finding the right number, you end up with a much more complex pattern - there's an approximately uniform distribution in the 0-50 successes range with ~70% of the overall outcome, and a huge peak at the 100 successes point with ~30%. https://www.r-bloggers.com/2014/08/update-100-prisoners-100-lines-of-c… has a bunch of distribution plots showing this.

it's new as of R 4.1 along with the new pipe operator https://www.r-bloggers.com/2021/05/the-new-r-pipe/

Every number is a number, including zero. Therefore some number of members of any given group have been discriminated against. Therefore any group is indeed discriminated against. You can't argue with objective mathematical proof. Slightly more seriously, if someone is discriminated against for being part of a group, you don't get to appeal to the fallacy of composition applied to all groups in order to claim that there is no discrimination against that individual. Yes, yes, you're not actually writing out that argument, you're just leaving its ingredients lying around next to a naked flame. Entirely seriously, here is your proof: https://www.r-bloggers.com/2019/12/modeling-salary-and-gender-in-the-t… Now switch to some other attempt at distraction via reducio of the actual argument.

Notice that the explosion of rap, hip-hop and punk were because the barrier to entry to music making through sampling and distribution was lowered. The 1980s and 1990s were replete with high profile artists that became famous by using the discarded drum machines and synthesizers of the time [0]. Today, (I believe) most artists use synthesizers and DAWs to create full albums. Eletronica, rap and hip-hop are some of the most popular musical genres [1] [2]. It's not uncommon to see artists that are just one or two people, enabled by a professional grade studio on a PC. [0] https://www.theverge.com/2017/4/3/15162488/roland-tr-808-music-drum-ma… [1] https://ucladatares.medium.com/visualizing-trends-and-patterns-in-pitc… [2] https://www.r-bloggers.com/2017/12/clustering-music-genres-with-r/

https://www.r-bloggers.com/2018/07/about-lazy-evaluation/ tldr: basically, R passes all function arguments as bundles of `(expr_ast, env)` [called "promises"]. normally, they get evaluated upon first use, but you can also access the AST and mess around with it. AFAIK this is called an "Fexpr" in the LISP world. (originally i had a nice summary, but my phone died mid-writing and i'm not typing all that again, sorry!) it's very powerful (at the cost of being slow and, i imagine, impossible to optimize). it enables lots of little DSLs everywhere - e.g. lm() from stats, aes() from ggplot2, any dplyr function - which can be both a blessing and a curse.

Pipe operator is now in base R so one of the nice features is now standard |> https://www.r-bloggers.com/2021/05/new-features-in-r-4-1-0/

HN Techmeme NewsBlur for RSS feeds (newsblur.com) Email - fastmail.com, hey.com, purelymail.com R coding - https://www.r-bloggers.com COVID related: vaccine tracker (https://ourworldindata.org/covid-vaccinations), Apple's mobility tracker (https://covid19.apple.com/mobility) nytimes.com Endpts for biotech news (https://endpts.com/news/) Nature (https://www.nature.com) Science (https://science.sciencemag.org)

The authors also published a set of very good lectures covering content in the book: https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-i…