Software Carpentry

We're teaching a course at ETH Zurich [1] where --besides the actual payload of solving partial differential equations (PDEs) on GPUs-- we put a lot of emphasis on "tools". Thus students learn how to use git and submit their homework via pushing to a repo of theirs on github, we teach testing and continuous integration, writing documentation, running code on a cluster, etc. In their final project, again submitted as a GitHub repo, they need to make use of all of theses skills (and of course solve some PDEs). Note that excellent work in this space is done by the Software Carpentry project which exists since 1998 [2]. [1] https://pde-on-gpu.vaw.ethz.ch/ [2] https://software-carpentry.org/

The fundamental problem here, as you note, is that scientists are rarely also engineers, and don't really share our desiderata. The point is to develop and publish a result, and engineering analysis code for resiliency is of secondary concern at best when that code isn't likely to need to be used again once the paper is finished. The "Software Carpentry" movement [1] has in the past decade tried to address this, as I recall. It's very much in the vein of the "basic literacy" course you suggest. I can't say how far they've gotten, and I'm no longer adjacent to academia, but based on what I do still see of academics' code, there's a long way still to go. [1] https://software-carpentry.org/

That is basically the mission statement of Software Carpentry (https://software-carpentry.org/). They have a _ton_ of great material for those in scientific computing wanting to improve their software engineering skills.

This looks good. I would also recommend the Software Carpentry program: https://software-carpentry.org/

I suggest you look into the Software Carpentry foundation - they are making a great effort to improve scientific programming through short workshops: https://software-carpentry.org/

I really like what this group is doing: http://software-carpentry.org/ I took their instructor training last summer but I haven't had a chance to run a workshop yet. I think it's a great idea, especially given my recent exposure to researcher-written codebases.

Great article. I am a instructor for Software Carpentry[1] , the goal of these workshops from my experience is to try and help mostly scientists get started on the journey to becoming early programmers. In biological sciences with more and more data becoming available, the Expert blindness Zed speaks of is a major problem. We need to invent better systems and actually take heed of research based teaching methods as SW does if we wish to improve this situation. https://software-carpentry.org/

Really interesting take, very much in line with my experience that scientific code is often broken from a software engineering point of view. Science benefits greatly from using more standard toolkits in open languages instead of proprietary code of any sort. This way the kind of reproducibility the author talks about would be easier to evaluate. That said, it's often hard in practice. I remember asking a colleague why they were so matlab-heavy when nothing they did couldn't be done in say, python. Their response was that all the previous code in the lab was written in Matlab, so re-writing and re-engineering it would be a huge and somewhat fruitless undertaking. You don't get publications from refactoring. Globally though I think science is heading in the right direction, big fan of efforts like Software Carpentry (http://software-carpentry.org/).

I think that's what the folks at Software Carpentry [0] are trying to do. I went on one of their courses, and you're taught the basics of writing good software, version control and databases (SQLite). I've frequently recommended it to fellow scientists. [0] http://software-carpentry.org/

Cool! I love how universities are acknowledging that there is a fast-growing "long-tail" of programmers who are scientists and data analysts, and that they need different sorts of programming training than CS majors ... Reminds me of Greg Wilson's http://software-carpentry.org/

When it comes to great screencast videos for Python (+ software engineering), I think Software Carpentry (http://software-carpentry.org/) is really nice. and funny. This might be a different from Khan Academy and thenewboston, which seem to aim for a different level/audience. But for what it is and does, I feel like SWC is one of those hidden gems that I wish would get discovered a little more...

The Carpentries[0] provide some great resources and training for academics who are interested in this. Check out Software Carpentry[1] and Data Carpentry[2] in particular. Publishing research code is admirable, and in an ideal world everybody would publish their code and data. That said, we shouldn't pretend that there aren't tradeoffs. Time spent polishing your code to make it presentable is time not spent on other aspects of your research. Time spent developing software development skills is time that could be spent learning new research techniques (or whatever). Reproducible research is great, but it's certainly possible to take it too far at the expense of your productivity/career. You should also take your own personality into account. If you're a perfectionist you might struggle to let yourself publish research-quality code rather than production-quality code and consequently over-allocate the time you spend prettying up your code. [0] https://carpentries.org/ [1] https://software-carpentry.org/ [2] https://datacarpentry.org/

https://software-carpentry.org/ has been around for over 10 years. Also, software engineers without the requisite background are usually worse for a project (and for teaching researchers) than anyone else.

I agree with you, having been on both sides of the divide and researched & written my masters thesis on teaching programming to undergrad science students. Are you aware of https://software-carpentry.org/? It started after I graduated and I knew people who were involved with it at the time. It seemed like a good idea.

https://software-carpentry.org/ Similar concept, but targeted towards non CS majors

Software Carpentry would be a good place to start: https://software-carpentry.org/

Please note that the Software Carpentry [0] project exists as well. All welcome . [0] https://software-carpentry.org/

Things are changing, slowly but surely. Those of you interested in this sort of work should check out http://rse.ac.uk/, and also https://software-carpentry.org/ (Run a workshop! Improve the software environment around you!)

You're not alone in feeling this way. Greg Wilson organized 'Software Carpentry' [0] to teach researchers the basics of software engineering. They regularly do workshops (list/schedule at site). [0] https://software-carpentry.org/

I glanced at the Clive Thompson article. I disagree. I feel movements like "software carpentry" (https://software-carpentry.org) are a good indicator of what mass computer literacy will look like in the workplace and society as a whole. Essentially more folks will know how to develop small scripts to support their main work (or customise their home automation systems). In turn, I also suspect, in the long run, programmers that act as the equivalent of a scrivener (the 'blue' collar workers of those articles) are going to dramatically decline as a percentage of folks engaged in software development.

> who cares if the code is a write-only mess as long as it does the one job after which no one will care about it anyway This is exactly the kind of attitude that projects like 'Software Carpentry' (https://software-carpentry.org) are pushing back against.

If this issue is interesting to you from either the researcher or software engineering side please consider working with, taking classes from, or just supporting Greg Wilson's Software Carpentry http://software-carpentry.org or the spin off Data Carpentry. Full disclosure: I am not affiliated in any way shape or form. Just a fan of the intent.

If you're interested in improving this situation, the Software Carpentry Foundation trains scientists in modern programming practices, and also teaches scientists and programmers how to teach programming to others: http://software-carpentry.org/

Have you seen http://software-carpentry.org/ ? I haven't looked close enough at what you are doing or what they are doing to pretend to make a comparison, but I guess it isn't too risky to say that they are at least thinking about some of the same things you are.

I know that there's some cross-over in the Python/Scientific community with http://software-carpentry.org/ - which is interesting. I also know that in Chicago 'apprenticeship' is a word that is thrown out there a lot. http://www.8thlight.com/apprenticeship https://engineering.groupon.com/2012/software-apprenticeship/groupons-… https://www.devmynd.com/jobs/apprenticeship

If you get a chance go on a Software Carpentry [1] course. I went on one 4 months ago and it changed my whole outlook. Greg Wilson take you through the psychological aspects of programming - why you should break every hour - the difference between "sip of tea" and "make a cup of tea". How your code relates to short-term memory and much more. Don't think "I know how to program". Even if you know the syntax and the idioms you don't know everything. Simply going through a workshop will change the way you think. [1]http://software-carpentry.org

It sounds as if http://www.software-carpentry.org might be something to join for most participants in this thread - a non-profit organization that runs bootcamps for scientists to learn proper coding, version control etc. I think they're always looking for more volunteers.

do you know about the organization Software Carpentry? http://software-carpentry.org/ They work on teaching scientists to be better at software, and are always looking for people to do workshops

There's a company called Software Carpentry ( http://software-carpentry.org/ ) that runs workshops for scientists but they might do the same for other programmers or have an idea of who might.

My feeling is that a one-semester required course for students in "software carpentry" [1] (as developed by Greg Wilson and discussed in the article) would cure many of the most serious ills in scientific software development. Students can't know they should be using version control, debuggers, and testing if they don't even know such things exist. [1] http://software-carpentry.org/