Cal Paterson

Yea and this created "bank python" informally a good article is here. https://calpaterson.com/bank-python.html The best description about how it became a problem is one of the paragraphs. "New starters take an exceptionally long time to get up to speed - and that's if they don't resign in fit of pique as soon as they see the special, mandatory, in-house IDE (as I nearly did). Even months in, new starters are still learning quite fundamental new things: there is a lot that is different." I think it took me til I was there around two and a half years to fully comprehend it when I was working on it. Not much modern training til they figured out they had to teach it again that was better. The worst part is to make an UI around it coding it and it wasn't approved for new projects.

A must-read before diving into asyncio: Async Python is slower than "sync" Python under a realistic benchmark. A bigger worry is that async frameworks go a bit wobbly under load. https://calpaterson.com/async-python-is-not-faster.html

read this amazing article on "bank python": https://calpaterson.com/bank-python.html

These are just 2 recent links from my bookmarks: - How a SQL database works <https://calpaterson.com/how-a-sql-database-works.html> - Making a change to SQLite source code <https://brunocalza.me/making-a-change-to-sqlite-source-code/> I can recommend following Phil Eaton on twitter: https://twitter.com/phil_eaton. They post lots of nitty gritty posts about database development. I likely discovered the above links via their feed.

Mozilla is a great comparison. Compare how CEO pay goes up while Firefox users are down 85% [1]. The Red Cross has also been mired in controversy (eg [2]). Eventually such organizations just seem to collapse under the weight of their bloat and have very little effective spending on the things they fund raise off. I'm also reminded of Yes, Prime Minister [3]. [1]: https://calpaterson.com/mozilla.html [2]: https://www.propublica.org/article/disturbing-things-about-the-red-cro… [3]: https://www.youtube.com/watch?v=Ke9ec_Ua3x0

Huh, information in https://calpaterson.com/mozilla.html are, delicately speaking, disturbing. One thing surprised me as well: "Firefox barely exists on phones, with a market share of less than half a percent. This is baffling given that mobile Firefox has a rare feature for a mobile browser: it's able to install extensions and so can block ads." Yes, indeed, FF is pretty great on the phone, it looks like someone fails to advertise it as such. I use it precisely because I can use add blocker. The person with 2.4M compensation should notice that something wrong is going on. I am afraid that Firefox/Mozilla at some point will become something like Yahoo or Commodore, a name that is being bought and sold by investment founds to extract some income and sell it further...

A nonprofit where the people at the top have multi-million dollar salaries (see https://calpaterson.com/mozilla.html) Yes, Brave is for profit, but not at the cost of user data and/or trust. Brave doesn't collect or exchange your data. Firefox, on the other hand, transmits your keystrokes to Google right out of the box. Leith did a great side-by-side comparison of various top browsers (including Firefox and Brave), finding the latter to be in a class of its own in terms of privacy out of the box: https://www.scss.tcd.ie/Doug.Leith/pubs/browser_privacy.pdf.

> This has never happened and never will. You simply are not omniscient. Even if you're smart enough to figure everything out the requirements will change underneath you. My best project to date was a largely waterfall one - there was somewhere around 50-60 pages of A4 specs, a lot of which I helped the clients engineer. As with all plans, a lot of it changed during implementation, actually I figured out a way of implementing the same functionality, but automating it to a degree where about 15 of those could be cut out. Furthermore, it was immensely useful because by the time I actually started writing code, most of the questions that needed answers and would alter how it should be developed had already come up and could be resolved, in addition to me already knowing about some edge cases (at least when it came to how the domain translates into technology) and how the overall thing should work and look. Contrast that to some cases where you're just asked to join a project and help out and you jump into the middle of ongoing development, not going that much about any given system or the various things that the team has been focusing on in the past few weeks or months. > It’s not hard to see that if they had a few really big systems, then a great number of their problems would disappear. The inconsistencies between data, security, operations, quality, and access were huge across all of those disconnected projects. Some systems were up-to-date, some were ancient. Some…

There is a surprising amount of important data in various Mongo instances around the world. Particularly within high finance, with multi-TB setups sprouting up here and there. I suspect that this is in part due to historical inertia and exposure to SecDB designs.[0] Financial instruments can be hideously complex and they certainly are ever-evolving, so I can imagine a fixed schema for essentially constantly shifting time series universe would be challenging. When financial institutions began to adopt the SecDB model, MongoDB was available as a high-volume, "schemaless" KV store, with a reasonably good scaling story. Combine that with the relatively incestuous nature of finance (they tend to poach and hire from within their own ranks), the average tenure of an engineer in one organisation being less than 4 years and you have an osmotic process of spreading "this at least works in this type of environment" knowledge. Add the naturally risk-averse nature of finance[ß] and you can see how one successful early adoption will quickly proliferate across the industry. 0: This was discussed at HN back in the day too: https://calpaterson.com/bank-python.html ß: For an industry that loves to take financial risks - with other people's money of course, they're not stupid - the players in high finance are remarkably risk-averse when it comes to technology choices. Experimentation with something new and unknown carries a potentially unbounded downside with limited, slowly emerging…

A few months ago I wrote an article testing out the relative speed of Python's async options for web apis/sites (https://calpaterson.com/async-python-is-not-faster.html). My findings were a bit controversial at the time as I found that uWSGI + sync Python is about the best option, largely because replacing interpreted Python code with native code is a very significant factor in the performance of a Python program. In the following discussion (and private emails) I was stunned by the number of async enthusiasts who proposed raising the number of database connections in the connection pool by two or three orders of magnitude (I had 20 conns in a pool for a 4 cpu machine) for reasons along the lines of "the async app is starved of connections". In my opinion that suggestion betrays a misunderstanding of what is likely going on in the database when you have 100s or 1000s of connections all contending for the same data: in SQL even reads create locks. Async applications yield a lot, by design, and under hundreds or thousands of requests per second there is a considerable buildup of "work in progress" in the form of async tasks that were yielded from and which have not yet been returned to and completed. Many hundreds of database connections is going to create an enormous volume of bookkeeping on the database side and is very likely to slow things down to a absolute crawl. Even idle connections are known to have detrimental effect in postgres. Someone at Microsoft is…

Thank you for pointing me to Linda -- fascinating and my mind is popping with fun projects related to this model. It actually seems quite spiritually similar to "Bank Python" (https://calpaterson.com/bank-python.html), but more elegant.

I can't say this is exactly what you're after, but this article is really interesting https://calpaterson.com/bank-python.html Similar to what the author describes, I wouldn't be surprised if a lot of this information is generally not public.

I don’t know about that. Python gets forked in other investment banks as well. I wouldn’t say that is evidence of any deficiencies, rather they just want to deal with their own idiosyncrasies. See https://calpaterson.com/bank-python.html

One of the largest, if not the largest python codebase in the world, implements similar ideas to model financial instruments pricing: https://calpaterson.com/bank-python.html.

Having observed the trajectory of Mozilla since the Internet Explorer days, I am saddened and reminded of Pournelle's Iron Law of Bureaucracy[1], which seems especially apt here. While Mozilla has become increasingly irrelevant in the eyes of its users and fans, apparently it is still quite lucrative for those at the center of its bureaucracy[2]. What's extra sad is that Firefox is clearly a valuable public good, which seems fated to go down with the rest of this rotten ship. [1]: https://en.wikipedia.org/wiki/Jerry_Pournelle#Pournelle's_iron_law… [2]: https://calpaterson.com/mozilla.html

I saw a thought-provoking post that similarly compared LLM makers to the airlines: https://calpaterson.com/porter.html

This is much more intelligent article on caching: https://calpaterson.com/ttl-hell.html

Mozilla wastes all its money on its CEO and bullshit DEI programs, while neglecting to develop Firefox, their flagship product. When they spend some money on the browser, they use it to implement user-hostile garbage like Facebook's client-side tracking bullshit. Here's a cute graph of CEO pay vs Firefox market share: https://calpaterson.com/assets/mozilla-boss-pay.svg Mozilla is completely fucked. I just hope the company goes bankrupt so its niche of an open source browser can be filled by an organisation that actually cares about its product, and is not just a sham for execs to get millions of dollars from Google for being faux-competition, while fucking over their users.

Cool to see some "bank Python" that Cal Paterson had described previously is now open source https://calpaterson.com/bank-python.html

The title chart from this article caught on like wildfire a few years ago: https://calpaterson.com/mozilla.html "Firefox usage is down 85% despite Mozilla's top exec pay going up 400%".

Is this kind of like the Rust equivalent of Bank Python? https://calpaterson.com/bank-python.html

The original HN thread from 2016: https://news.ycombinator.com/item?id=12096250 > "The banking programming world is a completely different world than what most of us are used to" If you're after another read on this topic, "An oral history of Bank Python" is good: https://calpaterson.com/bank-python.html (also previously on HN: https://news.ycombinator.com/item?id=29104047)

I want a system more like Bank Python[1] or Unison[2] or Pharo Smalltalk[3] that gets us beyond the idea of "code in files". And I have plans to build this on top of Common Lisp + SLIME [1]: https://calpaterson.com/bank-python.html [2]: https://www.unison-lang.org [3]: https://pharo.org

https://calpaterson.com/mozilla.html Mozilla knows how to siphon money, they play double face non-profit and for-profit at the same time with the same name

I enjoyed this article from Cal Paterson my excolleague. It's about Python async not being faster: https://calpaterson.com/async-python-is-not-faster.html I think the idea is that while your blocking waiting for IO in one task you can serve a different task, potentially from a different user. Coroutines, green threads, communicating sequential processes as in Go or Occam.

Goldman Sachs famously has a proprietary-everything dev environment that works rather well for the kind of applications they need: https://calpaterson.com/bank-python.html

> Relying on PageRank / links Search engine companies learnt long ago that parsing all that content was time-consuming and costly, so they simply use the 'meta' tags instead.[0] [0]: https://calpaterson.com/metadata.html

https://calpaterson.com/bank-python.html > One of things that tends to boggle programmer brains is while most software dealing with money uses multiple-precision numbers to make sure the pennies are accurate, financial modelling uses floats instead. This is because clients generally do not ring up about pennies.

Perhaps the one that has multiplied her own salary while the browser gets less and less relevant. https://calpaterson.com/mozilla.html

I'm no expert, just reading web pages: Apparently Firefox market share dropped from 30% to 3.46 over the same time her income went from 500 k to 4 million. An old write up: https://calpaterson.com/mozilla.html