A PyData London 2018 talk on how JPMorgan scaled its Athena Python platform over ten years to about 35 million lines of code and thousands of developers. It covers infrastructure, tooling and culture for analytics and visualization at that scale.
Why Hacker News points people here: Cited as the definitive public account of Python at massive scale, especially in discussions of monorepos and bank Python.
“4500 developers making 20k commits a week. Codebase with 35m LOC.”HN comment, 2020
“4500 developers making 20k commits a week. Codebase with 35m LOC.”
Athena's origins, uses, architecture, infrastructure, dev tooling and culture.
Full comment
As the comment by timsworkaccount says, this is definitely Athena at JPMorgan. For those interested, I gave a talk on Athena at PyData UK 2018 called "Python at Massive Scale". 4500 developers making 20k commits a week. Codebase with 35m LOC. The video is here: this video It covers Athena's origins, what it is used for, application architecture, infrastructure, dev tooling and culture.
Cites itstevesimmonsJul 2018
“My company has a 50m LOC Python codebase in a monorepo. It works really well”
Talk on a 50m LOC Python monorepo and supporting devtools.
Full comment
My company has a 50m LOC Python codebase in a monorepo. It works really well, given the rate of change of thousands of developers globally. That is only possible because of the significant investment in devtools, testing and the deployment infrastructure. Here is "Python at Massive Scale", my talk about it at PyData London earlier this year: this video
Cites itmcintyre1994Oct 2018
“I've seen a struggle to manage a far smaller codebase than that in Python and really interested to see what you're doing differently”
Full comment
If anyone else is looking: this video :) I'm definitely interested in this - I've seen a struggle to manage a far smaller codebase than that in Python and really interested to see what you're doing differently to make it work.
Cites itBarrin92Aug 2019
“JP Morgan operates a ~30 million LOC Python platform for trading and analytics.”
Large JP Morgan Python platform for trading and analytics.
Full comment
>How stable is python to run a full trading / quantitative algorithm on? JP Morgan operates a ~30 million LOC Python platform for trading and analytics. (related talk: this video ) Yes, there are very, very large working python codebases in fields out there that demand correctness. I'm honestly getting tired of the static typing circlejerk that has entered the industry.
Cites itstevesimmonsNov 2021
“For a view of Python at JPMorgan, I did a talk at PyData London”
Author's talk on Python at JPMorgan.
Full comment
Nice stuff! For a view of Python at JPMorgan, I did a talk at PyData London "Python at Massive Scale" - this video
Cites itstevesimmonsNov 2021
“Seeing Python at the core of trading, risk and post-trade processing for Commodities, FX, Credit etc was such a great developer experience.”
Author's PyData talk on Athena for more details.
Full comment
I worked on Athena at JPMorgan for 8 years, and loved it. Seeing Python at the core of trading, risk and post-trade processing for Commodities, FX, Credit etc was such a great developer experience. By the time I left JPM, there were 4500 devs globally making 20k commits weekly into the Athena codebase. (I did a PyData presentation on this [1] for more details). The one downside was the delayed transition from Py2.7 to 3; I left just as that was getting underway. [1] this video The
Cites itstevesimmonsJan 2022
“Athena had 4,500 Python developers doing 20,000 commits a week.”
Talk on JPMorgan's Athena trading and risk system.
Full comment
See [1] for a PyData London talk I did on "Python at Massive Scale", about JPMorgan's Athena trading and risk management system, used for FX, Commodities, Credit, Equities, etc trading. At that time, Athena had 4,500 Python developers doing 20,000 commits a week. Also see one recent HN discussion on "An Oral History of Bank Python" [2]. [1] this video [2] link
Cites itBarrin92May 2022
“Python is being used to built seriously large software nowadays”
Talk on JPMorgan's 35 million LOC Python codebase.
Full comment
>Python shines beautifully as a (somewhat) cross platform scripting language Python is much more than just a scripting language. I remember attending this talk[1] a few years about JPMorgan's 35 million LOC Python codebase. Python is being used to built seriously large software nowadays and I don't think performance is ever a minor issue. It should always be in the top 3 for any general purpose language because it directly translates into development speed, time and money. [1] this video
