Deepseek, open source and the growth of the Internet
History is rhyming again...
If there’s one topic that I’ve all but worked to death, it was drawing the parallels between the growth of open source software and the internet. I’ve talked about this on YouTube, written about it in books, essays and articles, given seminars at universities, tweeted1, podcasted2, posted and SMS'd about it. To put a fine point on it, I’ve even spoken about this on basic cable3.
Here’s the tl;dr of all that: As the internet grew, only free software could keep up with the demand for web page rendering, internet serving, operating systems to run those servers on, and even protocol implementations that implemented the fundamental technologies that made up the ‘web’.
It wasn’t that commercial organizations like IBM, Oracle, Sun and Microsoft didn’t want to charge a pile for the tech they would develop, they simply could not serve their customers or meet new customers fast enough to satisfy the demand.
Suppose you pinged your account manager at Sun or Microsoft and said you wanted to serve a web page to your customers like your competitors were doing. Depending on your importance to the organization, they would call4 you back and see what select agreement you were under, how many CAL5s you’d need, and how to charge you for them, etc. Often you would be offered a professional services team, who might return to you in x weeks or months to spec out the …. You get the point.
While waiting for a call back, IT departments would find an open source server they could just download start experimenting with. A <head> tag here, a <body> tag there and some janky table tags6 and boom: you’re serving web pages! Then you’re trying to figure out nonsense like page counters and email forms and the rest. Probably downloading some janky, insecure7, scripts from matts script archive and go to town!
Depending on your size, you would likely be picking up a content management system/user login system, figuring out how to take money for goods/services before a major IT company could even find the time to call you back.
Fast forward 25 or 30 years, and you have these truly remarkable llms emerging to incredible demand. While the AI companies have been doing a better job at satisfying user demand than those early companies did during the birth of the web, the open source models were always mere steps behind and as we came to see with the LLaMA and Deepseek releases, catching up *fast*.
A few months before my exit at Google, in late 2022. I would contact multiple people about how Google should be aggressively opening up its weights and models. Indeed, yes, go ahead and fight the lawsuits that would come with what I saw as a fair use of the information therein. It would only take two years for them to see that Facebook’s, and now Deepseek’s open source8 releases are not threats, but opportunities.
Until the last year or so, coverage of LLM approaches to problems in computer science has been, well, frothy, not very thoughtful9 and couldn’t answer the fundamental question of ‘what good is this for me and my users?’ As you saw people start using foundational models to access, process and understand their own data that is when you started seeing people say, okay, maybe this LLM stuff is relevant to my world or use case.
Please note, I’m not trying to say I told you so, or anything as silly as that. I’m just pointing out that again the computer industry seems to be ignoring history to its detriment. I can assure everyone that even if DeepSeek turns out to be a red herring in terms of training efficiency, one thing is laid bare: Scrappy open source approaches to llms will focus on taking current code, datasets and models, making them all more efficient on hardware that normal users have available to them, and figuring out how to best apply it to their individual problems.
My pal Simon’s whole substack and blog and code is about just that. There’s little need to pay attention to me when you could be paying attention to him.
You’re still here? Okay, I’ll leave you with this…
There’s a question I ask myself and those I advise when writing software or building out infrastructure: “Do you really think that you’re the only one to have this problem?” Inevitably and without much effort, I’ll find code on some repo somewhere that is , maybe 50% , 90% or 95% of what I need. So I fetch, tack on my 5%, 10% or 50% and keep moving forward.
In the end, LLMs and the assorted tech makes me feel like it’s the 90s and I just wanna put up a webpage.
Twitter was a website that allowed you to post your sms/finger status’ to a website. Sort of like Bluesky now.
Seriously, we didn’t even call it podcasting when I started ‘internet radio’ with my pals on geeks in space we were so young. We still do it weekly-ish and you can find it wherever you might get your podcasts. It’s not always , you know, good, but it’s fun for us? We’ve been doing it on and off since the 90s. Which is insane.
I’m pretty sure my family cut our cable in 2002 or so…. so that’s how long I’ve been haranguing on about this.
On the phone! A phone was a thing that used to sit on peoples desks at companies! Think a cell phone that wasn’t portable, not just with a bad battery, or something….
Client access licenses. You’d need to buy one for everyone who might visit your website. Laughable? Maybe! But it was a way they could think about the users on a website that fit their business model… or didn’t as no-one went for that nonsense.
For a while, there were more misspelled attribute tags in html than correctly spelled ones, and web browsers … adapted.
but boy did they work. people loved themselves Matt’s script archive.
Seriously, if I see another llm written Haiku, especially one without a nature call out , that maybe doesn’t even meet the 5-7-5 structure as proof positive that one should go spend 20/month for access to such a creative font of genius, I’m gonna scream.