[00:02:50] They probably know the name Alexander Graham Bell, and they know that Bell invented the telephone. Fast forward a few decades, like several decades, six, seven decades, and the Bell Systems Telephone Company. Is this massive enterprise? I mean, it is basically, I think, like the second largest employer in the country outside of the defense department at one point, because they run the phone system and the phone system at that time is everything.

[00:03:13] Within the Bell system, they create this organization sort of like what Google did with its moonshot factory, where they created Bell Laboratories. So the idea was like, we have this labs arm, right? They do all this like kind of interesting research to advance the company. And the idea factory is about Bell Labs, and the reason there should be a book about Bell Labs, frankly, more books about Bell Labs, is because Bell Labs was arguably, like, one of the highest concentrations of just pure innovation ever in history through all time.

[00:03:40] They win six Nobel Prizes, they invent the transistor, they invent touch tone dialing, make bazooka, they invent communication networks, like, they invent satellites. So it’s basically the entire underpinning of modern life. In that book, there’s one section, a tiny section about this guy who worked there, whose name is Dr.

[00:03:59] Claude Shannon. And they talk about this guy who’s like, unicycling in the hallways, playing chess in the cafeteria. He’s kinda weird, he’s like, sort of a loner, but like, with really smart people he becomes really good friends. And, in 1948, while working at Bell Labs, He publishes this paper in the Bell Systems Technical Journal.

[00:04:18] And that paper really is one of the reasons why modern digital life exists the way it does. And he didn’t collaborate with anybody on it. I call it the ultimate side hustle because he like did it on the off hours, nights, weekends, et cetera. And this paper, what one person described, it sort of came like a bomb, right?

[00:04:36] Because all of a sudden you have this new field information theory that’s invented basically out of whole cloth. And he does it. And he’s super interesting. He like, invents the world’s first chess playing computer. All of these things. I’m reading in this book, The Idea Factory. So I go on Amazon. This is market research for authors.

[00:04:52] And I’m like, all right, let me buy a book about this guy. And I type in his name and I can’t find anything. And my friend had actually suggested, the friend who sent me the book said, Hey, you should really look at this Claude Shannon guy. He might be a good subject for you to write about. And I realized, I was like, there’s just like a space on the bookshelf where his story should be.

[00:05:07] And he’s more than just some scientist. He’s like a wildly interesting person in the way he lived his life. And so I was like, so those are the key ingredients for a biography. One is the first thing interesting to read about because otherwise it’s just like watching paint dry or doing your homework or something.

[00:05:22] And the other is, did they do something important? And I kind of had both of them in this subject. And I also got the sense that like Richard Feynman became famous and became a personality. Albert Einstein became famous, became personality. This is someone with the same. IQ points, the same contribution scientifically at the same level, we should say. But Claude Shannon actively avoided becoming famous and so it made his story in some ways more interesting, but it was the reason that no one had done a biography. And so that’s the kind of story that led me to writing the book.

[00:05:55] I was just like, I mean, really, if you built boiled down, I went on Amazon to try to buy it and I couldn’t find it. So I decided to do it. There’s a lot of steps from there, but I found him. Curious and inventive and interesting and super important. And it was kind of like, somebody should do this. Like, and then I just sort of dubbed myself as somebody.

[00:06:14] Preston Pysh: Well, no, to your point, I know I forget how I became interested in him. And that was exactly what I did. I immediately went to Amazon. I was like, all right, I’m going to deep dive on him. This is like I said, a couple of years ago. And your book is really kind of one of the only books out there. And obviously it had great reviews and then I read it and it’s amazing. And maybe that’s why you’re not going to have any competition is because you’ve covered it so well. But you’re right. I mean, there’s not many people that have written about them.

[00:06:41] Jimmy Soni: That’s right. And I would add two disclaimers for your audience, because it’s sort of important to get some of this stuff for people to listen to. Like, why should we keep listening to this? The other thing that Claude Shannon has This is an incredible IRR as an investor.

[00:06:53] I assume people don’t know this, but he’s actually like a very gifted investor and got super obsessed with the stock market, wrote a bunch of papers and ideas and concepts in this space. He was friends with Henry Singleton, who Warren Buffett admires, and Warren Buffett I think once called Singleton like the single best capital allocator in history.

[00:07:10] Shannon served on the board of Teledyne, the Teledyne Corporation, and there were all of these things in the investment world, too, that Shannon did that, again, nobody knew about, and, like, I only discovered them later in my research. I would say the other interesting thing about Shannon’s life is he wasn’t a tortured genius.

[00:07:26] Right. Does that make sense? We have this vision of like, if you’re going to do something great in the world, then you need to have like deep scars. I like you to hate yourself. And there were definitely times in Shannon’s life that were hard, but on the whole, one of the remarkable things about Shannon’s life is he’s basically like a playful genius throughout his life.

[00:07:45] Right. And it’s why I called the book of mine to play. And it’s what people can take from him that’s bigger than just like, hey, there’s this scientist. It’s really, if you read the book, I think the way that I intended, what does play have to do with serious work? Like science or like investing or like building machines or anything else?

[00:08:05] Why is playfulness important? That’s one kind of through line in the book. And then the last thing I would say is like, for anybody listening who is like, Hey, this sounds super interesting, but I have a lot going on and I don’t want to read a book. There’s a really good documentary called The Bit Player that you should be able to Google and find.

[00:08:22] And it’s the only documentary about Claude Shannon. And to your point, I thought Shannon mattered. And what I didn’t know is that there are actually a lot of people who agree with me. And so the book has become kind of a cult hit among particularly, I would say investment professionals, but many others, because they actually want to know more about him.

[00:08:39] Yeah. And I didn’t know that going in. Ironically, it’s like the thing Shannon would have liked the least, like being sort of pseudo famous now. And so it’s like he passed away many years ago, but like he actually particularly avoided this kind of fame and interest and whatever, because he just wanted to do his own things all the time throughout his life.

[00:09:01] Preston Pysh: You talk about this in the book a little bit. You talk about how after he published his information theory, how he would not respond to letters and communications and was really kind of detached from any type of comms. And the irony of this is that he mathematically figured out how to conduct communications, but then in his own life, in the way he was conducting them, he wasn’t communicating.

[00:09:24] And then to a second point in your book, if I remember this correctly and correct me if I’m wrong, but I remember there being something in there about how his parents were investors and making him read maybe like the wall street journal or something when he was a really young kid. So he kind of came very wired for understanding investing at a young age. Am I remembering that correctly?

[00:09:46] Jimmy Soni: Yeah. So I’ll do the first one first and then the second, let me tackle the exact one. Is he? No, actually he was the parent. Here’s what happened. So in their later ages, like when I think they were around 50 or 60, Shannon leaves Bell Laboratories for MIT and the family settles in a town called Winchester, Massachusetts, one of these bedroom communities outside of MIT.

[00:10:06] And they start to get interested in the stock market. So this is like in the seventies and eighties. So they buy a stock ticker. They start reading the quotes. They start buying copies of Wall Street Journal. And I interviewed Peggy Shannon and Andrew Shannon. Peggy and Andrew were son and daughter of Claude and Betty Shannon.

[00:10:23] And they told me about how like we grew up with a lot of weird stuff happening around the house because our dad was Claude Shannon. And I said, well, give me some examples. And they were like, well, not everybody’s dad like juggles. Not everybody’s dad has like a two story workshop that’s filled with like toys and robots and chess playing computers and stuff.

[00:10:40] And they said, and the other thing is our parents were very seriously interested in the stock market. And they would tell you like, we had the Wall Street Journal at the table. We were always talking about it. They had people over, they were talking about investment. So it was sort of in the air in that way.

[00:10:52] And that was interesting. I don’t know that it shaped those kids profoundly in any way. I mean, maybe it did. It wasn’t anything I got into because I was actually writing the story of Claude Shannon, but I found it interesting that all these things became a part of their family life too, right? It wasn’t like work was separated from life.

[00:11:07] There was this, just to give a story on that, there’s this great moment where they’re hosting some dinner party or something. And Peggy Shannon is carrying toothpicks, like a box of toothpicks. from one place to another. I think it was toothpicks or pins or something. And she’s walking, something happens, the box of toothpicks drops on the ground.

[00:11:23] And she looks up, her dad’s there, she thought her dad was gonna get mad or something, like a parent might get upset that she’s like made this mess. And her dad looks at it and says something along the lines of, it’s been a while since I’ve looked at this, but her dad basically looks at it and says, did you know there’s like a mathematical equation that can predict the pattern that would be created by you dropping the ground, right?

[00:11:42] Another story in this vein, when they would decide how to do dishes, instead of just assigning the roles to people, like, Hey Andrew, it’s your night to wash. Or, Hey Claude, it’s your night to wash. They had a multi sided die, like a 16 sided die or something, and they would roll this dice, and every person had a series of numbers.

[00:11:59] Like, that was the game of chance that they would play to see who had to do the dishes that night, right? So there were things like that that were in the ether in their home. It wasn’t the case that he was the parent. You asked him, your first question was about fame, and like his relationship with both fame and acclaim, like just like being acclaimed for this information theory.

[00:12:17] Here’s what happened. It’s actually kind of a great story. This paper is published in 1948 in two parts, and almost right away, his world of electrical engineering and mathematics and communication technology notices, and everybody’s like, oh, this is a big deal. Like, they’re like, Claude Shannon has done something significant here.

[00:12:34] So he’s put on all his lists of like the most promising scientists. He’s given a profile in Vogue magazine that this famous photographer comes and shoots photos of him. And he has this moment where he’s like a scientific celebrity for like a hot second. He doesn’t enjoy it. What he wants to be doing is he wants to be in his workshop tinkering.

[00:12:51] He wants to be building machines. He wants to be thinking, he wants to be writing. And he kind of just like, turns out like he just doesn’t embrace the fame that’s coming to him. People would send him letters and he had literally a folder that was like, letters I’ve waited too long to respond to, right?

[00:13:04] And he would just like, tuck it in there. He wouldn’t give talks. He didn’t say yes to invitations that he got to give talks unless his wife wanted to travel to the location that they were going to go. So like, he gets invited to give prestigious talks around the world. And it’s only on the urging of his wife, Betty, that he’s like, all right, fine.

[00:13:19] I’ll like, go give the talk so he can travel to Japan. But he wanted to keep to himself and do his own work. The other thing that happens that is in some ways more interesting, and I think it’s more relevant for an age of like TikTok and Twitter and meme stocks and everything, all of these people look at information theory and they’re like, Oh man, I could use information theory to explain everything.

[00:13:38] I can use it to explain birdsong, I can use this, that, and the other. So you have all these people jumping on the bandwagon and Claude Shannon actually publishes this very short piece, I think it was like 340, 350 words. Called the bandwagon and it’s to his people in his field. It’s to other electrical engineers.

[00:13:56] And what he basically says, I’m paraphrasing is he says, listen, I know there’s a lot of enthusiasm about information theory, but we have a lot of research left to do to take the work I’ve done and turn it into practical technologies for people and to write new proofs and to design new technologies and everything else.

[00:14:12] So basically it’s effectively, it’s like everybody’s slow to roll. This is cool. Cool. It’s not the theory of everything, right? And he actually does what so many people don’t do. Like, if you have a successful paper or a successful book or a successful anything in modern life, most people just like ramp that up.

[00:14:29] They do 25 versions of that, right? It’s like, Oh, that’s the theory is successful. We should make 12 of them. It’s like, let’s do that. Everybody get on the bandwagon. And you have this temptation and he rejects it wholesale. He says, no, this is like, Information theory. It needs to be further developed. There needs to be more research, more work.

[00:14:46] I understand that that work is valuable, but he says, let’s keep our house in order is what he says. Right? To me, it was like, it was super counterintuitive. It’s not what you would want to do. It’s not what a lot of modern people would do. Right? You have 1 viral song hit. About finance bros, and now you’re like, you’re like signing a deal with Gluttony, and Shannon has his viral scientific hit.

[00:15:08] And instead of doing what everyone else might have done, he leaves Bell Labs, goes to MIT, builds a workshop, and starts the next phase of his life, which is like this interesting, tinkering, game playing machine part of his life.

[00:15:22] Preston Pysh: Wow, it’s funny. You made the comment about everybody just basically saying information theory is this or that and everything else I found it really interesting and I know this is a huge tangent David Sinclair I don’t know if you’re familiar with his work out of Harvard and his information theory of aging That’s right.

[00:15:39] Which is talking about how the epigenetics in your cells are losing data and losing information and then they can’t communicate in order to keep you healthy is a huge pull on some of these ideas that Shannon has put forth.

[00:15:52] Jimmy Soni: I would say, by the way, he takes it very seriously. He, Dr. Sinclair and I, we actually corresponded about Shannon. He’s a huge fan and I wrote him when his book came out because he had mentioned Shannon a couple of times. I said, Hey, you might, and he had already read a mind at play. Yeah. We created some really nice nodes. Now, what’s interesting is he is carefully and rigorously applying the principles that are in information theory to talk, to discuss his field.

[00:16:14] And so there is this, like, I think what, what Shannon was saying was like, let’s electrical engineers try to puff this up too much in our domain. But the metaphor or the, some of the principles that are in information theory apply very well in other domains. I’ll give you another example. That’s probably more relevant to your audience and actually has some crossover with some of the other work I’ve done.

[00:16:31] Yeah. One of the things that people have mentioned is that there’s a fair amount of overlap in the world of Bitcoin and cryptocurrency and the work of information theory. And one of the reasons is because what those systems do is reduce noise in financial transactions, right? They reduce latency. And, well, noise is not latency, but they reduce latency.

[00:16:52] And, for a separate project, when I was interviewing Elon for the Founders book, which was about PayPal, and we were talking about the role of money, like, what is money? He sort of has this view, he said, money is just an information system. It’s a way for us to avoid having to barter. That’s almost an exact quote from something he said to me.

[00:17:11] It’s like a way that we don’t have to barter. It’s an information system. And he says, because it’s an information system, you could see how a universe of digital currencies is actually a much more efficient information system by Shannon terms than something like handing you, Preston, a dollar, right?

[00:17:27] Because there’s a lot of friction involved there. And so there are some interesting applications of information theory around the world and useful ones. In the case of Sinclair, information being coded into our genes, Shannon actually did work on genetics before he published the information theory paper.

[00:17:42] And so there are some useful applications. I think what was happening is that people were not treating the work with sufficient seriousness in 1949 and 1950 and 1951. And a lot of those people were within Shannon’s field. And that’s why he had to kind of throw the brushback pitch of that bandwagon paper.

[00:17:58] Preston Pysh: Wow. I’m so impressed that you coordinated with David on the work he was doing there too. That’s so cool. And if people are curious about that, oh my gosh, go read his book. What’s the, Lifespan, is the name of the book? Oh, phenomenal.

[00:18:11] Jimmy Soni: He’s got another one as well.

[00:18:12] Preston Pysh: Yeah. It’s phenomenal. And he does such a great job kind of making it accessible for people that maybe aren’t into biology.

[00:18:19] Okay, I would love for you to explain information theory for people that have no idea what it is. Like if you were going to really kind of break it down into like the basics, explain what it is and why it’s so groundbreaking and why it’s actually this underpinning for all communications we see today.

[00:18:41] Jimmy Soni: Yeah. This is one of those where I’m going to do my best. And I translated it and spent a lot of time thinking about that chapter of the book when we were writing it. If you feel like my explanation is inadequate, find me on Twitter, shoot me a DM or shoot me an email, and I’ll just send you that chapter of the book because it’ll do a much better job than I can in 30 seconds of explaining it.

[00:18:59] But here’s the visual that I would use that helps best explain what information theory actually does and why it matters. Right now, you and I are able to communicate over Zoom. Or I can like pick up my phone and download a YouTube video, right, and watch a YouTube video. Right. And the reason that that process can happen is because of the transmission of information, but it’s compressed information.

[00:19:24] And what information theory does is it provides a model for the compression and transmission of information. Without loss of meaning. So, I’ll give you an example that emerges in the paper. There are a bunch of different ways to convey a piece of information without losing meaning. You can shout louder.

[00:19:45] You can repeat yourself. You can do many things to communicate information better. But some of those ways of communicating information better will just add to the amount of time or the amount of bits that you need to communicate information. So the real trick in information theory is, how do I tell Preston a sentence in this video, and how is it communicated as efficiently as possible?

[00:20:08] There’s a number of components to this, but one of the things that Shannon explains to people in variation theory is there’s a lot of redundancy in the English language, right? So he’ll take a sentence, like the fox jumped over the something, and he’ll show how even if you eliminated most of the vowels or other letters within that sentence, Someone reading it could actually still make sense of it.

[00:20:29] And that compression is one of the ways that you and I can have this exchange without it taking seven years. And the reason I’m doing that is because I want to connect the theory to the technology, which is this is. Effectively, the theoretical plumbing for all of modern life, it affects everything from the phone calls we make to this exchange to a video or a song.

[00:20:53] If you’re using Spotify, you are in some meaningful way using Claude Janin’s information theory. The idea is how do you most efficiently and in the most compressed format possible. Communicate information with high fidelity. And what Shannon outlines is he outlines a theoretical limit for some of these.

[00:21:10] It’s called the Shannon limit. And since he laid down that limit, engineers have basically been trying to get us technologies that communicate up to that limit. And that’s why it was such an earthquake. Someone finally figured out that there was a model for this and he lays it out in a little drawing.

[00:21:25] And if any of your listeners are curious, they can just Google information theory and they’ll see a little drawing. And he basically outlines that there is a sender, there’s noise, there’s a channel, and there’s a recipient. And he shows you different ways to reduce noise in that channel. And so that’s what we’re trying, that’s the basic guts of information theory, is how do we communicate.

[00:21:44] Now, a number of surprising things emerge from this. One of the surprising things that emerges, Shannon posits that the actual value of information is not the meaning, it’s the surprise. So, so that’s a really interesting thing.

[00:21:58] Preston Pysh: What do you mean by that?

[00:21:59] Jimmy Soni: Yeah, so what he’s saying is, it’s that all information can basically be abstracted into a mathematical formula.

[00:22:06] That there’s no difference between me saying the fox jumps over the dog. Mathematically, there’s no difference between me saying the fox jumps over the dog and the dog jumps over the fox. They’re actually identical. So meaning doesn’t matter. The actual bits, mathematically, they’re equivalent. He also points out, That because the English language has so much redundancy in it, it’s surprise that is the thing you’re trying to control for.

[00:22:28] Like if you have a word that’s unexpected after another word, that’s new quote unquote new information. So a good example of this, you, the letter U always follows the letter Q. So if you see a letter Q, I think it’s like a 99. 999 percent chance that the next letter after Q is U. But if all of a sudden, let’s say next year, Merriam Webster, there’s some word that goes viral and it’s cute and it doesn’t have a you after it, that is actually, from an information theory perspective, that is more information, meaning this new word is surprising and ergo is new information because we have to do more to convey it, we have to do more to communicate it.

[00:23:08] It’s a little abstract, but I think people can get the idea. The basic idea is there’s a ton of redundancy within the English language. And you can eliminate whole words and whole vowels and people can still understand something. And so you can use that redundancy to your advantage. Because what you can do, and this is where Shannon applies the information theory, you can encode messages with certain kinds of redundancy so that my message to you is conveyed, not perfectly, but almost perfectly.

[00:23:34] And by the way, I don’t know if people have ever watched the show Silicon Valley, but when they’re talking about, like, lossless compression algorithms, the lossless compression algorithm idea traces directly back to this paper. And so for people who are working in modern communications, you stand in a 70 year legacy of something that started with Claude Shannon, which is a, how do I get a text?

[00:23:56] On my phone to be transmitted from planet Earth to a satellite and over to Preston without losing any of the meaning. And that is the theoretical work that Claude Shannon did.

[00:24:09] Preston Pysh: This is mind blowing stuff. Something that I’ve heard recently is with like chat GPT 4 and because this is when we’re talking about intelligence, we’re talking about compression, right?

[00:24:19] We’re talking about the compression of patterns. One of the things that I read is they took a, and I might be getting this way off, but generically speaking, I think this is accurately described when you take a wave file and you compress it into an MP3, it has way smaller amounts of data that’s being stored and also to send.

[00:24:38] I heard that they could take a wave file and they could run it through one of these AI models, and it would give you better compression than the MP3 compression, meaning it’s being stored and with the same level of quality as an MP3, but with less data required to store it. This all relates back to Shannon’s, right?

[00:24:58] This is all interconnected with the math and the theory that he came up with and, and correct me. I’m probably saying this way wrong, but with more entropy, and I think this is what you were getting at there is with more entropy, you allow for more data to be transmitted. Is that being stated correctly?

[00:25:14] Jimmy Soni: Yeah. And people are going to have to forgive me. It’s been a number of years since I wrote this work and since the book’s been out. Yeah. And I think we have to be careful. Because data and entropy are really specific terms within this field. So I want to be careful about not sort of going over my skis or getting things wrong.

[00:25:30] I’m way over mine. I’m way, but what the basics of what you said are correct. Two things. One is these compression algorithms have gotten better over time. And it doesn’t surprise me that large language models, which are studying language broadly defined, that they have found ways to compress files further.

[00:25:49] And make the amount of data that is transmitted smaller, because what they’re probably doing is they’re, they probably have better ways of predicting what the next word or what the next bit or what the next thing will be. And when you can get that prediction improved, you don’t need as much data transmitted.

[00:26:07] And so if you didn’t need a specific word or bit and you cut it out, you’ve taken, let’s say, 10 units of data and you’ve made it into 8 units of data. And there was a time when we needed to convey every single bit. And now, because of these predictive technologies, We can communicate with fewer bits. And by the way, we’re at, we’re at the 10, 000 foot level on this.

[00:26:29] We’re also at the limits of my understanding about how LLMs actually do this. But

[00:26:35] these things are intimately tied together because the whole thing, the whole sort of genius insight for Shannon was we actually don’t need most of the English language. And that applies across different fields. We actually don’t need this, this, this, and this in order to communicate a message.

[00:26:52] And it allows for more efficient communication. So it doesn’t surprise me that the world of AI and LLMs has embraced this to make compression better. And I actually suspect that this is going to herald an era in which technologies like 6G are come to market more, much more quickly. Like I think there’s going to be a bunch of changes because of this, because honestly, it’s part of the magic of playing around with some of these tools is seeing exactly how they know what to do.

[00:27:18] Because they are predicting things that used to be done by us, even on image creation, right? Yeah. On image creation, they can predict that this shade of blue should be, based on statistical probabilities, probabilistically speaking, this shade of blue should be next to this shade of blue on a picture of planet Earth.

[00:27:33] That’s a probabilistic determination. They’re making a best guess at what something could be. But it turns out the machines are pretty good at guessing. They’ve gotten better.

[00:27:41] Preston Pysh: Yeah. Well, I might go way over my ski tips here on this next one, but I want to ask it. Anyway, I’m curious if Shannon wrote about the increase in demand for energy for basically unwrapping the messages on the receive side.

[00:27:59] Because they’re so deeply compressed. So when we get into compression, like you, we could compress a message down to, I mean, as an example, we could take one of these LLMs that are 600 gigabyte files. That’s all the bigger that some of them are. And then, I mean, they’re insanely powerful. They can answer almost any question and they’re only about half a Tara.

[00:28:19] We could compress that using two, a shot, two 56 encryption down to 64 characters. Like all 600 gigabytes down to 64 characters and transmit it very simply, but on the other side of that, to understand what was in that half a tera file, it would take look, I did a little bit of math here before I did this with you.

[00:28:39] It would take 3. 6 times 10 to the 49th years to decompress that file that is only 64 characters by taking a half a tera. Okay. So there’s energy required on the receive side of a message that’s been deeply compressed. And so you’re putting the onus on the receiver. So I’m sending you this message right now, as we’re talking, you’re receiving it, you’re computing it inside your head.

[00:29:07] But if I was going to simplify and compress all of that message into two words and send it to you, you now have to do a whole bunch of work on your end in order to decompress those two words, assuming I could compress it into the two words. Just getting dramatic here with the example. So I’m curious, did you study anything that Shannon ever talked about that?

[00:29:29] Because I think this is in the reason I asked this question, I think in the coming 10 years, 15 years, the energy demands that we’re going to have in order to deal with all of this compression, especially AI compression is going to be off the charts. And I don’t think the world’s prepared for it. I think it’s about to get crazy.

[00:29:46] Jimmy Soni: It’s a great question. And in some ways it was a very simple answer, which is Shannon was in the following great position or privileged position. He pushed information theory, not the application of information theory. And so he had this, this great, so there’s a bunch of great lines about this from other people in the field.

[00:30:05] And they basically said, they were like, Shannon didn’t identify what the error correcting codes were going to be, or what the algorithms are actually going to be that did this. All he did, his genius insight was identifying that this was a statistical possibility. That it was actually possible for engineers to do this.

[00:30:26] And what that did is it was like a starter gun. It basically sent the rest of that domain off on a search to find the codes that would most efficiently compress information, that would reach the Shannon limit, that would enable the technologies of today. Shannon didn’t do that.

[00:30:41] He had a hugely groundbreaking insight in that this is possible. We can statistically do this, but he laid out the theoretical architecture for it. A good analogy or a good way to think about it is when Galileo was basically challenging the church and saying, no, the earth revolves heliocentrism versus geocentrism. Galileo didn’t have to do Everything that came after the acceptance of geocentrism, meaning there’s a bunch of stuff that we’ve discovered about the fact that the earth revolves around the sun after Galileo.

[00:31:19] He wasn’t doing all of that, but it was important that the theory of geocentrism be broadly accepted as the truth so that we could do all of these other things. In the same way, once Shannon identified that it was possible to do this, It sent the rest of his field off on the research that for the next 60, 70, and it’s still going on today, would create the algorithms that we use to transmit information.

[00:31:44] So, the short answer is not really, and he wasn’t really thinking in practical terms about what energy demands would be required, because remember, at the time that he’s doing this writing, 1948, computers are still like these room sized things, you know, they, like, computers are mechanical devices, they’re not digital devices yet the digital world is still super far off, and so, he eventually becomes an enthusiast, he obviously builds some of these things, he gets an Apple II that was delivered from Steve Jobs, all the, he embraces all of that.

[00:32:12] But the technologies are very primitive in 1948, so it’s a conversation that’s a little bit, it’s years away. But it is why, like, why with the subtitle and with the way that the book kind of was embraced, it’s like, we really do live in Shannon’s world. Everything that is going on today is Shannon’s world.

[00:32:28] And I agree with you, by the way, that I think the information demands based on these technologies are going to be significant. Ironically, I also think there’s a world in which these technologies help us figure out that problem.

[00:32:38] Preston Pysh: Yeah, true.

[00:32:38] Jimmy Soni: There’s a lot of promising stuff happening in this world, and I’m not at all an expert in that.

[00:32:42] But my sense is that ambitious, smart people are trying to figure that out, and they’re giving it a college try that seems pretty serious.

[00:32:52] Preston Pysh: Well, to your point, and you mentioned 6G earlier, that’s exactly what you have. So when we’re looking at the electromagnetic spectrum and all of this demand. Of people wanting to zap communications through the air.

[00:33:05] The bandwidth is extreme. I mean, it’s limited. There’s only so much bandwidth that you can shoot these messages through. But with AI and compression, now we can take the code that we’re communicating in, run it through AI and figure out ways to compress it even further, which then allows more space to shoot these messages through the electromagnetic spectrum.

[00:33:28] So perfect example of what you’re saying.

[00:33:31] Jimmy Soni: And you used the word code, as did I, because there are codes, there are error correcting codes that are used to communicate digital messages. It’s an interesting word, though, and the reason it’s interesting is because the other cool part of Shannon’s life is what happens right before the Information Theory paper.

[00:33:47] He is a codebreaker during World War II, and so part of what he’s studying, like here’s an example. There was a phone system that Churchill used to communicate with FDR. It’s called Sig Salley. Sig Salley was designed to develop at Bell Laboratories. I think it was a pretty top secret project. A lot of information emerged about it after the fact.

[00:34:05] But Shannon was one of the engineers working on these code breaking devices. He was also somebody who like, that was an important job during the war, was figuring out how can President Franklin Delano Roosevelt send a message to Winston Churchill that cannot be intercepted by the Nazis. And so part of what Shannon has to do is think about, well, We have to actually figure out a way to do this so that we’re not transmitting a ton of data so that it can avoid detection.

[00:34:31] You have to figure out little tips and tricks and tools to, like, take a certain word and turn it into something entirely different. Alan Turing was doing this work, obviously with, like, ciphers and other things in the UK. Turing and Shannon, funny enough, became friends. There’s a part of this that is about encoding, and a part of it that, interestingly, like, runs through a legacy of code breaking and code making.

[00:34:52] Which I always found, like, one of the more interesting parts of this, like, the idea of secret codes, things that we were trying to hide, ultimately lead to a world in which We don’t have to hide anything. You can be anything about yourself, right? It’s a kind of cool thing. It’s a, it’s a very interesting historical irony that it is information we were trying to conceal from people is what gives us the ability to conceal nothing from people.

[00:35:17] Preston Pysh: A little context of a comment you made earlier in the conversation where you said that his children said that they had to roll a dice and create entropy to do the dishes. Now that they’ve got a little taste on what he did for the world with some of these theories, it makes more sense, I guess, to hear that that’s the kind of thing he did with his kids.

[00:35:36] Jimmy Soni: The other piece of it I’ll tell you is like a bunch of my favorite Shannon stories, ironically, are not scientific stories. They’re parenting stories. Yeah. One more example. I think this made it into the book, but I don’t know if it did. There was an oversized tree on his property at one point. I was like, a big tree, they had to build a little bit of land, and the natural thing to do was like, it was an aging tree, and you gotta take it down because it’s unsafe, or whatever, or there was an eyesore, I don’t even remember why they were gonna cut it, but they needed to cut it.

[00:36:00] And Shannon had this thought, instead of cutting it down, What we should do is we should hire some people to basically file the tree down so that it’s sort of like a flagpole that’s growing in the ground. And then we should carve a skull at the very top of the tree and hoist a pirate flag up this carved tree with the skull on top.

[00:36:18] And that’s exactly what they did. So they built They like, they like fashioned the filed this tree down. Like, got a skull on top, hoisted a Jolly Roger up the flagpole, and the whole reason they did it, there’s no purpose, they just did it because Sharon thought it was funny and kind of interesting. I’ll give you another example.

[00:36:33] One morning, I guess Andrew, his son, woke up, said he’s got a performance at school, and I guess he was going to do a solo performance, it was some kind of competition or something, and he was going to play the trumpet. The end. Shannon said, well, would it be interesting if your trumpet breathes fire? And his son was like, yeah, it would.

[00:36:51] So, so Shannon constructed a flame throwing trumpet that Andrew used at school to perform. Again, with no practical purpose, just because he thought it’d be funny and kind of clever and interesting. And this is, we talked a lot about information theory and about the details and the arcana of the science.

[00:37:09] It’s worth appreciating that a lot of the fun of doing this project was just figuring out kind of the zaniness of the Claude Shannon. He was just like somebody who like had an idea and did it. It was a part of it that’s more endearing because we, especially adults, we take ourselves super seriously, right?

[00:37:24] We all take ourselves a little too seriously. Shannon never did. It was one of those things that I still remain inspired by it. A mind at play, right? I mean, you nailed the title. And when you’re saying all these stories that are just, they sound so loony, I go back to a statement that you said where you were describing the word, the sentence there that you kind of flipped, but you were like, it’s the same amount of data, but it’s the surprise in it that, that communicates or is.

[00:37:51] What was the word you used?

[00:37:52] The information is contained in the surprise. Meaning, meaning if I can predict it, it’s actually not informative by Shannon’s definition of information. It’s not informative. Information is contained in the surprise. That’s one, it’s like a, it’s like a little bit of a, of a very simplified way to think about information.

[00:38:09] Preston Pysh: I’m just thinking of the flagpole with the small, I mean, it’s just so it’s such a surprise. All right. Unfortunately, I’m so sorry. I, I take great pride in reading any book of a person I bring on the show. I have not read the founders yet. I need to read this. What gladly send a copy. I would love to read this, especially because our show covers all these various billionaires and what they did and to give people a, just a quick rundown on what the book was and what you were trying to accomplish and maybe your motivation on it.

[00:38:37] Jimmy Soni: For sure. And I’m happy to come back on and chat about it because it’s a much longer conversation. I suspect in some ways, like your audience. Is more familiar with the people who are at the heart of that book than they are with Claude Shannon. You know, maybe. PayPal is a company that has existed at this point for over 20 years.

[00:38:54] If you’re listening, you’ve probably used one of their products because they own Venmo now. Obviously, the core PayPal ecosystem, everything else. And it also has given rise to a group known as the PayPal Mafia. That includes people like Elon Musk, Peter Thiel, Reid Hoffman, David Sachs, the founders of Yammer, Palantir.

[00:39:14] I mean, every, every significant company to come out of Silicon Valley, arguably for 20 years, has some tie back to this ecosystem. Whether they’re an investor or something or whatever. Many years ago, I think at this point, five or six years ago, maybe actually longer. I got this idea in my head. I was like, everyone’s talking about these people, but how come no one’s ever gone back and asked them like how they all ended up at the same company at the same time?

[00:39:38] That seems kind of weird, right? The question was just like, wait, hold on. Elon and Peter and, and, and, and all these people worked at the same company, what was in the water? I mean, it was literally the question I was asking, it’s one of my sentences in the introduction to the book. What was in the water there?

[00:39:56] Because if you think about it, if it was a graduate school, literally every person on the planet would want to get in.

[00:39:59] Preston Pysh: Yeah. Right. Yeah.

[00:40:00] Jimmy Soni: And I got the sense that there was a story there that hadn’t been told because what happened is that these people became so larger than life. That the focus of everyone’s attention is kind of like, what are you doing today?

[00:40:13] Not what did you do 20 years ago? So I do hook and crook and a lot of intros and everything else. I interviewed around 300 people and I wrote the book, it’s called the founders and it’s about the, the story of PayPal from 1998 to 2002. So it is the story of creating this payments architecture and this startup.

[00:40:32] During the dot com bubble bursting and during all these other events happened, but basically it’s this crazy four year sprint to build this company and it has everything in it from like what you’d expect, like kind of like how did he learn, get the idea to do this company, why was it called X dot com at one point, all of those things in addition to like, there are a lot of things that were invented at PayPal that you probably use without knowing it.

[00:40:58] I’ll give you one example, CAPTCHA tests. These tests where you have to identify that you are a human and not a robot. Those were commercialized at PayPal. And the reason they were commercialized at PayPal was because when PayPal designed its system, a bunch of people started creating bots to create fake PayPal accounts so they could get the sign on bonuses.

[00:41:19] The company needed to devise a way To separate Preston flesh and blood human user from like 10 robots Preston created to collect 10 bonuses, right? They did that with the first application of the CAPTCHA tool. It was the first time they did it. Surprise. It was surprise. And at first, the people who were on the product team were like horrified because they thought this would like nuke new signups.

[00:41:45] And it turned into an innovation that It’s literally all over the internet. I mean, you can’t, you can’t use most websites these days without having some kind of financial services website, certainly without having some kind of gate like that, I can go on and on. That group created things in financial services that still affect our lives today.

[00:42:00] And then I thought it was just interesting to know, like before they flew private and ran the world, what were these, who are these people? Who were these people when they were in their early twenties? How did they figure life out? How did they get to Silicon Valley?

[00:42:13] Preston Pysh: What’s a common thread you found in that question is what they did what they did early in their life

[00:42:18] Jimmy Soni: There’s a lot of people at the heart of the story. So there’s a few common threads But I don’t want to paint with too broad a brush. Yeah, it’s like a little tough I would say that a lot of them were Building things on computers when it wasn’t cool to do so and when it wasn’t lucrative to do so So remember the internet economy doesn’t really become an economy until the mid to late 90s when Netscape goes public.

[00:42:39] Before then, the internet is like this thing that academics use, and it’s a thing that like nerds use. It’s not a way to become a billionaire. In the 80s in particular, if you wanted to become very, very wealthy, you went into financial services, you didn’t go into technology. And if you went into technology, you did hardware, not software.

[00:42:53] You didn’t do the internet. So all these people like kind of tinkering, they’re playing with computers, they’re doing things, they all move out west, the boom starts. A lot of them study computer science, engineering, not all of them. There are a bunch of people on the business side who are more traditional humanities tracks.

[00:43:08] But I would say that like one common theme is I noticed that many of them had built businesses before they ever joined PayPal

[00:43:15] So for Max Levchin, who’s one of the co founders one of the best known co founders He is now the CEO and founder of a firm PayPal was his fifth business and we had four failures in college while he was like moonlighting as a student He was also building businesses.

[00:43:28] It’s his fifth or sixth company, right? Elon had a computer company built in college Then he had Zip2. Then he had an arcade when he was living in South Africa. And then he had PayPal. So it was like his fourth, maybe his fifth. And so you just have people who are serial entrepreneurs, but minus the word entrepreneur because that sounds, makes it sound fancier than it is.

[00:43:48] They were just building businesses. And trying to figure it out along the way.

[00:43:52] And so I would say that’s a common thread. I would say the other common thread is there’s a lot of game playing, puzzle playing, chess playing. There’s a lot of like that in the air at PayPal. There’s a bunch of stories in the book about that, but obviously Peter Thiel was a well known chess player.

[00:44:06] But I would say there are a lot of chess titans within this group. A lot of poker players. There’s a lot of that kind of puzzle solving game playing in the ether as well. But it’s interesting, it’s fascinating, to me it’s a fascinating story because it’s like, who are these people before they become relentless objects of attention today?

[00:44:24] I find the most interesting thing to me about writing biographies is figuring out who people are before they become rich, famous, or successful. Yes, totally. Because to me, I want to know who they are when they’re closer to me. I’m trying to solve these Rubik’s Cubes for myself. I don’t actually find most of the contemporary coverage on these people all that interesting.

[00:44:41] Because the truth is when you’re at that level, it’s not, your life becomes a little bit more like boxed in, but when you’re in your twenties and nobody knows who you are, the world is a

[00:44:50] blank page.

[00:44:50] Preston Pysh: You got to go upstream, right? Yeah. What is one of the stories? I mean, you’ve interviewed some just incredible people, 300 people for the second book.

[00:44:58] Oh my God. What is a really unique, a surprising, let me use that word. What is a surprising story from one of these billionaires or very unique people that you had the opportunity to interview that you just off the top of your head? It could be any one of them.

[00:45:12] Jimmy Soni: Oh, I mean, there’s so many, but let me pick a couple.

[00:45:15] One, one that people may know, but they should really think a little bit more about, which is there is this temptation these days that I think if Elon ate a piece of fruit, there would be a bunch of hate about that piece of fruit. I think if he suddenly got into like raspberries, there’d be like an anti raspberry coalition or some nonsense like that.

[00:45:36] And I think that one of the nice things about writing about a very specific and defined period of his life. Is it? I wasn’t asking him any questions about SpaceX, Tesla, Twitter, none of that. I was studying the years 1998 to 2002. And one way to think about the years 1998 to 2002 in his life is that it’s during this period that he gets kicked out.

[00:45:57] He gets overthrown at PayPal as a CEO. He almost dies because he contracts malaria and meningitis while on a trip abroad. And he loses a child and people don’t really know because he’s not going to talk about it all the time. But I went pretty deep on the history of that little moment. I wrote about it at the end.

[00:46:18] And you think about what someone’s been through and it looks like these days, his life is a glide path. He can say whatever he wants, et cetera. There’s this real profound four year period of difficulty in which he struggles with the hardest thing that anyone I think could, could struggle with. As a parent myself, I can’t fathom losing my kid.

[00:46:37] And he has an infant died, a sudden infant SIDS, whatever that acronym is. Yeah. And, and it’s this really interesting thing. And I think at the end of chapter 14, Is when he leaves a story because you know, he CEO. But I think people would really benefit from reading that, especially people who kind of throw stones and don’t realize there was a period in which it wasn’t a glide path.

[00:47:00] There wasn’t a 56 billion payday. You need to have a little bit of, I think, humility about the people in our business life who become like business celebrities and to appreciate that there’s a humanity there. There’s a challenge there. There’s some struggle there. So that’s one story. It’s a little surprising.

[00:47:16] The other thing I would say is the fact that this company, PayPal, it’s created like as the dot com bubble is, is just getting to its peak. And months after its creation, March of 2000, The bubble bursts and all of a sudden, like the NASDAQ loses 86 percent of its value that year. 86 percent of its value.

[00:47:38] People like Barron’s is writing like they had this famous cover story. It’s like yeah, dot dot bomb, right? And it was about Oh, Amazon dot bomb. It was like Amazon dot bomb. And the idea was like Amazon’s share share price at the time is 9. And everyone’s like, Oh, this is it. The internet’s like a total hype machine.

[00:47:54] It’s all nonsense, right? It’s all foolishness. September 11th happens during the early creation years of PayPal. 9 11. And it massively changes things like KYC rules, know your customer rules, and financial regulation because the government suddenly wants to crack down on Paris financing. There are all of these things that happen.

[00:48:11] They get sued. They get hacked. People trying to defraud them. They’re in this epic battle with eBay. They are in one of the worst markets for IPOs after 9 11 that’s ever, there wasn’t an IPO for three or four months because I mean, 9 11 is like the financial services industry is in rubble, literal rubble.

[00:48:29] You have two competitors, X. com, which is Elon’s company, and Confinity, which is Peter Thiel and MaxLegend’s company. They fight for three months, then have a merger that’s basically organized within four or five weeks. And this is all just before the bubble bursts and there’s no financing left in venture for this kind of company.

[00:48:46] So I would just say that the surprising thing about PayPal is the mere fact that it survived. The most surprising thing about PayPal is this company should have gone under. None of this should have happened. And it stayed alive. And that’s the cool part of the story is how do you keep a company alive when you’re under siege and the pressure is on and everything is like chaos and there’s not low interest rates and there’s not easy money and there’s not a hundred million dollar rounds that are being thrown at people.

[00:49:13] To my mind, that’s actually like one of the cool things about this. And it’s one of the things that this team walked away with is they built something successful, but it came during a period of real hardship. And it meant that every subsequent thing they did, YouTube, Yelp, Palantir, SpaceX, Tesla, LinkedIn, that they were accustomed to this difficulty.

[00:49:36] And that’s probably a good way to preview that particular project for those who are interested.

[00:49:41] I like that point a lot. Well, an interesting tidbit, and you might have to correct me here on this, but when Elon left as the CEO and Peter Thiel, I believe was the one that came in and replaced him. There was obviously bad blood there when that took place.

[00:49:55] But what’s interesting is decades later, Elon is really struggling with SpaceX. It’s about to go bust. And Peter Thiel was the guy that showed up. I can’t remember the number, but I believe it was over a billion dollars. And gave him a lifeline that was somewhat of a repayment or karma or whatever you want to call it later because I did this,

[00:50:17] this homework.

[00:50:18] It’s funny you, you referenced something that is one of the final lines I have at the end of chapter. Oh, really? 14 of the book. Yeah. And and it was actually when SpaceX not at a billion, it was, I think it was 14 million or 20 million or something.

[00:50:28] Preston Pysh: Oh, okay. Sorry.

[00:50:29] Jimmy Soni: And it comes from Founders Fund and it is one of the parts of the story that people miss.

[00:50:36] Which is they did have a feud. They did have a moment where Peter replaced Elon as CEO. There was a lot of bitterness on both sides. If you were to talk to Peter and Max, they thought they were doing the right thing. If you talk to Elon, they were doing the wrong thing. But it is the case that Peter Invest in Elon and more than that, like invest in Elon.

[00:50:57] And we’ll often, he had said to me something like, there’s a reason I don’t invest in other companies and industries that Elon is in, because I know what it’s like to compete against Elon and you’re going to lose, right?

[00:51:07] Preston Pysh: Relentless.

[00:51:08] Jimmy Soni: And so it’s interesting because it isn’t just that there is respect.

[00:51:12] There’s a difference between respecting somebody and giving them. A 15 million check for their rocket company. And so you’re right on, on the broad outline, which may have been some temporary misgivings about how things happened, but over time, there’s a partnership there that has actually withstood a lot.

[00:51:30] Preston Pysh: People underestimate Elon’s proof of work from his early days. Like the zip too. I remember reading that like he slept underneath of his desk. And so when he buys Twitter and he goes in there and he’s conducting the audit and then he fires 80 percent of everybody in the company and everybody’s like, the place is going to fall apart.

[00:51:47] There’s no way you can do that. It’s because Elon went in there and was like, all right, let me see all the code contributes of every programmer in this company. And he’s going through and he’s auditing all of these like, okay, yeah, we can downsize this by 80 percent and we’re not going to have any issues because I’ve written the code for this GPS navigation system.

[00:52:06] Three decades ago or whatever it was from that moment in time. So yeah, there’s people that underestimate like what that guy has done in his life and how gritty he has gotten his own hands in writing code or whatever it might be from an engineering standpoint.

[00:52:21] Jimmy Soni: And it’s hard, right? Cause he’s, he’s one of the most covered subjects in the world right now. And by the way, like my book is not, it doesn’t praise, it doesn’t like, it’s not a book where I’m trying to praise him or trying to criticize him. I just went where the facts took me. So I interviewed a lot of people. I had a bunch of email, I had about six gigabytes of email.

[00:52:40] From that era that someone shared with me. So I had a day by, so for people who are listening, maybe the reason to read the book is because you can read Elon’s resignation note. You don’t even have to take my word for it. Like, it’s all there. It’s all just facts. But I would say that one of the things that happens is that people underestimate his Willingness to do whatever it takes to have his companies be successful.

[00:53:02] And the way that that kind of focus inspires engineers to do their best work too. A lot of the people writing about Elon are not engineers. And so the problem is they’re writing about things, but they don’t have any kind of understanding of how those things work. And so it’s easy to look at something like, I’m going to lay off 80 percent of Twitter and say, that’s terrible.

[00:53:24] It’s another thing to, for somebody who has been the CEO of, at that point, two internet companies, Zip2 and HIP, to look at a situation and say, well, here are the things that matter in this business, here are the things that don’t matter, and I need to cut costs, because that’s part of the issue here, is that it’s just, things are deeply inefficient.

[00:53:41] I can’t be, I don’t know enough about that situation to know what was right and what was wrong and what was executed well and what was executed poorly. But I think that one caution I would give to listeners who are reading things about Elon is to remember that most of the people writing about him don’t have an engineering background.

[00:53:56] And it’s actually a real deficit when you’re writing about things like rocket engineering or car design or anything like that.

[00:54:02] Preston Pysh: Love it. Jimmy, I am, I’m so thrilled with this conversation and your book is phenomenal for people that want to pick this up. A Mind at Play. And the other book is called The Founders and is there anything else that you want to highlight or point people towards website or I know you’re active on Twitter or X.

[00:54:21] Jimmy Soni: Yeah, I mean, I’m not that active on X. I mean, I’m as active as I, as I need you, but I love hearing from readers and you can just find me. I have a website, JimmySony. com, but just find me on X and DM. I love hearing from folks who are in this space. And I would just say, I really appreciate your enthusiasm for Shannon.

[00:54:35] It’s actually one thing that unites the two worlds. The people I interviewed knew who Claude Shannon was. Like, Elon references Claude Shannon on X from time to time. Because these ideas still last decades later. And there’s something really inspiring about that. I mean, like, ultimately, all of us want to do work that stands the test of time.

[00:54:53] And so it’s, it’s interesting to me that Shannon has had a kind of a second life. The only other thing I would say is I, I am quite curious if Anthropics Claude is named after Claude Shannon and nobody’s been able to pin that one down for me yet. And so I’d be really like, if any of your listeners or you or somebody put a signal into the universe and figure that one out, I would be really grateful. That’d be cool.

[00:55:12] Preston Pysh: Let us know folks. Yeah, we might get, I don’t know, we get 100, 000 listeners on

[00:55:17] Jimmy Soni: There you go. Maybe somebody, maybe somebody had anthropic notes and they’re like, no, no, no, it’s actually because you don’t applaud Van Damme. They just thought that was really cool.

[00:55:25] Preston Pysh: Awesome. Well, Jimmy, thank you so much for your time.

[00:55:27] This was a real pleasure and hopefully we’ll get another chance to chat sometime in the future.

[00:55:32] Jimmy Soni: That sounds great. Thank you, Preston.

[00:55:33] Preston Pysh: All right. See ya.

[00:55:34] Outro: Thank you for listening to T. I. P. Make sure to follow Bitcoin Fundamentals on your favorite podcast app and never miss out on episodes. To access our show notes, transcripts, or courses, go to theinvestorspodcast. com. This show is for entertainment purposes only. Before making any decision, consult a professional. This show is copyrighted by The Investor’s Podcast Network. Written permission must be granted before syndication or rebroadcasting.