Episode 205: Chris Lattner Interview Transcript
ATP interviews Chris Lattner
Transcript of Accidental Tech Podcast episode 205, an interview of Chris Lattner on January 17, 2017:
- Introduction and LLVM
- Creation of Swift
- Swift’s initial reception
- Swift adoption at Apple
- Achievements at Apple
- Swift and Tesla?
- Open-source Swift
- Swift 3 and ABI stability
- Server-side Swift and expanding Swift’s relevance
- Systems programming in Swift
- Progressive complexity: from novices to experts
- Swift language influences
- Why adopt Swift now?
- Swift’s style and community
- Rust-style memory model?
- Swift in web browsers?
- ABI stability soon?
- Swift left in good hands
- Hobby language?
- Dynamic features
- Apple reads blogs
- RxSwift and reactive programming
- Garbage collection vs. ARC
- After-show: Woodworking
- After-show: Neutral (cars)
- After-show: Bonus
Casey Liss: Let’s start with some follow up. Why did Chris Lattner decide to leave Apple?
John Siracusa: Last week we talked about this and we tried to list all the reasons we could think of why he might be leaving. I think we covered most of the major threads of speculation. We didn’t really get a lot of listener feedback about it. Anybody that did reply was mostly voting for one of the things we talked about.
Here’s the thing on this topic: unless there’s some kind of secret doomsday thing about Apple that only Chris knows, it’s not actually that important that we figure out [0:30] why he left Apple or what was in his heart of hearts, aside from just us being busybodies and just wanting to know, gossip type stuff. Second, pragmatically speaking, this is just not the type of thing that people who leave Apple talk about or, if they do talk about it, it’s years and years later after no one cares anymore or it’s not a big deal. Even if there was some deep, dark reason, it’s not like we would know about it until years from now and, honestly, the straight-forward boring explanation is probably the right one anyway. I don’t know. What [1:00] do you think, Chris?
Chris Lattner: That sounds about right to me, John.
Introduction and LLVM
Marco Arment: Would you ever buy an electric car in white?
Casey Liss: You are such a jerk. Why are you so mean to me, Marco?
Chris Lattner: No, I’m a fan of red cars.
Marco Arment: There you go.
Chris Lattner: Yellow, if I can get them.
Casey Liss: Yellow. That’s bold. Alright. We obviously need to talk about what’s happening right now. ATP is not an interview show but, in extraordinary circumstances, weird [1:30] things can happen. So in today’s episode, we have an extraordinary circumstance. Former Apple employee, Chris Lattner, was nice enough to join us on the show. So this is going to be three people interviewing one person, which is, to say the least, a bit peculiar and it’s something that we’ve never really done before. So this could be a little bit of a rough ride, but we’re going to try to make the best of it.
I’ll just say right up front, thank you so much, Chris, for coming on the show and we definitely appreciate it. To [2:00] kick all this off, would you mind giving us a nickel tour of, who is Chris Lattner?
Chris Lattner: Well, I can try. I’m a programmer. I love coding. I have for a long time. I’m most widely known for starting LLVM and getting it off the ground and handing it off to other smarter people. I started Clang, which is a C, C++, Objective-C compiler and, likewise, got it to the point where it was thriving and then let other, smarter people run [2:30] it. Then started Swift and got Swift going and it recently got past 3.0, and it has an amazing community and has a great team of people running it. Now I’m ready to jump to the next thing. I guess the short version of what I’d like to do is tackle really hard problems and do things that most people think are impossible.
John Siracusa: We’re going to go back in time a little bit and I hope that you can tell us how you ended up coming to Apple? If you can remember back [3:00] that far. You started LLVM before you were at Apple, so how did you end up at Apple?
Chris Lattner: Great question. I started working on LLVM at the University of Illinois. It was a research project, and it really was hoped for and designed to be fully general and production quality and all those great things that you’d want as a graduate student for people to use your code — but it also wasn’t. It had a [3:30] lot of the great architectural ideas, maybe, but it wasn’t fully baked. It wouldn’t solve all the world’s problems. And by the time I got to graduate, the question was: what do I do next?
Of course, my dream was to continue pushing it forward and build it out to be something real, and to do that, I would have to go to a place that would allow me to work on it. I was really fortunate to build some relationships with folks at Apple and they took a chance on me. I jumped to Apple and started building LLVM [4:00] as my job. It was totally a dream of mine so that’s how I got to Apple.
John Siracusa: Did you graduate? Did you get your degree and then go to Apple?
Chris Lattner: Yeah.
John Siracusa: Well, that was nice.
Marco Arment: Otherwise how would you ever get a job? You can’t get a job without a degree. We all know that now.
Chris Lattner: Yeah. Clearly.
John Siracusa: How would you explain LLVM to somebody who doesn’t know stuff about compilers? Because we all just rattle off “LLVM compiler-building toolkit” or whatever, but how would you explain that to someone, because not all of our listeners are programmers, who isn’t intimately familiar with how a compiler [4:30] works?
Chris Lattner: First, we start with what is a compiler? A compiler is a thing that takes the code that a programmer writes and turns it into something the machine can understand. There’s lots of different kinds of computers with lots of different kinds of processors. Most programmers don’t want to have to think about that or know about that and they want program and think in a much higher level than what the actual processor can do, and so the compiler’s job is to transform what the human wrote [5:00] to something that the machine can understand. Then, LLVM is a modular system for building different kinds of compilers because there’s lots of different reasons people would want to use compiler technology. I think LLVM was most successful over the years because it was built as reusable components, kind of like a framework, and then the framework can be used to solve lots of different problems. One of the fun things about LLVM is it was eventually over the years used to solve problems that we had never even imagined [5:30] which is really exciting.
John Siracusa: One example of the type of problem is, I remember from past WWDCs, the idea that, the Xcode IDE, the GUI, that programmers use to make their applications, very often in that GUI you need to do things that have an understanding of the code. But when the compiler is this big, monolithic thing, the best you can do is compile everything in the background and spit out some stuff. When it’s a modular toolkit, you can take those pieces and just use the part that understands [6:00] the syntax tree or whatever and integrate them. Was that the first use of LLVM as something other than a straight up compiler that you just point to your code and it produces output? In other words integrating it into applications or was there some use before that?
Chris Lattner: I think the first year in production was in the OpenGL stack which ended up shipping in a late software update for Tiger (10.4) as well as in Leopard (10.5). [6:30] There it was used in a completely invisible way to basically repack data and upload it to the GPU, so when you’re making OpenGL calls to, say, glVertex3f(), or whatever the low-level GL call is, those vertices need to be transformed into a very specific data format the GPU can understand and you have a whole bunch of different kinds of calls that can be used to produce vertex or other geometry data. But you also [7:00] have a matrix of different kinds of GPUs you need to support and they all have different formats and different capabilities and requirements, so LLVM was used to generate very small chunks of code to do that. It was part of the 64-bit bring-up in the Leopard timeframe. That was probably the first completely invisible use of it that used and proved a lot of the code-generation technologies.
John Siracusa: Forgive me, one more LLVM-related thing because we’re going to go from what is a compiler all the way down to nitty-gritty stuff here. [7:30] My vague recollection of what you just described, the whole video-driver thing, was that there were files on disk in the shipping OS that were basically LLVM bytecode, like these little BC files, and they would get slurped up and converted to machine code appropriate for the GPU. Am I right about that?
Chris Lattner: Yeah. You’re absolutely right. Basically, the way to look at it is the code files, the LLVM that shipped with the OS, were little snippets of code that then at runtime, [8:00] were recombined and optimized across. That was kind of the library of primitives that the OpenGL runtime used to assemble what it was trying to do.
John Siracusa: So my question about this, and maybe you kind of alluded to it with either a slip of the tongue or a revealing understanding, is what is the difference between LLVM bytecode and what Apple currently brands as Bitcode?
Chris Lattner: Well, are you asking for, is this a historical question or do you …
John Siracusa: It’s technical and historical because I’m confused by the branding and I’m also confused by the tech.
Chris Lattner: [8:30] LLVM prior to LLVM 2.0 — and these version numbers are the open-source version numbers, not related to the Apple version numbers, and they’re confusingly different for not-very-interesting reasons — but before 2.0 it used a really horrible encoding in binary that was branded “bytecode” because that’s what Java used. It had a whole bunch of problems: it wasn’t extensible, it was fragile, LLVM was changing, it didn’t [9:00] have compatibility.
The LLVM 2.0 timeframe was probably, I don’t know, ten years ago or more. It’s been a long time ago now. I redesigned the whole thing and it now uses this Bitcode format, and used “bitcode” as a term for two reasons: it was different than “bytecode” so the two could live side-by-side in the directory tree for a while, and the encoding is a bitstream [9:30] format instead of a byte stream. That’s the origin of the term Bitcode. You can find documentation on the file format. It’s actually a very general encoding format that you can use for XML-type purposes. It’s a self-describing file format and multiple different things can be encoded in Bitcode.
John Siracusa: There was definitely some confusion about that because when Apple presented it, they presented it with the requirement that Apple Watch apps have to be in Bitcode. Marco, you would know this.
Marco Arment: Watch is required, iOS is still optional.
Chris Lattner: Yep. [10:00] And Apple TV is required.
John Siracusa: They would have you put your stuff in Bitcode and there was a lot of speculation early on about what that meant: does that mean that what you’re uploading is not a finished binary and therefore, it’s like this machine agnostic can be targeted at any CPU? And the more prosaic version: it doesn’t mean it’s CPU-agnostic, all it means is that Apple has slightly more freedom to change instructions on the CPUs that they do target? [10:30] What advantages are there of compiling something to Bitcode and then uploading it somewhere versus sending someone a binary?
Chris Lattner: There’s a couple of different advantages for that, one of which is that the compiler keeps getting better. So if the compiler learns a new optimization it can be applied to existing applications without app developers having to re-upload their app. That’s one small thing. Other things are that Apple does periodically add new instructions [11:00] to its CPUs. One example of this historically was the hilariously named “Swift” chip that it launched which was the first designed in-house 32-bit ARM chip. This was the iPhone 5, if I recall.
In this chip, they added an integer-divide instruction. All of the chips before that didn’t have the ability to integer-divide in hardware: you had to actually open-code it, and there was a library function to do that. [11:30] That, and a few other instructions they added, were a pretty big deal and used pervasively, and the only way to tackle that and handle that and use it effectively was to introduce an entire new slice. If you’re an app developer back then, you had the “v7” slice which was all the older chips and then you had a “v7s” slice, “s” for Swift, which was the slice that included the integer divide.
This is a pain for app developers. It’s a pain for Apple. Everybody [12:00] has to convince the developers to take it seriously, and if app developers don’t support it then the new hardware doesn’t benefit from it, and so Bitcode allows that kind of thing to go completely away. So Apple can just recompile the applications in the store and say, “Ha ha, this device has an integer-divide instruction, and it works perfectly with the thinning architecture that they have for deploying resources to specific device types, and it all floats together.”
Bitcode is not [12:30] a magic solution, though. You can’t take a 32-bit app, for example, and run it on a 64-bit device. That kind of portability isn’t something that Bitcode can give you, notably because that is something that’s visible in C. As you’re writing C code, you can write #ifdef pointer size equals 32, and that’s something that Bitcode can’t abstract over. It’s useful for very specific, low-level kinds of enhancements, but it isn’t a panacea that makes everything [13:00] magically portable.
John Siracusa: The same thing I would assume for architecture changes, especially if there was an endian difference, because endianness is visible from the C world, so you can’t target different endianness?
Chris Lattner: Yep. It’s not something that magically solves all portability problems, but it is very useful for specific problems that Apple’s faced in the past.
John Siracusa: Going back to the “slice” thing, that’s terminology from the Mach-O format where you get to put different architectures in executables, or is that different?
Chris Lattner: Yeah. I think that’s technology that goes all the way back to NeXT when NeXT supported [13:30] really heterogenous different architectures like PA-RISC and SPARC and PowerPC and Intel, all back in the NeXT days. It allows you to compile your application with multiple slices, and then there’s a tool called Lipo which sticks them together, and at runtime, the OS picks the slice that best matches your CPU in practice. So an app developer can support many different types of architectures, and the OS just does the right thing.
[14:00] In today’s world, it’s very common to build an iOS app, for example, for both ARM64 and ARM32. The way that works, in the simplest case, is that it’s just two different binaries that are stuck together and then the OS picks the right one to run.
John Siracusa: All right. I could go on with this forever but I don’t want to get too far down this rathole. It’s just some pent-up questions about Bitcode and bytecode, but now I feel satisfied and we can move on to broader topics.
Casey Liss: Chris, [16:00] how was your transition from coding to being a manager? Because from an outsider’s point of view, it looked like you had arrived at Apple, you were working heavily on LLVM like you had said earlier, but over time, even just looking at the resumé on your website, it seems pretty clear that you were doing a lot more managing lately, than perhaps, coding or maybe you just worked around the clock. That wouldn’t surprise me either, but how did you feel about your transition from being just a code monkey to being a manager? I ask in part because, [16:30] at my phase in my career, I am still a code monkey but I think it’s not too far away that I might become a manager, and so I’m curious to hear from one code monkey to another: how did that transition go, and what did you learn from it?
Chris Lattner: That’s kind of interesting. I don’t think it is a hard transition. I’m still writing code. I still love writing code. That’s something that I think is a key part of my identity. But on the other hand, it is very strange. I started managing [17:00] people at Apple really pretty early on, maybe a year or so after I joined I started managing two or three people, because I was kind of the best person to manage the nascent LLVM effort at the time. Initially, it was mostly just being a tech lead. If you’ve ever driven a project where you have coworkers that don’t report to you but you’re guiding them and helping make technical decisions, [17:30] that’s where it started.
Over time my team grew and I eventually ended up picking up and running a second level managing role where you’re managing other managers, and each step along the way it introduces new challenges. All along the way, I told myself that managing was something that I had to do because I was the best qualified to do it and I didn’t want anybody else to and that coding was my real joy and passion. I think [18:00] that coding really is my joy and passion but, along the way, looking back, I realize that I am actually pretty good at managing and I really do like building teams. I like working with people, and I’ve never considered myself to be a people person, which is very odd for me to even think about that. But I actually do understand how people work and can really care about getting the most out of people and working with them to make them successful.
It’s been [18:30] kind of a transition over time but I’ve always hung on to coding and even though I have a full-time management job, I code like crazy on nights and weekends. I’m not the kind of guy that likes to sit around idle, so I’ve always just been busy.
Creation of Swift
John Siracusa: How did you get Swift to happen inside Apple? Because from the outside, it’s mysterious to us how big, important projects come to be. [19:00] Like, name the headlining things that have come out of Apple. We hear that Apple does a lot of things that don’t see the light of day. How did you get probably one of the biggest things ever to come out of Apple — an entirely new language, created in-house — how did you get that to happen, and not just be some little thing that was off to the side and just fizzled off and disappeared?
Chris Lattner: Well, I can tell you about Swift, but I don’t think you should project this onto every other project at Apple because I’m sure they’re all different, so I can just talk about my experiences. Swift started [19:30] in 2010. The timing is suspicious because it’s right after a blogger wrote something about how Apple needed a new programming language.
John Siracusa: I am not a blogger!
Chris Lattner: Okay. ArsTechnica wasn’t a thing?
John Siracusa: That’s not a blog! That was a prestigious website. Still is.
Chris Lattner: Okay. Okay. I stand corrected. I don’t remember if I read it exactly then and that was the causal [20:00] link to starting working on Swift, or if it was just because I had just finished up Clang’s C++ support and we had launched Clang C++. Of course, if you write C++ code, you know how horrible and ugly it is, but if you implement C++, it’s a whole other level of challenge. Getting that first release out was a major accomplishment for the entire team, and it was a major effort for a ton of people involved. As is typical, you get something over [20:30] the finish line, of course, it wasn’t fully finished. It still had a long way to go, but a lot of the intellectually interesting (to me at least) pieces were done. That’s when I started thinking about, okay, what comes next?
I’d been implementing other people’s stuff — for example, the C, C++, the Objective-C standards and languages — but there were things that we weren’t happy with and things that I wasn’t happy with. So we started out as just a “Let’s see [21:00] what could be done. Let’s see what a much simpler language that doesn’t need a pre-processor, doesn’t need trigraphs, doesn’t need all the weird things that have accumulated into C and C++ over the years, let’s see what we can do and try to build something nicer.”
Initially, it was really just me messing around and nobody knew about it because it wasn’t anything to know about. But eventually, it got a little bit more serious and I said, hey, after [21:30] playing around a little bit, I think that this actually could make sense. So I started talking to my management and some of the engineers that were working on Clang, and they seemed excited about it. We got a couple people working on it part-time and I convinced my manager that it was interesting enough that we could have a couple of people work on it. Now, this wasn’t a major commitment. This was an, “Okay, let’s see what there is here,” and that started a [22:00] kind of internal demo schedule and other things where we set milestones in place and tried to justify the investment that was being put into it.
Eventually, it got to the point where it was far enough along that it was posing strategic questions. The strategic questions were of the form: We have Objective-C. Objective-C is a great language. It is probably directly responsible for the iPhone being successful because [22:30] it’s really high-performance. It allows you to get your job done. it supports high-level frameworks in a beautiful way. There’s a ton of stuff to love about Objective-C, and while there are a few things that are ugly about it, some “@“ signs and semicolons and other stuff like that, we can make Objective-C better. The question was always: Why not just make Objective-C better? Why don’t we just keep evolving Objective-C? Why do we want to face potential disruption in terms of moving the entire development community to something [23:00] new?
We kicked that around for a long time. We talked about both sides and we came to realize that, yes, we can and should make Objective-C better, and we continued to invest in Objective-C. We did things like ARC, for example, which is a major effort, but…
Marco Arment: We got dots!
Chris Lattner: Dot syntax predated this. The Objective-C 2.0 features and the block syntax were all prior to this. I worked on blocks but it wasn’t [23:30] in this timeframe.
We were talking about, okay, can we just make Objective-C better and can we feature-creep it to the language we want for the fullness of time? Because if we can, that would be much less disruptive to the community. We decided that, yeah, we can move Objective-C a lot closer to what we want so we can get automatic memory management with ARC, for example, but we can’t ever take away the problems that lead to Objective-C being unsafe. The fundamental problem was Objective-C was built on top of C. [24:00] C inherently has pointers. It has uninitialized variables. It has array overflows. It has all these problems that even if you have full control of your compiler and tool stack, you just can’t fix. To fix dangling pointers, you would have to fix lifetime issues, and C doesn’t have a framework to reason about that, and retrofitting that into a compatible way into the system just wouldn’t really work.
If you took away C from Objective-C, you couldn’t use C arrays on the stack, for example. And if you [24:30] couldn’t do that, there’s entire classes of applications where the performance just wouldn’t be acceptable. We went around, around, around. We said the only way that this can make sense in terms of the cost of the disruption to the community is if we make it a safe programming language: not “safe” as in “you can have no bugs,” but “safe” in terms of memory safety while also providing high performance and moving the programming model forward. That was really kind [25:00] of the ideas that came together to make Swift being worth the investment and being worth being disruptive to the community. A lot of these kinds of pitches and ideas were being held in very small, small, small meetings. Coming out of WWDC 2013 is when we and the executive team decided okay, let’s really commit to this, and that’s when the developer-tools [25:30] team came to know about it and really started working hard on it.
Swift is a huge, huge effort by hundreds of people and that’s when all the work on bringing up the debugger, the IDE support, Playgrounds, and all the other things that charged into Swift 1.0 really started coming together and make happen. That’s kind of how Swift came to be. It was, again, one step after another building small things, explaining to people why it made sense, [26:00] why it was worth it.
Also, I think that a major, important aspect of Swift being worth taking a risk on, was that we told people that we would not guarantee source compatibility in 1.0. That was really interesting because that actually was a big relief for people, because internally that meant that it didn’t have to be perfect when it launched. It meant that we could learn, we could adapt, and that as long [26:30] as we were clear with the community about that, that would allow us to get to ultimate greatness and learn from actual usage and practice. I think that was a really key piece to Swift coming and actually making it to the market.
John Siracusa: Strategically speaking, when you were doing the work to improve Objective-C, did you have in mind the ability to potentially leverage some of that work for Swift? I’m trying to think about the strategy of how you get it done. [27:00] Part of it is personal relationships and knowing the right people. Part of it is persuasion: you have to persuade them with your technical arguments and your business arguments and stuff like that. But was part of it also laying the groundwork with efforts and technologies that have multiple uses?
Chris Lattner: Yeah. Absolutely. One great example of that is ARC. It was really clear to me that if we were to get to memory safety, we had to have automatic memory management. I don’t know if you want to go down the GC-versus-ARC rabbit hole or not, but [27:30] ARC seemed like obviously the right model to me and to some other people. So we said, “Okay, we need to get to an automatic memory management model.” Objective-C isn’t, because at the time it had the Obj-C garbage collector which had numerous problems, so we have to get Objective-C to be memory-managed. That’s why ARC became a thing. It became a really high priority, and I think that it’s a great [28:00] example of something that, by itself, had a huge amount of value to the Objective-C community. Then, after that, we had similar other efforts like modules, for example, that was a huge thing that both improved build time but was also absolutely essential to enabling Swift code to eventually just say “import UIKit” and get everything.
There’s a number of things that rolled out over the years, and it was really funny at the time because the Objective-C developers externally and even internally to Apple always accused the compiler and [28:30] language team of taking Objective-C on this random walk, and they didn’t know how it all fit together, and where’s the strategy, and why are you guys working on this and why aren’t you working on some other syntactic sugar that I want? Of course, we could never tell them, but there was a good reason.
Another example of that is the Objective-C literals feature. I think we introduced that in 2013 if I recall, for array literals and other things like that. That was an interesting feature in that it was [29:00] a relatively small amount of engineering work that brought the Objective-C language much closer to Swift in terms of having arrays and dictionary literals as part of the language. But the real reason for doing it is that it was a small amount of work that moved Objective-C forward, which allowed the compiler and language team to focus on Swift because Swift was a very large amount of work. That was appeasing the community and [29:30] solving real problems, but buying time to actually pour into the multi-year project of making the Swift language happen.
John Siracusa: That’s when you pull out the syntactic sugar. The programmers love it. It takes ten minutes to implement. Here you go.
Chris Lattner: That’s exactly right.
John Siracusa: Everyone applauds when they see the slide.
Swift’s initial reception
Marco Arment: In all fairness, that was a really big deal! Anyway, building on the Apple internal reaction, this language in its development and then its public release is probably [30:00] a fairly unique language in the context in which it was released. Nobody probably had more Objective-C programmers than Apple does, so you have this massive company with tons of internal programmers and a massive internal code base that is using this other language. You have this entire community of iOS and Mac developers. Then you have the entire world watching every move Apple makes and putting it under very heavy scrutiny. So the release of this language in [30:30] this context is probably not only very high-pressure, but also unique in that you knew that you were going to face a lot of people upfront.
Most languages probably start out with a few people using them in some random corner of the internet, and eventually maybe it slowly takes off. This was not that way. What considerations went into that, how were you able to convince people even inside of Apple to use Swift, [31:00] and how is that going?
Chris Lattner: There’s different answers for that for different phases. We had no idea that the reaction would be as strong and as positive as it was, so I think we were blown away and caught unaware in some ways. But in other ways, I think we did the right thing. For example, if you look at Swift over time, Swift 1 [31:30] and Swift 2 are as similar to Objective-C as we could make them. If you look at the way that selectors are named, for example, there’s a very direct correlation. If you look at the way Swift 1 was explained to people, it was explained to people as, it’s just like Objective-C. You can use all your same patterns. You can use all the same frameworks. You can continue doing objecting-oriented programming. You can do all these things, but it also has some cool new things [32:00] like enums and pattern-matching and these other things. It’s great because it’s just cleaned up syntax for what you already know plus it has some cool new stuff that you can play with.
Then you fast forward a year or two and the Swift community was really taking off. People were understanding what Swift was about. Swift open-source also enabled this. And Swift 3 is really about Swift becoming its own language that was really true to itself and stands by itself. That’s [32:30] where you see the method call syntax, for example, being radically changed and a lot of things coming together and really focusing and polishing the language.
In terms of the rise of the external community and people using it, that was really, really hard for us. My experience with LLVM and with Clang — Clang as a C and C++ compiler and Objective-C — was completely [33:00] different than Swift, because when we open-sourced and launched Clang at Apple, everybody yawned. We open-sourced it very early on and people said, “Okay, you’re nuts. Nobody can build a C++ compiler. That’s not a thing. Why are you even messing around with this?” And we got one contributor to the project, and we had no users. A month later, we might get another contributor or another person part-time sending in patches. [33:30] It was a very slow growth that was ultimately really great because it was relatively low-pressure.
With Swift and with the way it was launched, it was completely different, because it went from nobody knowing about it to everybody knowing about it overnight, literally, and there being so much excitement, so many people that were nervous and had bad reactions, other people who were super-excited, a lot of people that were suddenly hitting bugs, having challenges and problems, [34:00] and “Why did you do this and why didn’t you do that?” Some of that was misguided just because they didn’t know Swift yet, but other aspects were really spot on. That was when we really started learning about how people were using it and started iterating and moving Swift.
Between the WWDC launch and Swift 1.0 release, Swift changed dramatically, and that was a really, really hard time for the entire team because we really wanted to get it as close [34:30] as we could to what we wanted but we had a very short amount of time to do so.
I think that’s been true for Swift 2 and Swift 3 as well, where we have really high goals for the team and really high goals for what we want to get done and that makes it a lot of pressure and a lot of stress. But it’s, I think, worth it to serve and see the community around Swift grow and see people use it and see people learn it. It’s been a lot of fun.
Swift adoption at Apple
Marco Arment: I’m curious, [35:00] I know when Swift first came out it was hard to get a lot of adoption inside of Apple for various tooling reasons, among maybe whatever else was going on. How is internal adoption going today? Are we seeing meaningful adoption now, or is it mostly iOS programmers who are using it and not anybody on the inside?
Chris Lattner: The Swift team itself has specific goals they need to achieve before there can be truly, across-the-board adoption at Apple. ABI stability is the number-one thing [35:30] that prevents framework developers, for example, from adopting Swift. That’s a really important thing. That’s one of the reasons it’s always a really high priority. Swift has been adopted by application developers and other things. The Dock is public. Swift Playgrounds app is public. The Music app in iOS is publicly known. So there are definitely some big adopters.
More broadly though, the big problem is that I think, I won’t speak for everybody but many, many people doing [36:00] Objective-C development at Apple are chomping at the bit. They want to be using Swift. It’s really just a matter of getting the technology problems solved and checking off the things that are holding people back. It’s not about people dragging their feet and not wanting to use it.
Casey Liss: From the outside, it’s so hard to figure out, is Apple internally like, “Ugh, really?” Or is Apple internally, like you said, really excited? I’m sure that there’s some of both, and that’s really good to hear.
Chris Lattner: I think [36:30] it has also changed over time. When Swift first came out, of course, it was a huge surprise for most of the software team, so many people went through the usual phases of, “I already know Objective-C. Why would I learn this new thing? Objective-C is great.” And Objective-C is great, so that’s an objective statement. But I think things have shifted over time, and as Swift has matured, it’s definitely become a lot more appealing.
One other technology problem [37:00] that is hilarious but also really important is that the Apple frameworks stack has to support 32-bit Mac apps. 32-bit Mac apps have this interesting challenge: they have the “classic” Objective-C runtime, which doesn’t support things like non-fragile instance variables and things like that. At some point in time, the Swift team will need to make the Swift runtime work in that mode, or figure out some other solution to adapt it, because until that happens, it won’t be [37:30] possible to use Swift in AppKit, for example.
Achievements at Apple
Casey Liss: That’s fascinating. So, taking a more broad view, could you name either your most proud accomplishment during your time at Apple (that you can discuss publicly anyway) or, if not your most proud, just the first one that jumps to mind?
Chris Lattner: Wow. I don’t know. I’ve been really fortunate to work on so many really important projects. I think that between [38:00] LLVM and Swift, for example, it’s really a hard call. Swift, I think, impacts more people in terms of the number of people that actually know they’re using it and want to learn it and things like that. But I think that LLVM has been picked up and used so far across the industry and is standardizing the world’s compilers. I think that that’s probably more profound at this point. It’s really a hard call. There are a ton of [38:30] really, really great opportunities that I’ve had and it’s been a lot of fun over many years now.
Also, outside the open source pieces, working with the Xcode team and working with the other teams in Developer Tools has been really phenomenal. There’s so many great people at Apple and in Developer Tools, it’s just been a joy.
John Siracusa: What do you expect to miss most about being at Apple? I’m going to let you expound on this but I’m going to take it off the table and say you have to pick, like, your second choice. You’re going to say you miss the people because of course you do.
Chris Lattner: [39:00] Yes. That is actually what I would say.
John Siracusa: Right. So go ahead and tell us about all the great people you worked with at Apple, but besides that, what do you expect to miss most about Apple, the company, working for them?
Chris Lattner: Well, the great thing about Apple, and the thing that I don’t think many people really get about Apple, is that it is really one of the few best places in the world where you can work hard, but also work on products that ship, and that ship to millions of [39:30] people, so that your hard work ends up mattering.
There are a lot of places you can work hard. There’s a lot of places that you can work on interesting things. But there’s also a lot of places that end up not shipping a lot, or if they ship it, very few people actually use it. Apple’s really magical in that it combines being able to work hard and do great things with it actually mattering, and to me, that’s the thing that is just so [40:00] phenomenal about Apple.
Swift and Tesla?
Marco Arment: Moving on to your new role at Tesla, do you think that Swift will still be a major part of your work life in some way? Or will it basically be something you do in the evenings and weekends? We all know that Silicon Valley tech start-ups don’t usually work people too hard so you’re going to have lots of free time.
Chris Lattner: Yeah, this jump I’m making is definitely to a cushy, easy job with no stress. That’s what I’m looking for here.
I [40:30] don’t know. We’ll have to see. I haven’t started yet, and I think it’s going to be a very challenging job. One of the reasons I’m excited about it is because I really want to throw myself into another hard technology problem, so I’m really excited about that.
I do expect to be involved with the design of Swift and helping with the core team and helping with the Swift-evolution process and helping in all of those respects. One of the things to keep in mind is that Swift was actually a really tiny slice of my job [41:00] at Apple. I had a very full schedule dealing with lots of other stuff, and it was already kind of a nights-and-weekends project that I was coding on it and contributing at the level that I was.
I’m hoping that there won’t be a big change but, honestly, I’ll have to see. Obviously, I care a lot about Swift and I really want it to get to its goal of world domination.
Casey Liss: Obviously Swift is open-source, I don’t know if you were aware of that, Chris, but …
Chris Lattner: Oh, I read about that.
Casey Liss: Right? How do you feel that it’s been going so far? Similar to John, I’m going to take away the obvious answer of, “It’s going well,” but maybe to [43:30] cue conversation, what’s been really surprising about going open-source? You can get as specific or as broad as you’d like, but I have to imagine that you guys had a bunch of expectations about what going open-source would be like, and I would imagine some of them were accurate and some of them weren’t. So how do you feel that the Swift open-source project has been going?
Chris Lattner: I think that to summarize, it’s not just going well, it’s going far better than we ever anticipated — and that caused problems. [44:00] For context, I and many of the other people on the team had quite a bit of open-source experience: LLVM and Clang, for example, were open source. They are vibrant and amazing communities with hundreds of contributors and dozens of companies all collaborating in public and doing really amazing stuff. Even in some of the most staunch competitors, AMD and Intel or AMD and NVIDIA or these [44:30] kinds of companies that really don’t get along on the business side of things, the engineers were great at working engineer-to-engineer, solving problems, and staying focused on making things better.
With Swift, we assumed that it would be very similar to that. But what we found is that there was a major difference between launching Swift and launching something like Clang, for example, which is that when Swift launched as open-source which was December 3, 2015 (just over a year ago), [45:00] it already had, I don’t know, a million users. Where Clang started and everybody yawned, didn’t really care, and grew slowly, Swift open-source started and was amazingly fast-moving, changing, tons of people, hundreds of patches on the first day. It was totally crazy and we didn’t really know what to do. I am super-impressed with the Swift open-source community and all the people involved [45:30] and the energy.
It’s hard to believe how great everything is, but it really did create problems because we had to figure out things like how are we going to keep up with the patch volume? Okay, we’re getting hundreds and hundreds of emails a day to the Swift-evolution mailing list, what are we going to do about that? How do we balance Apple’s goals for pushing Swift forward with the goals of the community which are somewhat more chaotic and less directed?
I think that over the last year we’ve [46:00] learned a lot, and you can see that reflected in terms of how the project’s being managed and how things are being done.
But it’s always been super-important to not just be open-source. If you look at an open source project, there’s three easy levels to look at. The first basic requirement that you have to satisfy to be open-source is to make your code available. You can produce tarballs periodically and that’s technically open-source. You [46:30] can allow contributions, and if you do that through pull requests or patches or whatever, then people can engage with it and make the code better. There’s open-community, so not only do you accept a patch, but you discuss things and talk about things in public. Then there’s open-design, and with Swift, we went all the way, and I think it was totally the right thing to do, because we really wanted to benefit from all the smart people in the world. It was (and still is) amazing, [47:00] but it’s also very challenging.
Casey Liss: I can imagine. Building upon the open sourcing of Swift, it was very surprising to me (and I think a lot of the community) that Apple chose to upload the entire commit history. It wasn’t just, “Here’s day one, initial commit” and it’s 11 gazillion files and 80 gazillion lines of code. Was that planned from the start that you guys would upload the entire history?
Chris Lattner: I wouldn’t say it was planned from the start that we would upload the entire history, but it was kind of assumed [47:30] from the start that it would be open-source at some point.
Casey Liss: Interesting.
Chris Lattner: If you look at Apple’s history in the tool space, I think that’s fairly straightforward. LLVM wasn’t started at Apple, but almost everything Apple does ends up going public in the LLVM space. Clang was started at Apple, and Clang was contributed by Apple to the LLVM project, and that included its [48:00] full revision history. Other projects like LLDB and libc++ and compiler-rt and all these other smaller projects were started at Apple and contributed to the open source project.
So with Swift, it was assumed that that would happen at some point. The real question was, when is the right time for that to happen? If you remember the days of Swift 1.0, Swift 1.0 to 2.0 in particular was a super-chaotic, super learning-as-we-went, trying to keep [48:30] up with the community, trying to keep up with the pace and the challenges that people were facing, and trying to get key pieces like the error-handling model and protocol extensions and that kind of stuff that had always been planned but didn’t make it into Swift 1.0, so really focusing on that was the priority. We decided that open-sourcing Swift at 1.0, for example, would be really actually bad for the Swift community because there were major pieces of language that weren’t there. We would have to spend a lot of time community-building [49:00] and trying to explain to people the difference between what was not there that should be there and what is there and known-broken.
We had to do a lot of that when Swift was eventually open-sourced last year right after Swift 2.0, but it was in a much better place and the team could actually handle the load of doing the community. We didn’t expect the kind of response that we got, but we knew that building a vibrant open-source community is a lot of work, and it’s not something to be underestimated. We wanted to make sure that Swift [49:30] 2 was out the door and it wasn’t a resource contention between, do we finish Swift 2 or do we start open-source? That’s why it was announced to be open-source by the end of the year, because that meant that we could start working on the open-source infrastructure project, bringing up all the other things that it takes to make something open-source, after Swift 2 is complete.
Swift 3 and ABI stability
John Siracusa: My recollection of Swift 3 in the planning stages is that ABI compatibility was somewhere [50:00] in the list of, I don’t know if you want to call them “goals”, but somewhere on the list of items that were in contention for Swift 3. At some point, along the development process of Swift 3, ABI compatibility got postponed. Given what you just said about the importance of ABI compatibility for the framework people inside Apple or whatever, what can you say about that? Did you learn lessons from the ABI compatibility being planned for Swift 3, or do you think it was never really a [50:30] hard-and-fast goal and you totally made the right call to delay, or are you disappointed that it didn’t make it?
Chris Lattner: I think many of those things, we’re disappointed that it got delayed, but it was totally the right thing. I think that there’s a couple of things to learn from that. One of which is software schedule planning is hard.
Marco Arment: Breaking news.
Chris Lattner: Yeah. That’s definitely still a thing. There’s also that we didn’t anticipate — the original plan that you’re [51:00] referring to was announced the same time Swift was open-sourced, and we had no idea that the community would descend on Swift and suddenly have all these new demands and consume so much time and energy that took away from programming to do the community-building piece. That was part of it.
Another part of it is that ABI stability is super-important, but it’s not as important as people think it is for application developers. It’s really important to Apple, [51:30] but what we realized in the Swift 3 timeframe is that the thing app developers would benefit from the most was actually source stability. Who actually wants their application to be broken when they get a new version of Xcode? Really nobody, right?
Halfway through the release, we pivoted and source stability became the goal, so I’m really excited that when Swift 3.1 or Swift 4 comes out that it’s still going to be able to build [52:00] Swift 3 code, and even if there are minor changes that need to be made for one reason or another, that you can upgrade and you have great compatibility with your old code and you don’t have to start the migrator before you can do anything. So it’s going to be a great improvement for people’s lives.
Server-side Swift and expanding Swift’s relevance
Marco Arment: Moving on to a branch topic of server-side Swift, Swift on Linux, etc.: [52:30] I know Swift on the server is possible and there’s seemingly some early efforts now to get that going. Do you see this as a major goal for Swift as the language? Not in any official capacity, but just you as a person who likes and uses Swift and has a strong role in the community, is server-side Swift something that you want to be a major thing to compete with Python and Rails and other stuff on the server, or is it [53:00] intended more to be a specialized thing that is not going to be in widespread use by web and back-end programmers?
Chris Lattner: I think that everybody involved with the project wants it to be the way to do server development. I think that to understand that, you have to understand we’re just entering phase two of Swift’s world domination plan so this will probably sound insane, but maybe in a few [53:30] years it will sound less insane. The way that I look at it, and the way that many other people at Apple look at it, is that we want Swift to be the next big thing. We want it to be more popular than Java was or is. We want it to be more popular than Python. We want to be maybe even more popular than C someday. That would be a stretch, but that would be great. If that’s the problem to solve, how do you solve that problem?
Well, open-source is a huge piece of that. Without open-source, it’s not [54:00] going to be available on other people’s platforms, it’s not going to be used in education and teaching, that’s just not going to happen, so open source is a major piece of that. If you go back farther, before that, Swift world domination doesn’t happen unless Swift gains relevance. I think Swift has nailed the relevance side of things with the iOS and Apple platform development side of the story where maybe, not Marco, but most other people are [54:30] switching to Swift very quickly and seem very happy using it.
Marco Arment: Actually, I started using it today.
Chris Lattner: Okay. Good call.
Marco Arment: I switched today in honor of this interview. I wrote my first Swift today. An entire extension in Overcast 3.0 is written in Swift.
Casey Liss: Wow. Look at you go. I’m proud of you.
Chris Lattner: See? Fantastic. It is a good day.
Marco Arment: And more will be coming, although there’s a lot of Objective-C there, so I’m probably a long way away from it being all-Swift.
Chris Lattner: That’s okay. Swift loves Objective-C.
Marco Arment: [55:00] Exactly.
Chris Lattner: The way I look at it is, the first step of Swift world domination is getting to relevance, and for a language to get to relevance, it needs a killer app. It needs a reason why people would want to use it. For Swift, that’s clearly iOS and Mac and other Apple-platform development. Second, it needs to be open-source. Third, it needs to go beyond.
The next step, the frontier of where it makes sense to expand out to — or if you’re a [55:30] real-time strategy game player, the natural expansion — is a place where the language actually only needs a few things added to it to be good. I think server development is really that next step, because server development is really similar to app development in terms of the kinds of code that you need to write. It’s very similar in terms of many of the frameworks. The major missing piece is that it needs to run on Linux and other pertinent [56:00] server development platforms, and then there’s the whole library ecosystem that needs to be built out.
I think that what we’re in right now is that we’re in a really exciting phase where those platforms are being built. Those frameworks and different pieces of infrastructure are coming together. That’s really the idea behind the Swift Server APIs working group that’s an official part of Swift.org, getting all the people in the space to talk to each other, and I’m very excited about that.
Going beyond that [56:30] though, with the goal of being more relevant than Java, for example, if we’re going to pick on Java, the dream of Swift is to scale all the way from scripting language all the way down to low-level systems programming, like you could write a kernel or some firmware in it.
I think the scripting side of it is maybe more straight-forward when the Swift community and the core team have time to deal with it, because they’re talking about [57:00] integrating regular expressions, multi-line string literals, and other features like that that people love from scripting languages. Those are big projects, but those are kind of well-known, I guess, and they will fit and drop right into the existing system that Swift provides.
Systems programming is, I think, the big next frontier, and I think that’s where Swift can really distinguish itself from Java or [57:30] other competitive languages, because among the widely used languages out there, there’s really nothing that can unseat C or C++. Rust, for example, is an interesting language, but it hasn’t gotten a lot of adoption yet and I think that Swift is more interesting than Rust in some of the higher-level application demands, but we’ll see. Rust is a great language as well.
I’m really interested to see the Swift-for-systems-programming [58:00] work get going. I think that’ll be a couple of years out before it really comes together, but once it does, I think that Swift can really be the next big thing and I’m excited about that.
Marco Arment: Stepping back for a second to the server role, here I am using PHP and Objective-C, these two languages that are fairly old (and PHP is way more hated than Objective-C), but these are the two I use [58:30] for various historical and practical reasons.
A lot of what makes a language succeed or fail on servers and in web development is not necessarily how good of a language it is, but how healthy, universal, and developed the frameworks around it are for things like web development, and how practical and easy it is to deploy and run on servers. So Swift being its own [59:00] good language, that alone I don’t think is enough to get traction on the server. Do you think it’s necessary for the Swift team, the core group who’s running Swift, whether that’s Apple or the community of contributors with Apple, to develop an official set of server and web frameworks, libraries, and tools, or do you think it’s sufficient [59:30] to let the community do that, but at the risk of fragmentation and maybe nothing ever taking off?
Chris Lattner: That’s an interesting question. Keep in mind that I don’t have a lot of practical experience building server apps…
Marco Arment: You’re not missing much.
Chris Lattner: My opinion based on talking to other smart people, the approach the Swift project is taking is it’s looking to standardize on what’s colloquially but not correctly known as “server foundation” [1:00:00] technologies. It will not be named Server Foundation, but that kind of the idea: take the encryption, network transport, HTTP, kinds of things that everybody needs for server technologies and standardize that as part of the Swift project, but at least in the short term, not look to standardize a specific web framework, because there are a half a dozen completely different approaches that are very interesting in different demands, and I don’t think there’s a clear winner yet. [1:00:30] It’s possible that after the first few rounds shake out, then there will be an equivalent Ruby on Rails kind of dominant pattern and framework that emerges. If so, maybe that would make sense, but I don’t think Swift needs for there to be a winner at the high level.
As far as what can Swift core language and compiler folks do about this, I think there’s really two things, one of which is the Swift package manager. I think [1:01:00] the package manager is really key to scaling out the ecosystem and getting people to work together with a common ecosystem. It’s also the best (and, I think, only) way to build Swift code on Linux and other platforms. I don’t mean to single out Linux. It’s just the biggest server development one. I think that’s a big piece.
I think the other piece that’ll be really amazing is when the concurrency model comes out, because I think one of the reasons that makes Go [1:01:30] particularly compelling for server development is it has really well-developed story there along with all the libraries that go together for server and cloud development.
When the pieces of the concurrency model start coming up, which will probably — the first pieces will ship in Swift 5 is my guess. That will make Swift an even more compelling answer for that kind of space. I don’t think we need a server language feature, if that’s what you’re asking.
Systems programming in Swift
John Siracusa: Going to the other extreme, from [1:02:00] the server side and scripting to getting back to systems programming, one thing I’ve seen related to this, and I think this all stems from this quote from the original Apple Swift book, that Swift was designed to scale from hello world to an entire operating system. The world domination plan was there from the beginning, right, in the first paragraph of the thing?
Chris Lattner: Absolutely.
John Siracusa: When I talk to people now about Swift as a systems language, often what I hear [1:02:30] is people relating the ways that it is not ready to be a systems language yet, which you just talked about, but then deciding based on that that because it’s not ready to be a systems language now, it never will, and then they dismiss it. In other words, that may have been the plan, but clearly for reasons X, Y, and Z, you still have to use C for this, therefore Swift is just a way to write apps for iOS and Mac OS.
I’m [1:03:00] not quite sure what you can do about that, but do you think about how to change perception as the language gets older in it’s old age? Now it’s up to 3.0. Nobody puts Swift in a corner. People think they know what Swift is, and they say it’s this language that does this thing, and that your plans for world domination may be thwarted that people are pigeon-holing the language because it’s been around for a few years and they think they know it.
Chris Lattner: I think that you’re onto something. I think that’s a real problem. I don’t think systems programming [1:03:30] is a place where that’s a problem yet, though, because I don’t think it’s reasonable for me as a crazy man going around saying, “Swift will someday do systems programming really well. You should remember that.” I don’t think that’s a good way to spend time at the moment.
When it does that, then the question becomes how do you make people aware of that and how do you get people to change their prior misconception? But at that point in time, you can have a very practical, “Look, you can do this, [1:04:00] and this is so great, and now you get all the advantages of C but it’s safe.” You can point to specific reasons why it’s better.
I’m not too worried about it in the space of systems programming, but I think there is a practical problem today which is Swift is open-source. The Apple development community knows Swift very well, but pretty much everybody else doesn’t. If you go to a random, say [1:04:30] Linux developer, just to continue taking about Linux, they all perceive Swift as being an Apple thing and only relevant to Apple platforms, and they don’t even learn about it. I think that there is an interesting challenge there, and probably the best way to do that is for the existing communities to keep just growing and as the Swift-on-server community grows and more and more people are successful building and deploying server apps written in [1:05:00] Swift, I hope that naturally more and more people will become aware of that effort, will then find out about Swift, maybe try it out, find out that it’s actually really great, and if it’s not really great, let us know so that people can fix whatever the problems are, and then just grow out. I’m actually perfectly okay with it growing a little bit slower than it has been.
Progressive complexity: from novices to experts
Marco Arment: John asked about the span between basically low level systems programming and high level applications programming, I’m also curious [1:05:30] that it seems like the Swift team/Apple have presented Swift as not only spanning that wide range, which seems impossible, but also spanning a wide range of skill levels of programmers. There’s the Swift Playgrounds app on iPad where you have total novices writing Swift as their first language, all the way up to people writing systems-level things and complicated apps who are very experienced programmers.
Do you think it’s possible for a language [1:06:00] to do all of these things well? If so, do you think Swift has achieved that?
Chris Lattner: I do think it’s possible. I think it has achieved it for the things that it’s doing so far, and I think the challenge is that as new capabilities get added, that it doesn’t jeopardize that. But knowing the folks involved and the source compatibility guarantees and everything else like that, I’m pretty sure that won’t be a problem going forward.
Here’s the way I look at it. You’re unlikely [1:06:30] to run into anybody that says that Swift is a simple language. Swift is a complicated language. It’s not simple in the sense that Scheme is simple, for example. It’s also not simple in terms of when you talk to an Objective-C developer, they’ll often claim that Objective-C is simple, ignoring all the C parts.
Marco Arment: I would never claim that, for the record.
Chris Lattner: Right? C itself is a very complicated, weird language. It’s just that people don’t typically use the weird parts, and so they perceive [1:07:00] C as being very simple. The way I project this onto Swift is that the secret to Swift and being easy to learn, easy to use as a teaching vehicle, but also powerful enough to solve the problems that need to be solved, is that the complexity inherent in the language needs to be progressively disclosed.
Imagine if someday Swift had inline assembly support, for example. Only really low-level systems people or [1:07:30] really bit-twiddling assembly programmers would use this feature, everybody else would ignore it, and it would not increase the apparent complexity of the language to people who didn’t know what it was. For a feature like that, the most important thing is to make the syntax clear so that when you run up into it, you discover it in somebody else’s code, you know that you don’t know what it does. So you say, “That’s interesting. I don’t know what that is,” and then you can do a search and find out, find the Stack Overflow article or whatever it is, to explain what this is [1:08:00] to me. And then you can say, “Oh, I get it,” and I either care or I don’t, but at that point in time you can dive in and learn more information.
Assembly is never something that a Swift Playgrounds person working through Learn to Code 3 would ever want to know about. I think that one of the secrets to Swift’s success in terms of getting people on board and being really approachable for people at all levels is that, just like in Python, you can start with just print(“Hello world”). You [1:08:30] can do the exact same thing in Swift, and with Swift you don’t need the “\n” on your string. It’s really just print(“Hello world”). From there you can then introduce new concepts, you can grow it out over time, and this really allows people to understand either the syntax of the language or the conceptual programming things that they’re learning as they go. The design of Swift is really geared around that.
What that means if you project onto the systems programming features, comparing Swift to Rust, for example, [1:09:00] I think it’s very likely that Swift will get features for memory-ownership control, which will allow really, really high performance: it will allow solving performance problems with ARC, for example. Unlike Rust, we can’t make that be a core part of the type system that everybody has to use. It has to be something that sufficiently smart programmers, when they’re solving a specific performance problem, end up using, or an embedded-kernel programmer might want to use, but an application developer can [1:09:30] completely ignore. That’s really the challenge: it’s deploying similar type-system mechanics and other low-level language geekery to the problem, but do so in a way that is tasteful and allows most normal people to just ignore it.
John Siracusa: Everything you just said kind of reminds me of a saying that I once heard, that you’re free to adopt for Swift if you would like, which is: Make easy things easy, and make hard things possible.
Swift language influences
Marco Arment: Well, I see Go in it. I don’t see any PHP [1:12:00] in it!
Casey Liss: Alright, fair enough. You get what I’m driving at, though. I guess what I’m asking, Chris, is obviously there was a huge Objective-C influence in Swift, but was there any one or maybe a couple of other languages that you feel really, really inspired Swift, or do you really think it’s the melting-pot language that one would always hope to create?
Chris Lattner: I think it really is a melting pot, because the goal of Swift isn’t to ape some other language, the goal of Swift is to just be great. [1:12:30] I think that the people in the core team, the other people working on Swift, have looked at many, many other languages. If you’re a Haskell nerd, there is tons of Haskell concepts built in. Protocols really are just like a very similar Haskell construct, for example. It’s really about taking the best ideas from where we can get them and assembling them together. One of the major goals of Swift is for it to be familiar-feeling, [1:13:00] because that makes it easier for people to pick up and they’re not fighting unnecessary barriers that just get in the way of adoption.
It being familiar is actually a success in that way, and it’s not a result of Swift trying to be Go or C# or whatever, it’s about taking the best ideas from them and assembling them together. I think that you can probably pick a language and there’s some good idea that came from them [1:13:30] including D or Dart or Go or whatever. Swift does really draw from many, many different sources and it’s really hard to tease them all out.
Why adopt Swift now?
Marco Arment: So the question that Casey wrote for me to ask you right now…
Casey Liss: Whoa, whoa, whoa, whoa. How do you know that was me?
Marco Arment: That’s definitely you. Is: Why should I give a sh*t about Swift right now? I think I’m going to modify this a little bit.
Casey Liss: That was written for you, [1:14:00] my friend.
Marco Arment: Definitely by you, right? I figure that’s not quite John’s style.
Casey Liss: Yes it was.
Marco Arment: As somebody who is just dipping a toe in Swift, as of 12 hours ago, and has waited this long, you know, Objective-C faithful/holdout/lazy people who don’t like learning new things very quickly. What’s the sales pitch for adopting Swift now?
Chris Lattner: I guess, two different answers. One of which is, you don’t have to adopt Swift. I don’t [1:14:30] think Objective-C is going to go away anytime soon. Apple still supports C and C++ and there’s no obvious benefit of dropping Objective-C, and obviously they have a ton of Objective-C code themselves. I don’t think that there’s any reason you have to adopt Swift. Swift just tries to be a better answer so that you’ll want to adopt it.
In terms of why, Swift, in its name, kind of connotes a whole bunch of different [1:15:00] things including performance. But really, to me, the most important aspect is programmer productivity.
The idea that Swift is really optimizing for is not letting you pound out the code as fast as possible — it really is designed and optimized for, as a programmer, you can spend the least amount of time to get to a working program as fast as possible.
Getting to a working program includes the time it takes to debug something, to iterate and develop and fight the type system, and all the other things [1:15:30] you have to do. But that time you spend having to fight the type system or figure out optionals or things like that pays itself back because your program works more often. You’re actually not spending time chasing own dangling-pointer bugs or, in the case of Objective-C, you get an unrecognized-selector error, whatever those kinds of bugs are, and Swift tries to help you by the language guiding you and helping design things.
Swift also has [1:16:00] a lot of things that just make common things easier. You can switch on a string, for example, and match the different cases. You can use enums — enums are just from the language geekery world of just algebraic data types. They’re in tons of different functional languages. They’re not novel by any means, but they’re so useful and they solve so many problems just out of the box. Just having them at your fingertips is great. There’s a lot of things about Swift and it probably depends on who you are, what you’re experiences [1:16:30] are.
One of the problems that I’ve seen with people moving from Objective-C to Swift initially is that they tend to write Swift code just like they wrote Objective-C, and while that’s a really good way to maybe migrate an application over or to get familiar with Swift, it really means that you’re missing a lot of the value in Swift. I think that that’s okay, it just means that an Objective-C programmer going to Swift first starts out by saying, “I don’t see what the big deal is,” and only after they spend some time in it do they start to realize [1:17:00] they are more productive. They do feel better. They reach for things that they otherwise wouldn’t have.
I think it’s really interesting to talk to people that have done, for example, a year of Swift programming but then have to go back to Objective-C. To me, that the really interesting thing.
Casey Liss: I’ve been there.
Chris Lattner: What was your experience?
Casey Liss: It felt absolutely archaic. Granted, I like to think of myself as a nice guy and so I’d probably say something like that even if it wasn’t true, but the reality of the situation is, it felt just so [1:17:30] old. It just felt old. I wish I could quantify that better but it just felt like I was back in time in a way that didn’t feel good.
In some ways, it was kind of nice. I do like optionals. I understand why they’re a thing. I am all in on it. But, at the same time, there are times when I have to do a guard-let or whatever the case may be that I really would just like to move on with my life.
Being able to be a little [1:18:00] more cavalier was kind of nice, but the reality of the situation is, as you said earlier, that that’s a potential hazard down the road and that’s why I like the way that you and the team have set up the type system in Swift. Going back to Objective-C is so frustrating for me because it just feels like I’m going back in time, like I’m going backwards.
Chris Lattner: Yeah. I think the challenge that Swift faces is because its focus is on building programs that work and that can be maintained [1:18:30] and all the other things that we care about for large scale software, it’s maybe not the best solution for, “I want to pound out a quick script, use it once, and then throw it away.” In that case, you don’t care about maintenance. You can fit all the code in your head. You don’t care about good documentation or the other things that Swift encourages. That’s a case where having a dynamic type system can be really liberating, so maybe that’ll [1:19:00] be a challenge for some people. But I think that as Swift gains ecosystems and libraries, its other advantages will be so great that hopefully it will be a good solution for those kinds of scripts as well.
Marco Arment: A minute ago you mentioned how Objective-C programmers, people like me who are new to Swift, tend to write Swift like Objective-C. That’s a concern I’ve had, and it’s one of the reasons that I’ve procrastinated adopting Swift up until now: [1:19:30] I have a pretty good grasp on Objective-C now, and I write Objective-C code that I know is basically idiomatic and soundly designed. I write code in the style of what I think Apple’s code would look like, based on the little bits of headers and stuff that we see. With Swift, I assumed that the idioms would not yet be developed and established for a while, [1:20:00] so I didn’t want to write a bunch of bad Swift code, or incorrect or weird Swift code, by investing heavily in it so early on. Do you think it’s at a point now where the idioms are starting to get worked out enough that that’s less of a concern now?
Chris Lattner: Yeah, I do. I think it’s definitely fair to say that in the Swift 1 and 2 timeframes, Swift as a language was changing really rapidly, and [1:20:30] I could see why you’d feel like you’re on uneven footing and not really sure what the language is, much less what the idioms are. Swift 3 really is quite well baked out, and I expect that going forward the new things are going to be additive, not changing the existing patterns. I think the Swift community in general has gone through the same kind of thing that you’re feeling there, where it’s not really clear the one right way to solve the problem, and maybe there are different good ways to do it with different trade-offs that weren’t [1:21:00] clear to people. I think people are starting to understand that now.
There are a few style books that are being written by people that I’ve heard about, and when those come out, those will hopefully be good, and those will be a good resource. Another thing to keep in mind is that Swift is opinionated, I guess is the way to say it. It really does encourage you to do the right thing where it can. For example, if you use var for everything, the Swift compiler [1:21:30] will say, hey, you marked this as a var but it could be a let, and let me fix it for you. That’s just its subtle way of encouraging you to use immutable values, which is a very small thing, but it’s just pushing you in the way that it thinks leads to a better code. Immutability for a local variable doesn’t matter that much except that it communicates something more to the person who has to read and maintain your code.
I think that Swift really does [1:22:00] encourage you down the right lines in some ways. But on the other hand, in other places where you’re saying, “Should something be a class or a struct?”, the trade-offs are more nuanced and it’s a harder thing, and the Swift compiler can’t just know what problem it is that you want to solve, so it can’t help you with that.
Swift’s style and community
John Siracusa: You mentioned Swift style guides before. As the creator of a language, how much influence do you feel like you want to have over things [1:22:30] like style — things that really are not features, not technical, just different style things? You should only have one exit at the bottom or you should exit early. There are things you can do, even things like spacing and brace style for crying out loud, or just the sort of idioms like, “This little three line sequence is how you do this transformation on this data structure.” You mentioned language features can influence that, so obviously as language designer and participating [1:23:00] in that process, you can effectively guide the community style-wise in one direction or another, but eventually you come to the stuff that’s just plain opinion and style. Several other languages have had fairly concrete styles set forth early on by the creators — I don’t know if you want to go K&R C-style type things. How much do you care about being able to influence the personal preference or the aesthetic portions [1:23:30] of style that are outside the things that you can influence as part of language design?
Chris Lattner: As with any programmer that has a pulse, I have my preferred style and so, clearly that’s the right way.
Casey Liss: Naturally.
Chris Lattner: Naturally. I guess, more seriously, the language itself does influence some of the things you mentioned. For example, it’s my personal opinion that the style where you have a single entry and a single exit from every function leads to really [1:24:00] difficult to maintain and read code. That’s one of the things that led to the guard statement existing. The guard really does encourage you to use early exits, so you handle the exceptional cases and then you don’t have to worry about them. That’s an example of where the language design is encouraging you to have multiple returns from a function. Other more subjective things like the number of spaces: I, personally, am a fan of two-space indents. I know some people are staunchly four, other people are staunchly eight. Some people think 80 columns is the [1:24:30] right answer, all these things are actually really good arguments on all sides. Some people argue that you should use three-space indent because then you certainly can’t have tabs in your file.
Marco Arment: Wow.
Chris Lattner: You have to pick your battles, and people will disagree for a long time. You can make arguments on both sides, but where we’ve come down is it’s just more practical to let people write code they like to look at. [1:25:00] That comes at a cost of, if you switch between teams or you pick up somebody else’s package then maybe it’s indented the wrong way, but I don’t think that really actively hurts the comprehensibility of the code. That’s merely annoying. Having something like Go’s gofmt which rigorously enforces the one true way, I think, would have been nice, it just wasn’t a priority early on and it also just wasn’t a battle that we were willing to tackle. We’ll see.
I [1:25:30] think this is still also an open question in terms of what other tooling gets built and what other things come in to the fray, because it would be great to have better tooling around this, it just hasn’t been built out yet. It’s a great opportunity for open source as well.
John Siracusa: I was just thinking about the doomsday scenario where one of these Swift style-guide books comes out and it makes some blanket recommendation, like you should always use protocols for this and you should never use classes for that, and it becomes wildly popular [1:26:00] and accepted as the Bible of how you write Swift programming, and you’re just somewhere shaking your head going, “No. You got it all wrong.”
I guess we’ll cross that bridge when we come to it. I think, at this point, the language design itself and the community has had enough influence that that’s not likely to happen. I guess what I was getting at is: do you feel like you want to write a Swift style guide, like if you had unlimited time?
Chris Lattner: No, I don’t think that would be a good use of my time. But if I had unlimited time, maybe. It’s hard to answer that theoretical question. [1:26:30] Also, to your concern about somebody coming out with a style book that was so wrong that would cause me to be really unhappy, I actually have a ton of faith in the Swift community, because if something came out that was so weird, I don’t think that the community overall would accept it. The community has, again, just blown me away with how clueful they are, and how many great people have gotten involved with Swift and the Swift-evolution side of things.
It’s been really amazing to see people [1:27:00] go through their own personal trajectory on the Swift mailing list where initially, they’re saying, “Hi. I had this idea. What do you think?”
The response is, “No, that’s a terrible idea for this reason.” Then they come back, “Hey, how about this? Did you ever think about this?”
“That’s interesting. That almost worked but it didn’t work because of that.” Then they come back, “Hey, what about this?”
Then we say, “Well, we actually tried that and we had that for a while, and then had to back out and change it to this other way because we didn’t even think [1:27:30] about this other thing.”
And then the next day, they’re the ones saying, “This is the way it needs to go,” and everybody says, “You’re right. You should do that.”
It’s really hard for people that haven’t been involved in the overall design of anything as complicated as Swift to be able to discern the difference between something that is the way it is out of an intentional decision, or the way it is as an accident of history. Now that the design process [1:28:00] is public, I think that it becomes a lot easier for people to understand that difference.
The one big debate that I haven’t actually seen is, why is the func keyword the func keyword? Luckily, we’re now beyond the Swift 3 source-compatibility threshold, so we can’t just have that discussion anymore. I think that’ll be great.
John Siracusa: One of the best things of Swift open-source: the [1:28:30] evolution process has the list of questions like the Asked and Answered questions, like this is a commonly asked question, we’ve dealt with it before, you’re probably going to want to ask it too, so here’s how it was asked before and here’s what the answer is. Not that everybody’s going to go and see that, but I love that that exists.
Chris Lattner: It’s also hilarious that that started out as the Commonly Proposed list and then, at some point, it became the Commonly Rejected list.
John Siracusa: Right. Because if it’s commonly proposed, obviously it’s not getting through, and so you have to go to the Commonly Rejected list if you’re thinking about asking to change “func” [1:29:00] to “fn”.
Rust-style memory model?
John Siracusa: I know you talked about this before, but it’s worth touching on again, because many people have asked me to ask you this: Swift and a Rust-style memory model. You phrased it before as an add-on feature for people who know that they need that type of thing, but that Swift wouldn’t become a language in which all memory is managed that way. Instead, this would be a special-purpose feature for particular roles. [1:29:30] What roles do you think it’s suited for, and why is it not the right memory model for all of Swift?
Chris Lattner: Here’s the way I look at Swift in terms of memory management today. Swift is using ARC, not GC. If you want to talk about the trade-offs there, I’m happy to do that — that’s it’s own rat hole we can fall into. ARC, I think, is overall an amazing default for Swift because it gives you deterministic destruction, it gives you relatively [1:30:00] predictable performance, and the ARC optimizer is able to eliminate most of the reference counting operations that happen.
The problem that I see with ARC, as of today, or say Swift 3, is that it is not a very controllable model. If, for example, you have an inner loop and it turns out the ARC optimizer is doing a bunch of retains and releases and you know that they’re not necessary because you know the objects [1:30:30] are alive, there’s nothing that you can really do to solve that problem unless you’re willing to drop down to something like the unsafe operations that Swift gives you for manipulating references to objects. It’s really kind of dissatisfying to me to have a really safe, great memory model that for performance reasons you have to drop down to unsafety. This is a huge cliff and it’s also a huge complexity cliff that you have to drop into to solve that problem, and that’s not really great.
[1:31:00] The promise, the dream, is to instead say, “Hey, if you are a sufficiently advanced programmer and you know how this stuff works, we can give you a new option. Instead of dropping down to unsafe constructs, you can actually just use more static type information annotations in your code.”
You add a few annotations to say, “Hey compiler, this is okay,” the compiler can then validate those type annotations and check to make sure they’re correct, so it’s still memory safe, it’s still guaranteed to be okay, but you’re [1:31:30] paying for it in terms of more annotation of the code and it’s more heavyweight in terms of coding. What you get out of that is the guarantee that no ARC operations are happening.
Once you get to that model, then it becomes really interesting because you can say, “Hey, in this inner loop I found in my profiler that it really does matter for performance and the ARC optimizer is doing the wrong thing. Well, I can add a few local annotations, and now the code is fast. It’s guaranteed to be fast, and the clients of my code don’t have to know about it. This [1:32:00] is an implementation detail.”
I think this will really bring us to the best of both worlds, where you can have memory safety, you can have a progressively disclosed complexity system where most people don’t have to know about it or think about it, but you can get to really low-level performance guarantees that otherwise weren’t possible.
Once we do this, I have a feeling that there will be certain communities of people that will only want to work in this model. These are the communities that like Rust predominantly, so if you’re writing a kernel, for example, [1:32:30] maybe you don’t want to have a reference count in your object at all. Maybe you just want to say, “I will always use these annotations in my code pervasively,” and if that happens then there’s a couple of options that we can investigate, and we can say, “Hey, if you mark this class with some attribute, that prevents the compiler from ever doing reference counting, and if it would ever do that, it just raises an error.”
Or we can even, theoretically, have a language dialect mode that produced an error anytime it did an ARC [1:33:00] operation. I don’t like language dialects, so we’d have to find a way to handle that. Maybe it would be a warning and then they would upgrade it to an error, or something like that, but I could imagine doing something like that.
Once we do that, I think you get to a really interesting mode where those systems-programming type people that otherwise would be using C or C++ can now use Swift, and that means they only have to learn one language, and they could use the same toolset, IDE, and everything else [1:33:30] that works great with Swift.
It means that they can scale and use the same language for different layers of their stack, because many applications have very low-level pieces and very high-level pieces, and having them written in different languages is really kind of gross.
I think we’ve seen this in the Objective-C community, where one of the best and most annoying things about Objective-C is that it has C in it. This has been hugely important for Objective-C in practice, because if you run into a performance [1:34:00] problem with objc_msgSend, you can always rewrite that algorithm in C. That’s really, really, really important for Objective-C being successful in both in the days of NeXT on 16 MHz processors and also today for the low-level code that people are writing. That’s also one of the problems that makes it so that Objective-C and C are so intertwined that you can’t actually take the C part out of Objective-C without producing a different language.
Swift in web browsers?
John Siracusa: We’re waiting for Swift in the web browser [1:34:30] because it’s your only escape, speaking of applications that have to have one language at the high level and a different language at the lower one. That’s a tough one.
John Siracusa: You’ve mentioned Dart before. It’s been tried.
John Siracusa: That’s the difficult part. It’s not the language and mechanics. It’s [1:35:30] getting something that can understand that in front of as many people as possible.
Chris Lattner: Probably, and I think that’s always been true for preprocessors. Even in C, if you do crazy things with the macro preprocessor, debugging those things is [1:37:00] so horrible. That’s one of the reasons that we’ve been careful about not adding a macro system to Swift early. It just brings lots of interesting trade offs in terms of the debugging, developing, and designing experience.
ABI stability soon?
John Siracusa: One last thing on ABI compatibility. According to the timelines that I’ve heard sketched out, ABI stability could be a thing before async or concurrency stuff lands [1:37:30] in Swift. Are there any dependencies or chicken-and-egg things involved in there? In other words, do you have to nail down some aspects of your concurrency model before you can commit to ABI stability to are they totally orthogonal?
Chris Lattner: I think they’re totally orthogonal. The memory ownership model has more of an impact on ABI stability because there’s some very low-level things like how the getters and setters are code-generated for a property, for [1:38:00] example, that end up getting impacted by the ownership model. That’s one of the reasons that the Swift team is actually actively designing and working on a proposal for the memory ownership stuff right now. It’s not so that it can be a Swift 4 feature, it’s so that the ABI stability work can be done in light of the ultimate model for the memory ownership support.
I think concurrency is not going to be something that impacts ABI stability at [1:38:30] all, but we’ll see, and if so, then the sufficiently smart programmers that work on Swift will figure it out and I’m sure it will be great.
I think it’s also an interesting question of whether ABI stability will be done for Swift 4 because it’s huge amount of work. The work is well underway of course, but it’s also not clear to me that’s really the most important thing for the Swift community. One of the things that Ted has been a huge champion [1:39:00] for, and I think he’s right about, is that the most important thing right now for the Swift community is making the compiler more reliable, making the error messages better, making compile times faster, and making it scale better to large projects. We’ll see how the rest of the Swift 4 schedule goes, but I wouldn’t be surprised if, at some point, they decide that focusing on that is more important than nailing ABI stability in Swift 4.
Swift left in good hands
Marco Arment: You mentioned Ted, that’s [1:39:30] Ted Kremenek who’s now the lead of the Swift project, right?
Chris Lattner: Yes, he’s the project lead. Ted has been managing the Swift and Clang projects for a number of years before that — he’s been at Apple for ten years now. He’s a Stanford PhD graduate. He almost single-handedly built the first Clang static analyzer. Yeah, that’s Ted.
Marco Arment: So as a — I love that I’m saying this now — as a Swift developer, [1:40:00] when you announced that you were leaving Apple, we saw some of the people in the community asking, “Should we be worried about Swift’s future?”
Is Swift in good hands now? It seems like most of us didn’t know Ted Kremenek, or at least didn’t know his name or weren’t familiar with his contributions. It does seem like he’s been doing quite a bit there anyway. Should we be worried?
Chris Lattner: I don’t think you should be worried at all. I think Swift is in amazing hands, as all the things that I’m leaving behind at Apple [1:40:30] are in really good hands, and that’s one of the only reasons that I was comfortable in moving on.
In the case of Ted, Ted is a rockstar. He has been running the Swift team and managing them for four or five years now. The roles that we had were that Ted owned and ran his team and then he reported to me. I would mess around and cause trouble by trying to get people to do the exciting, [1:41:00] important thing that I thought, and maybe it didn’t align with the actually important thing, and therefore I’d introduce chaos into the world. But Ted was always the stable rudder and the one that was really focused on getting the important things done.
He has a number of really exceptional engineers in the team. People on the core team like Doug Gregor, John McCall, Joe Groff, Dave Abrahams, these are all just amazingly brilliant people, and the team includes a bunch of other engineers that are [1:41:30] equally dedicated and just totally amazing people to work with.
The Swift team is, I think, one of Apple’s biggest assets. It’s crazy how brilliant and dedicated they are to the project and how excited they are to get to world domination, one step at a time.
Casey Liss: Coming back to a more broad thing, and this is not meant to be a tongue in cheek reference to your next job. Let’s say that you’re at home, your kids are in bed, your wife is out, [1:42:00] and you want to just write some code. If you can’t use Swift and you can’t use Objective-C, what are you writing code in? I know that’s often dependent on what project you’re accomplishing, but what’s interesting to you these days that isn’t Swift and isn’t Objective-C?
Chris Lattner: Well, I have a ton of different interests. I really love building ray tracers, for example, and I’ve done a bunch of stuff with that. I guess if I couldn’t use Swift for that, then I would probably have to use C++.
John Siracusa: I was going to call that, [1:42:30] but I wanted to give you a chance to not say C++. This is what I remind people when they mention it: you know what LLVM’s written in, right? You can’t write that much C++ and not have your answer be C++.
Chris Lattner: Do you know how much angst and personal suffering it causes to have to be writing C++ all day and building a beautiful world for other people? It’s just so unfair.
John Siracusa: Yeah. That’s where you toil down the mines.
Chris Lattner: Yeah. It’s pain and suffering that’s worth it for the bigger victory.
John Siracusa: That’s where Swift comes from: [1:43:00] years having to use C++. “I could do the same stuff, but not feel this much pain.”
Chris Lattner: That’s exactly right. It is a lot of suffering that led to Swift needing to happen.
But also, it really depends. Maybe if I was a crazy person, I’d use Perl for something. I don’t know if I’m that insane.
Casey Liss: Well done. Well done.
Marco Arment: You’d never be able to read it though.
Casey Liss: Nobody could.
Chris Lattner: Hey, if it’s throwaway code, then you never need to read it, right?
Marco Arment: [1:43:30] He’s fine.
John Siracusa: I’ll just wait for my Perl 6 grammars to land gently in Swift.
Chris Lattner: Perl really has nailed the string processing thing, even in Perl 5, and I think Perl 6 is even better. Swift has a lot to learn from it, and that’s something where Swift 4 is really learning from Perl. I think that it’s great.
Casey Liss: Please don’t give him any ammunition. This is terrible. I was enjoying this interview so much until this very moment.
Chris Lattner: [1:44:00] Look at it this way: once this happens, John will have no excuse.
Marco Arment: Oh, he will find one.
Casey Liss: Nothing is so perfect that it can’t be complained about.
John Siracusa: This is a perennial topic on the Swift mailing list. It’s about dynamic features in Swift, things that are “more dynamic in Objective-C and less so in Swift.”
In the Swift community, there’s a lot of angst about when, if ever, [1:44:30] will Swift allow me to — essentially like what you said before, you can write Fortran in any language, you can write Objective-C in any language. People will try to write their code in Swift, but using models from Objective-C that require things like sending messages to objects that you know will receive them but the compiler can’t guarantee, or having big heterogeneous collections where you just know everything’s “id” and it’ll be fine.
I don’t even know how to frame [1:45:00] this, but I think that the main question has been: Are dynamic features possible in Swift, and are they coming in Swift? Or is there some kind of philosophical divide where the most dynamic stuff that people really want will never be forthcoming, not for technical reasons but just for philosophical and safety reasons?
Chris Lattner: I’ve been pretty outspoken about this on the Swift-evolution list, so hopefully this isn’t a huge surprise, but I don’t think there’s any tension between dynamic features and [1:45:30] Swift.
A lot of people want to make hay and argue about how it would be impossible to do responders (or pick your feature) in Swift. In the worst case scenario, frantic blog posts I’ve seen are things that are along the lines of, “Someday Apple will rewrite all of its frameworks in Swift, and then it will be impossible to write responder chains, and then bad things will happen, and then we’ll be back in the days [1:46:00] of MFC or some other bad thing.”
Casey Liss: Oh God. Don’t even bring that up.
Chris Lattner: Exactly. This is quite the slope that the discussion ends up being.
I think this blew up most recently right before WWDC this year, and it was really interesting to watch the blog posts that people were flinging around, because everybody had a different idea of what “dynamic” meant. [1:46:30] Apple’s not going to counter-blog somebody and denounce a post on somebody’s blog and it’s bad form to do that. But I thought it was very interesting and odd that none of those people actually came to the Swift-evolution list, which is the right place to do that, and asked about it. They all just wanted to make these inflamed blog posts of various kinds.
My opinion is that none of those are in tension with Swift’s current design. It’s just a matter of prioritization, [1:47:00] and the major problem that Swift has is, again, there’s so much work to do. There’s concurrency model. There’s actors. There’s systems programming. There’s scripting. There’s lots of little things that go in. There’s better things to support static analysis, introducing type state into the language. There’s pre- and post-conditions.
There are so many things that need to happen, that will be bricks in the house of Swift that we want to build, [1:47:30] that it doesn’t really make sense to spend a lot of time building dynamic features so that Apple could theoretically someday do something. The Swift team is much more focused on the practical necessities of solving the real world problems, here and now, in front of people. Dynamic features, like many other things, will just plop in place when the team has time to design and roll them out.
I expect that you will be able to someday [1:48:00] reflect over all the methods that a class has, for example. Today even, the Swift compiler’s generating all the metadata so you can reflect over all the data members in your values. We just don’t have an API wrapped around that, but that metadata is what the Xcode memory-heap viewer uses, for example. There’s a lot of that kind of stuff being put in place, but it just hasn’t had time to fully bake out yet. When it does come in, I think its going to be really exciting and will open more doors for new kinds of [1:48:30] patterns that people can do, and a lot of those will be familiar to Objective-C programmers but they’ll also be super-useful for things that people really haven’t done in Objective-C.
I don’t see a lot of tension there though. It’s just a prioritization question.
John Siracusa: I think where people are coming from is in terms of prioritization, because if you’re used to doing things with certain design patterns, broadly speaking in Objective-C, and you wanted to come and implement those same patterns in Swift and you find that you can’t because you’re lacking some feature, it feels like [1:49:00] there’s a barrier up.
That’s getting back to writing your Swift in Objective-C style. That’s more of a barrier. They retreat back to Objective-C because if “I can write it the way I wanted to write it and it works in Objective-C, and when I try to write it that way in Swift, it doesn’t work for me,” your choice is to change the way you write it to be a better fit for what Swift has, or to go back to Objective-C.
Chris Lattner: I don’t agree with that, though, because Swift does support all the features for dynamic dispatch and method lookup and everything else that Objective-C does, it just requires you to use [1:49:30] the Objective-C subset of the language because it exposes it through the run time.
John Siracusa: Right. That’s the tension because they’re like, “I want to write in pure Swift,” as if there’s something special and magical about not using the Objective-C runtime, do you know what I mean?
Marco Arment: You do get a medal for that.
Chris Lattner: I guess it makes some people feel good? I can’t really counter that, but I just don’t think that it’s actually fair to say you have to fall back to Objective-C code.
John Siracusa: [1:50:00] Yeah, I got it. I didn’t write one of these blog posts — I’m trying to express what I saw, because that was a lot of the concern. They want to do it the new way and the right way, and part of doing that is not “@objc”-ing everything. That feels like a compromise in the old way.
So they are setting their own goal posts for themselves, arbitrarily, to try to enter into this new world. They want to jump in with both feet or not at all, and some of them might end up bailing.
Chris Lattner: [1:50:30] I look at that as an area where the language is not fully baked out, it’s not fully serving their needs, and I think that’s really unfortunate.
That’s not a good thing. I want there to be an awesome reflection model. I want there to be awesome features for sending messages like they’re used to. I just want it to be developed and designed in the right way, and at the same time they’re being annoyed that they can’t do those things, there are a lot of other people [1:51:00] being annoyed about compile times.
What we have to do is balance between — just picking two random things — where does the engineering effort go? That’s never a black and white thing. There’s never a right answer. It’s all based on gut and judgment and guesses, listening to the community, and talking to people.
One of the reasons why I think the Swift team is really phenomenal is that it really cares. It really cares about what people are doing. [1:51:30] The various people read the blog posts. There are people on Twitter. The engagement between the Swift team and the community is really high because people care. There’s no management dictate at Apple that says that this engineer should be on Twitter all the time talking to people. That’s certainly not the case. That’s because of passion from the developer and it’s because they really care.
That’s the thing that is so amazing about the Swift team is just how important it is in terms of customer service and understanding [1:52:00] what people are working through.
That insight is what guides the priority decisions and the trade-offs that have to be made all the time, because nothing really is as clear as you’d like it to be.
Apple reads blogs
Casey Liss: Building on that, and I was tangentially involved with this, when all of us were going back and forth about “Swift isn’t dynamic, it must be dynamic,” is it fair to say that that does eventually make it’s way inside of Apple? Just in that [1:52:30] you guys are aware of it, that that’s talked about in some way, shape, or form?
And I suspect that it does, but it’s nice to know that when the community gets perturbed, whether or not that feeling is justified, that at least somebody inside Apple has heard it. Whether or not they can respond to it, at least they’ve heard it. When we all get our pants all twisted up, does that make it in?
Chris Lattner: Absolutely. There are a ton [1:53:00] of people at Apple who care about what’s happening in the developer community and elsewhere. That’s one of the things about Apple’s approach in terms of not commenting on things that can be frustrating because you don’t know, but again, there are so many people at Apple that really do care and that saw it.
Again, I wish those people that were really worried about it would come to Swift-evolution and ask, because then we could have a discussion about it, but that wasn’t happening for some reason. [1:53:30] Certainly there were a ton of people that saw the blogs.
I’m sure there are tons of engineers at Apple that feel the same way and wish the priorities were different. I think that the engineering community inside of Apple reflects quite closely to the engineering community outside of Apple as well.
I hope that’s not surprising. Apple has many different engineers with different feelings and different opinions, and I think that’s a really good thing. If it were a super-monoculture, or something like that, that would probably be very unhealthy.
RxSwift and reactive programming
Casey Liss: [1:54:00] Taking a little bit of a turn, I started writing Swift almost exactly a year ago now, and the project that I’m doing at work, we’re using RxSwift and reactive programming.
I’m curious, have you looked into that at all, and do you have any particular thoughts on it? Does it seem barbaric to you? Does it seem clever? Does it just seem like an odd choice? What is your take on this whole Rx thing?
Chris Lattner: I’ve definitely seen it. I have no [1:54:30] real-world usage experience, so most of my opinion is based on reading blogs and reading people talking about how awesome it is while not talking about the problems.
From what I gather, it seems like it is a way of letting you write a lot less code, and code that often works better the first time, and I can see how that super-appealing. It also seems like it would be really challenging to debug it and maintain it, so I don’t know between the pros and cons how it balances [1:55:00] out.
If I were writing a lot of high level apps, I would probably try it out on an app or two and see if I liked it. But aside from that, I don’t really have a good insight either way if it’s the “right thing to do” or not.
Casey Liss: You mentioned a second ago that you see some problems and you mentioned debugging. Is there anything else that jumps out at you as being a big concern, or is it just that it’s so different?
Chris Lattner: I’m not afraid of different things. [1:55:30] The questions I’d raise are — and this is out of ignorance, not out of fear — what is the community doing? Is there a support line? Is it changing and evolving in a way that is good or bad? These are just things I don’t know because I haven’t used it, but I have nothing against the Rx community at all.
I’ve seen a lot of really cool things they’ve done, and I’ve read many of the positive posts on Rx, React Native, and many of the other things where people are using reactive [1:56:00] techniques.
ReactiveCocoa has also been around for quite a while, and I think that’s a really interesting application that maybe became a lot easier when Swift came out than doing it in Objective-C.
It’s great that Swift can support different kinds of programming models like that, because if an awesome new right answer to some problem comes out, you would hope that your system can support it and can make it really elegant and beautiful.
Garbage collection vs. ARC
John Siracusa: I feel kind of [1:56:30] like this is fighting the last war instead of the current war, but you mentioned garbage collection vs. ARC a bunch of times. Obviously, that ship has sailed, but I would love for you to give a reasonable summary of what the trade-offs were there.
Objective-C had garbage collection, as you mentioned. Eventually, Objective-C dropped the garbage collection and got ARC, and of course, Swift [1:57:00] doesn’t have garbage collection at all. Can you talk about the trade-offs there and why Swift is the way it is?
Chris Lattner: Would you like a comparison of ARC against the garbage collector Objective-C had, or garbage collection in theory?
John Siracusa: You can pick garbage collection in a language perhaps not as burdened as Objective-C is with C backwards compatibility. That’s worth mentioning as well, I think.
Chris Lattner: Objective-C had a system based on libauto. libauto had a bunch of implementation concerns and other things that, I think, [1:57:30] don’t obviously reflect onto the ultimate garbage collector that you’d ever want to have, and so I don’t think that comparing against that is interesting.
But let me turn it around. Why do you think garbage collection is interesting? What is compelling about garbage collecting beyond ARC?
John Siracusa: Well, the idea that memory management is completely out of the hands of the programmer, and that some magical fairy behind the scenes will make it all good for you. What you’re giving up, as you mentioned before, even with ARC, you lack some amount [1:58:00] of control that you’ll need to have a language that becomes a systems programming language. Obviously, you give that up with a garbage collection because you’re not going to be doing a systems programming language with a garbage collector behind it. Although I think Microsoft has some project to do that, the Singularity project, where they were trying to build a whole OS off of garbage collection. I don’t know how that’s going.
Chris Lattner: Right, but even of you ignore the systems programming side of things, because I think it’s fair to say a lot of people don’t care about that, garbage collection is proven in the application space [1:58:30] for Java, for example, and Perl but…
John Siracusa: Perl uses reference counting! Come on.
Chris Lattner: That’s right. It’s Python that uses reference counting plus cycle collector.
Let me ask you again. You said GC means that you don’t have to think about memory. Is that true?
John Siracusa: That’s the pitch. The pitch is that the programmer doesn’t have to think about it.
Chris Lattner: I understand the pitch but is it true?
John Siracusa: When the garbage collector starts doing things that you don’t want, then all of a sudden [1:59:00] you do have to think about it, but then there’s nothing you can do about it.
Chris Lattner: Here’s the way I look at it, and as you said, the ship has somewhat sailed: I am totally convinced that ARC is the right way to go upfront. It is better in a whole bunch of different ways. It gives you deterministic behavior, so it doesn’t have the unpredictable-stutter problem that people like to bash on GCs.
The stutter problem, to me, isn’t really the issue, even though [1:59:30] that’s what GC-haters will bring up all the time. It’s more about being able to reason about when the memory goes away. The most important aspect of that is that ARC gets rid of finalizers.
If you use a garbage-collected language, you use finalizers. Finalizers are the thing that gets run when you’re not object gets destroyed. Finalizers have so many problems that there are entire bodies of work talking about how to work around problems with finalizers.
For example: the finalizer gets run on the wrong thread, it has to get run multiple [2:00:00] times, the object can get resurrected while the finalizer’s running. It happens non-deterministically later. You can’t count on it, and so you can’t use it for resource management for database handles and things like that, for example. There are so many problems with finalizers that ARC just defines away by having deterministic destruction.
There are two arguments that people make [2:00:30] against ARC in favor of a tracing garbage collector, one of which is that ARC adds overhead because you have retain and release operations that run. That is true. The other is that you have to think about cycles in ARC because it doesn’t automatically collect cycles, and that is also true.
The rebuttal I’d give to people is that those problems are also true in garbage collection, just in different ways. In a garbage collector, for example, [2:01:00] people don’t think about it, but garbage collection injects additional code into your application just like ARC does.
There are many different garbage collection algorithms, and not all of them are the same. But most modern garbage collectors, that use a nursery for short-lifetime objects then promote them out — that are generational — use something called a write barrier. Every time you store to a property [2:01:30] of an object, say, you have to run additional code.
Garbage collectors also need the ability to stop all the threads, or at least to be able to stop threads at some point in time, and they need to be able to do so within a specific time bound because they don’t want the garbage collector to take forever. The artifact of that is that typical garbage collectors, in Java for example, will introduce what’s called a safepoint into loops. So now, in your loops, extra code is being run because of the garbage collector.
On more [2:02:00] aggressive garbage collection algorithms — for example, I was reading a blog post recently about Go’s tricolor algorithm — they’re touting the advantage of really low latency and the ability to guarantee response times in a more fine-grained level than most garbage collectors. But to do that, they use this tricolor algorithm which dramatically lowers throughput, because they’re doing almost exactly the same kinds of operations that ARC is doing.
The problem [2:02:30] that it then introduces, though, is that these operations that the garbage collector is introducing are sometimes but not nearly as well optimizable as the ARC overhead that the ARC optimizer applies to.
Furthermore, there’s no out on it. With ARC, I think and hope that the ownership model will give people the ability to take control of those overheads. And if it becomes a problem in practice, or if they’re just that kind of person, they can take full control over the lifetime of their objects, and then know that ARC will never happen. In a garbage collector, you don’t have that.
[2:03:00] The performance side of things I think is still up in the air because ARC certainly does introduce overhead. Some of that’s unavoidable, at least without lots of annotations in your code, but also I think that ARC is not done yet. A ton of energy’s been poured into research for garbage collection, particularly since Java has come up. There’s been hundreds of papers written in the academic circles, tons of work in HotSpot and other Java [2:03:30] implementations to do different tweaks and different tunings and different new kinds of algorithms in garbage collecting. That work really hasn’t been done for ARC yet, so really, I think there’s still a a big future ahead.
On the programming side of things, the cycle side of things, I think it’s also a really interesting question of how much should people think about memory?
When I was baiting you a little bit, you said that the great thing about garbage collection is that you don’t have to think about memory. Of course we know that’s not true, right? Because if [2:04:00] you have a reference to some big object graph that you didn’t mean to keep around (maybe it’s in your undo stack), then you will “leak” that memory. That’s true of a garbage collector, and that’s true of ARC as well. Any automatic memory-management approach has that problem.
There’s this question of if you’re building a large scale system, do you want people to [2:04:30] “never think about memory?” Do you want them to think about memory all the time, like they did in Objective-C’s classic manual retain-and-release? Or do you want something in the middle?
I think that ARC strikes a really interesting balance, whether it’s in Objective-C or Swift. I look at manual retain-and-release as being a very imperative style of memory management, or malloc and free, where you’re telling the code, line by line: this is where you should do a reference-count operation, [2:05:00] this is where you should release the memory, this is what you should do at this point in time.
ARC then takes that model and bubbles it up a big step, and it makes it be a very declarative model. So instead of telling the compiler that this is the place that you should do a retain, you instead say, “This is an owning relationship.” The cool thing about that to me is that not only does it get rid of the mechanics of maintaining reference counting and define away tons of bugs by doing that, it also means that [2:05:30] it is now explicit in your code what your intention was. That’s something that people who maintain your code benefit from.
By saying that I have a weak point or two, the parent object of my thing, that’s a really important relationship to know about and as you’re looking at the code, you’re maintaining the code. Having that be explicit is very valuable, because that talks about the relationship between values. To me, again with the goal of being able to write large scale applications in Swift, I think that’s really useful. [2:06:00] I also don’t think it’s hugely burdensome, though it’s definitely part of the learning curve of learning how Swift works that it has to be balanced in there as well.
So I don’t know. ARC has clear advantages in terms of allowing Swift to scale down to systems that can’t tolerate having a garbage collector, for example, if you want to write firmware in Swift. I think that it does provide a better programming model where programmers think just [2:06:30] a little bit about memory. And I think that going forward, it provides a really high performance model that you can get better than garbage collection in almost every way. I think that in terms of trade-offs, it’s the right one to push forward.
The third piece that garbage collection is really bad about, which is kind of a showstopper for Swift, is interoperability with C code. If you’ve ever worked with Java or other [2:07:00] similar garbage-collected languages, one of the major advantages the garbage collectors give you is that they move objects, and they need to do that so they can compact those objects so they can then efficiently do allocations. The problem is that once you start moving objects around, if you’re interfacing with C code, you can’t have some random C code having a pointer to your object and have it move because then you get a dangling pointer.
Once you get down that line, you end up with things like JNI, the Java Native Interface, where you have to [2:07:30] explicitly pin things, you have to maintain them, it’s very complicated, it’s really buggy. ARC completely defines this away by just saying that something’s in memory, It has predictable lifetime, you can reason about it. Swift provides tools for dealing with unsafe pointers and things like that, and that makes the interoperability with existing C code — but also with Objective-C, and maybe someday C++ code — really simple, really natural and really efficient. I think that’s a huge advantage that ARC [2:08:00] provides that really would be impossible to do with a garbage collector.
That’s my opinion. I think reasonable people disagree, obviously, but it’s something that does come up now and then.
John Siracusa: Or it used to come up, anyway. I’m convinced. I think you should use ARC for Swift. Get on that.
Chris Lattner: Yep. There we go.
Casey Liss: Chris, thank you so very, very, very much for coming on the show. Obviously, we are not an interview show, as the listeners have discovered over the last hour or two, but to take [2:08:30] your time on what is presumably a very busy time of your life and spend it with us three knuckleheads, I really appreciate it. It was very kind of you to do. Hopefully, this won’t be the last time that we’ll hear you on a show, or maybe even this show. We’ll see. Thank you, Chris. I really appreciate it.
Chris Lattner: Thank you for having me. I’ve had fun, even though I’m terrified and probably made a fool out of myself.
Casey Liss: Not at all. Trust me. That’s normal. I’m still trying to get over that.
Marco Arment: Plus, that’s our job.
John Siracusa: We didn’t talk about woodworking stuff.
Chris Lattner: Making splinters is fun.
John Siracusa: What kind of wood was that table made out of?
Chris Lattner: It’s walnut.
John Siracusa: That’s got to be tough. That’s a lot of cutting. That’s pretty hard wood, right?
Chris Lattner: Yeah. It takes time, but it’s fun. It’s one of these areas where you can learn a lot about it over time and keep going deeper and deeper in different spaces, and you [2:10:30] never know everything. It’s a lot of fun to dive into woodworking.
It’s also very different than sitting in front of a computer all day. You actually get to make something with your hands and you get to build…
John Siracusa: Much higher chance for dismemberment.
Chris Lattner: That is true. That is true.
Marco Arment: I think it’s interesting when programmers have incredibly analog hobbies.
Chris Lattner: It’s also hilarious because the woodworking community is going through this whole CNC thing, where they’re turning it into a very programmed thing, and people always say, “Why [2:11:00] don’t you get into CNC and do blah, blah, blah?” and I’m like “But that’s exactly what I’m escaping! I don’t want to have to program my…”
Marco Arment: I love that you can’t even escape community arguments over standards.
John Siracusa: It’ll make it so that you just cut down a tree and throw it in your workshop, and then out comes a table.
Chris Lattner: That’s the funny thing about people is that people are all the same, they just don’t want to notice and agree to that. You get to very different communities and they think they’re all different, but they’re actually the same at root.
After-show: Neutral (cars)
Casey Liss: [2:11:30] Chris, you’re obviously changing — not careers necessarily, but employers. Would you consider yourself a car person?
Chris Lattner: That’s a funny question. I obviously need a car. Cars are very important.
But I’m what I consider to be an un-car person. I want to accelerate the path to cars being appliances that solve people’s problems, and I am personally not the kind of guy who loves doing oil changes and fiddling around with them. I [2:12:00] just want something that is reliable, that works… ideally drives me to where I want to go, and I don’t have to think about it. It’s just solving my problems. It’s not something that I have to care for, feed, and maintain. That’s the way I look at cars.
John Siracusa: Can you drive a stick?
Chris Lattner: I did drive a stick before I got my Tesla a few years ago. Yeah.
Casey Liss: It all becomes clear now.
Marco Arment: Really, it kind of sounds like you want to make cars boring — in a good way, really. That’s [2:12:30] a noble goal and I can see why you went to Tesla.
However, it is kind of funny that Tesla is, in my opinion, clearly the most exciting car company that the world has known for quite some time. So to go there while having an almost nonchalant opinion about cars is interesting to me.
Chris Lattner: Yeah, well, I admit that I never considered going to work at a car company before, but that’s because I always considered car companies to [2:13:00] be not in need of the kinds of skills that I bring. But I think Autopilot is a really exciting and really big problem, and it fits with my desire to solve nearly impossible problems and take on new things. I’m really excited about it.
Marco Arment: I’m a little curious, on that note. We’ve talked on this show before about [2:13:30] when and whether we think fully self-driving cars will exist. In what kind of timeframe? Are we talking a decade? Our lifetimes? Ever?
It’s one thing to make something that could work on clearly marked highways, but it’s another thing to make something work where John lives where there’s ice all over the roads for most of the year so there’s no more lane markings and there’s barely any flat road surfaces, or you have all of the different ambiguities that you have in real-world roads.
Do you think — obviously, this [2:14:00] is not in any kind of official capacity as an about-to-be Tesla employee — but do you think, big picture, that we will see completely self-driving cars capable of driving on effectively all roads within, say, the next decade? If not, maybe within our lifetimes?
Chris Lattner: Yeah. I think that it’s very possible within the next decade, and hopefully that’s a long view. But keep in mind the goal is to drive better than a human, and [2:14:30] significantly better than a human.
I think one of the things that gives me confidence in this is that it’s not an even playing field, because the autonomous cars have more sensors than humans do. They have a radar, for example, and that means that they’re capable of doing things that humans just physically can’t.
There’s a huge technology problem. There’s huge software problems. This is not an overnight kind of thing that you wake up and solve immediately, but [2:15:00] I have confidence it can be solved, certainly within a decade.
Casey Liss: The good news is it’s not going to take a lot to be better than your average driver from Massachusetts because that’s a pretty low bar.
John Siracusa: No, because they know how to navigate these intersections that have seven different exits all at different angles, some of which are one-way, none of which are marked. The humans can figure that out. It’s going to be tough for the sensors.
Marco Arment: I’ll tell you one thing. I will drive in Manhattan without any problems. I would never drive in Boston.
Casey Liss: Amen brother. I completely agree. Alright, to [2:15:30] bring us back around, Chris, you made mention way early in the episode that you like yellow cars. What?!
Chris Lattner: Yeah. One of my favorite cars was a yellow Mini Cooper convertible with black hood stripes on it. It was totally awesome. That doesn’t get you in the carpool lane, but it was a beautiful car.
Marco Arment: In his defense, that [2:16:00] category of colors — yellow, orange, maybe teal, or even a pink — kind of bright, almost neon colors can work really well. They often don’t, but when they work well, they look pretty good. I could totally see that.
The only flaw I see in this plan is that Tesla does not currently offer either yellow paint as an option or custom paint colors as an option.
Chris Lattner: Well, now you’ve broke into the real secret of why they want to hire a compiler guy.
Casey Liss: That’s why.
John Siracusa: [2:16:30] Would you have gotten your Tesla in yellow if you could have?
Chris Lattner: Yes! Absolutely.
John Siracusa: It would look like a giant banana. What are you doing?
Chris Lattner: Yeah, I hadn’t really fully thought this through, John.
John Siracusa: I don’t know if that car could pull it off. We’re talking about a Model S, right?
Chris Lattner: Yeah.
John Siracusa: No. Yellow doesn’t seem right for that. We can do some Photoshop mockups. Let’s workshop it a little bit and we’ll see. Maybe if you got black stripes down the middle.
Chris Lattner: I admit this is a case where details matter.
Marco Arment: [2:17:00] If there’s any way for you to get a pet feature implemented in the model line, that seems like a pretty good one to do.
I would ask, I know you’re probably not going to be working on the Bluetooth UI but I’m sure you could probably put in a word with the right people to please improve Bluetooth audio.
Otherwise, if you can get them to make a yellow one, in addition to potentially saving thousands of lives a year [2:17:30] with Autopilot software, getting improved Bluetooth and a yellow option would be pretty impressive.
John Siracusa: Well, yellow would make your car more visible to other human drivers who need bright colors to know that you’re there since you don’t have any engine sound.
Chris Lattner: And I have to say this is the most effort I’ve ever done to win over one Swift programmer.
I’m not sure if this is a scalable approach.
John Siracusa: Yeah, [2:18:00] this is a bad strategy for Swift adoption. You should seek other means instead of coming on everybody’s podcast and talking to them individually.
Casey Liss: That’s amazing.