046: What's in store for Ember? with Yehuda Katz

In this episode, Yehuda Katz, co-founder of Tilde, OSS enthusiast, and world traveler, talks about what's in store for Ember.

Yehuda Katz: @wycats | blog | GitHub

Transcript:

ALEX: Hey, everybody. Welcome to the Frontside Podcast, Episode 46. My name is Alex Ford, subbing in for our usual hosts, Brandon and Charles, today. We have an awesome episode. We have a really special treat for you. Co-creator of Ember, Yehuda Katz is joining us today. Hello, Yehuda.

YEHUDA: Heyo!

ALEX: We also have a first time Frontside podcaster, Chris Freeman. Chris, do you want to introduce yourself?

CHRIS: Hey, everybody.

ALEX: We've also got a podcast Frontside favorite, Robert DeLuca.

ROBERT: Favorite? I don't know if you say that. Hey, everyone. How are you doing?

ALEX: I'm really excited about our guest today. Yehuda was just in Austin a couple of days ago. He gave a great meet up talk and a deep dive into Ember and it looks like you're going on-tour with that talk, Yehuda. Is that what I saw from your website schedule?

YEHUDA: Yeah, I'm not sure exactly. I change it up every time, largely because things happen. So if I say this thing is 'active' or 'in progress', and then it actually shifts, I have to change it up. I've been talking a little bit about what's up on what we were working on.

ALEX: Do you want to give us a brief outline as to what's going on in that talk for those podcast listeners who might not be able to attend? What's going on with Ember? What's new? What is it that you're trying to get across here?

YEHUDA: Sure. Actually, the talk I gave in Austin was, you're right, it was basically a deep dive. It was really focusing on a few targeted things that we're working on. I would say that at a high level, we're basically working on a couple of things. One of them is generally more integration with the ecosystem, things like ES6 modules, classes, components that look more like HTML and more graphic components and things like that, also improving EmberCLI so it's more integrated with other tools that people are using.

A lot of that stuff has to do with the fact that Ember started a long time ago now, like five years ago or so. And so, I think we've actually done a pretty good job of keeping up with things. For example, we adopted ES6 modules and promises a while ago now, and I think generally speaking, we tend to keep up with the ecosystem. But because we've been around for so long, there are certain things like classes, where it took a while for that feature to catch up with the functionality that we were using in Ember. Decorators landed a little while ago as a stage 2 feature in TC39, and that lets us really take a bigger chunk of the functionality that we have in our class model. I make it work for everybody with class syntax and that's something we're pretty excited about. So that's one area just generally taking things where Ember had its own stuff and try to integrate a better ecosystem.

Another big area is this mobile readiness and also, a lot of that has to do with the fact that things like service worker have just recently landed. For example, AppCache was a nice feature in some ways. Some people at Google will kill me for using the word 'nice' in AppCache in the same sentence.

[Laughter]

YEHUDA: But AppCache was trying to accomplish something for a long time. I think it did some version of what it was trying to do. But really, using AppCache is a default behavior for all users having - there's too many caveats to make it work well where service worker, because it's more of level one and more directly controllable is a better fit for something that we could ship with all Ember users. We basically want to use Ember and EmberCLI, you build an application, you get a good mobile experience out of the box.

Some of that has to do with trimming down parts of Ember that we don't need to be using in simple applications. Some of it has to do with service workers, some of it has to do with things like Glimmer 2, just making the performance better. But generally, that's the other [inaudible] so it's basically mobile readiness on the one hand and just integrating better with the one ecosystem are both big picture things we're work on.

ALEX: Something that you brought up in your talk where private Ember methods and how a lot of people use private methods and you have to keep them around, what you we’re just talking about that was unifying around the conventions of programming in Ember. Whatever JavaScript people bring in to Ember, you want to try to incorporate that as the language moves forward which is, I think, a really interesting problem.

Also, something you could talk about a little bit further is what you look for in the way people use Ember going forward and how you have to kind of bend the framework to allow it to be backwards compatible. I'm curious what that decision making is like.

YEHUDA: What you're talking about and what I talked about yesterday is what we call 'intimate APIs' and that basically means APIs that we never intended to be public. But for some reason or the other, people got their hands on them and started using them. I gave a somewhat elaborate example of funky case yesterday. But basically, the way we approach generally dealing with compatibility is pretty similar to how the web itself does it.

First off, there's a thing that we mark as a public API. We just don't break it unless we make a major version which is very rare. We have basically one of those the entire Ember, and I don't think there's anyone coming in the near future.

One option is if we don't like something, we just break it. That's very uncommon. Another option, and this is way more common, is that we try to build -- it's for public APIs -- we try to build a new API and we try to nudge people away from the old one.

One approach for nudging that is probably the most common is deprecation. So, deprecations themselves don't violate semantic versioning because we're allowed to say, "Please don't use this anymore."

The one area that's annoying about deprecations is that if backup code that powers the old feature has to still stick around. And so something that we've been working on around that aspect, around deprecations is something we called svelte build, which is basically the idea that we'll mark every deprecation with the version, that it will start to be deprecated in and people can ask for, "Please don't include any code that was deprecated out of 2.4, or 2.5, or 2.6, or whatever." Then, we'll automatically slip it away. You could think of it sort of like as a reverse feature flag.

CHRIS: Wow, that's actually super interesting. I didn't know that.

YEHUDA: We haven't finished it up yet but the RFC that talked about it and actually some old guy who actually wrote the RFC along with 1.13 when I noticed that 2.0 was going to end up being a pretty painful release for a lot of users.

We did a lot of things around 2.0 to make things less painful, like we made sure that 1.13 contained all of the deprecations that you could possibly need, as well as all the new features. So if you went to 1.13, you could look at all the deprecation warnings, switch to the new functionality. As long as you have no deprecations left, 2.0 was just the exact same code without any of the deprecated features. But as we're working on that, I realized that there is no real reason to give people such a heartbreak, if we could instead just slip away the code. So that's one approach and I think, more or less deprecations, and then eventually, svelte builds are the normal path.

With regard to intimate APIs, those are cases where people came to rely on very specific timing or very specific API, very specific details in some internal API and for those things, if we know that a lot of people use them -- usually they get used to like a couple of add-ons. Maybe Ember Wormhole, which is really popular, we'll use it. Then it's really hard for us to remove those things. Those APIs are harder to maintain compatibility for because the exact details of what they even did was never really well-defined in the first place because they were never documented. So usually what we'll do is we'll look at the usage of the API. We'll come up with a new API that is satisfying the exact same use case, and then we'll deprecate the old API. The policy these days is that you have to go through an LTS release so we'll make sure it's deprecated.

Let's say, you want to deprecate something now, make sure it's deprecated in 2.8. And you'll know that if you were actually doing the whole song and dance that deprecate what is intrinsically a private API, so it would be within our rights to say like, "Sorry guys. You used the private API. We're not going to help you."

But we really think it's important that for Ember, if something feels like a breaking change, that we're not doing it willy-nilly. If somebody upgrades to 2.9 and all of the sudden, Wormhole stops working, they're not going to understand that the reason that happened was because Wormhole did a bad thing. We basically need to do a clear pass.

So we'll do a deprecation in LTS release, then we'll wait a couple of releases before removing it. Then usually what happens is, in the meantime, we'll go ahead and we'll submit pull request for the big add-ons that we're responsible for, and we'll also try to talk and write down why it happened.

Historically, we've done that a few times and it's worked okay. There was an example of this, which is the lookup factory API in Ember which is really a boring API but it's used by a bunch of high profile add-ons. So the reasons why we needed to deprecate it were silly. They were just a bunch of bad behavior in the old thing that was making everything super slow. We can make things faster by giving people exactly the same functionality without exactly but identical guarantees.

So there were some 'guarantees' which don't even make sense for private API. But there were some things that in theory, the API did that we didn't want to support because of performance reasons. And so, we gave people a new API that is, for all intents and purposes, identical. All users will be able to use it in identical way. But it doesn't have exactly the same weirdness and that weirdness was pretty expensive.

ALEX: So you've trained Ember developers to be on the 6-week release cycle? They're looking at the blog posts. They’re looking to upgrade but you've been involved in a lot of open source projects where I'm sure that wasn't really the case.

Say, jQuery has a huge API and obviously, some things have to be deprecated on that and you were on the jQuery core team, I should mention.

YEHUDA: Rails has the same story whereas API releases every year, more or less.

ALEX: So, I'm just curious. The fact that you have Ember developers, I would like to think bingeing on your word and hinging on those updates, how would you go about, say, the Rails API? Or the jQuery API?

Maybe, now you're involved with Rust, and maybe the plan is to have Rust on a 6-week release cycle. I'm curious, if you don't have your developer’s attention, as you do the Ember developers, how do you deprecate an API like that?

YEHUDA: That's a good question. How do you deal with deprecations if you're releasing quickly? I think there's a couple of important points to make here. First of all, Rust is on the 6-week release cycle. Sometimes, as the same kind of story with intimate APIs, it's much less common with a strong-type system like Rust.

I guess, important things to point out, first of all, deprecations don't intrinsically break things. When we talk about intimate APIs and deprecations happening pretty quickly, those are APIs that are large, if someone is not paying super attention to Ember, they're probably not using those APIs, like they would not have known to use them in the first place. They might be using an add-on that use those APIs and the intimate API deprecation process causes the add-ons to update relatively quickly.

In terms of regular deprecations, those deprecations stick around forever so you could come back a year later. For the most part, you could make an app that was 2.2 and upgrade it to 2.9 as long as you upgrade the add-ons that you are using at the same time and everything will work.

We also realized that some people can't upgrade every six weeks and that LTS release process which is basically a six-month process, more or less. It basically gives us a communications channel to people who want to pay less attention. The way that that works is that, every four release cycles, so that six times four is 24 weeks -- about half a year. Every 24 weeks, there's another release.

We assume people are on that release channel. Some people operate at 2.4, 2.5, 2.6, 2.7, 2.8. Some people go directly from 2.4 to 2.8. For those users, we [inaudible] ecosystem, please make sure you support the last LTS release, which means that if your user's on 2.4, and 2.8 comes around, you know that you could have stuck to 2.4 and generally got add-on support and when 2.8 comes around, you should probably upgrade. Also, with intimate APIs, we make sure we always deprecation them one LTS release before we move into an LTS release.

Now, what that means is that if we want to remove something by the 2.8 LTS, we have to have already deprecated it in 2.4. If we want to remove something by 2.12, we have to remove it at 2.8. So six months is still not quite the one-year Rails release cycle but it's starting to get to a reasonable state.

Also, I would point out that the LTS releases, the support policy for them is that they're four cycles long. We do bug fix support for six cycles and security releases for 10. What that means is that we're actually supporting LTS releases. We were supporting two at a time for security patches -- two and half basically -- and we're supporting one and half at a time for critical bug fixes. The one and a half basically means that when 2.8 comes out, you have two release cycles which is basically three months to upgrade. If you're on 2.4, and 2.8 comes out, it's not like, "Oh, my God. Panic! I got to upgrade right now." You can take a few months to upgrade. Basically, 2.4 came out, you got all the deprecations you need to care about. You had six months to deal with deprecations and then another three months after that.

Even in terms of intimate APIs, where in principle, like Rails and jQuery don't even care about those things for the most part -- ah, jQuery cares about it even more. But most projects will get private APIs and say, "Sorry you used the private API. Why did you do that?"

Ember is a rare project in that we actually deprecate things that we know where we actually use them. We have a process for dealing with them. Even that process, like I said, it's not a six-week process. We don't deprecate something and remove it. We deprecate something and then give it a pretty long horizon before removing it.

ROBERT: I'm curious. You brought up that you are the common element between jQuery, Rails, and Rust. I know that there are, at least, between Rust and Ember and from Rails to Ember, there have been a lot of commonalities and lessons learned in how the projects themselves are managed. But I'm also curious with Rails, Ember is clearly pretty heavily influenced by Rails, which you were doing before. You've been working on Rust quite a bit and I'm curious, does your usage of Rust, even though it's a very low-level language, does that influence Ember at all? Does that change how you think about the framework or JavaScript in general?

YEHUDA: The number one thing that I got out of all those projects that I think used to be a thing is something like a conventional reconfiguration idea which is really not - I think the mentioned of reconfiguration is probably not even the best description of what it is. I think the idea that communities that are all working together on the same thing to build that thing bigger and better and better and better and build ecosystems around that thing, those communities are able to build much higher than communities that ask every single developer to put together a bunch of pieces themselves for profit. That's the basic idea.

If you look at Rust, which is conceptually very low-level, you’ll find that there are things like Cargo, which is a tool that not only builds your thing and not always package manager but it has a convention -- not only a convention -- it has a built-in support for documentation. So you're on Cargo docs, you get all your documentation for all your dependencies. You run Cargo bench. That's a built-in thing that runs your benchmark. You run Cargo test that runs your test. To mark your documentation as being Rust code, it will automatically run your tests for you when you run Cargo test. We will build your examples for you and make sure your samples keep compiling.

There's all this stuff in Cargo that you would not necessarily consider, like it's basic [inaudible] helping you get your workflow the same. Then there's things like Rust format which you've been working on and there's been a huge debate in the community about exactly how much configuration we want to allow in Rust format. But the irony of it is something that most people agree with is that we should try to come up with some kind of default style for Rust that everyone agrees to, that most people can pick up and use the Rust products where it's often used.

Then there's things like Futures and [inaudible], where the goal of those libraries is really to make there be a single central way that everybody does [inaudible] in Rust. These are all things that if you look at like C or C++, which are languages that are sort of in the same low-level in this space, in the same kind of area, you'll find that those languages have billions of ways in doing all of those things and there's so many different styles and so many different workflow tools, so many different things that you can make in then a million things that [inaudible].

Even the Rust is conceptually low-level, it doesn't really affecting every single environment as some sort of things that everyone doesn't need to do themselves. I think, an important thing that people don't always get about convention configuration is that it's not just that everybody doesn't have to do all of those things and it saves you some time, it’s that when everyone is doing the same thing, it’s a lot easier to build another level on top of that.

For example, a fast food is a great example of this in Ember. The fact that everyone initializes their application using an Ember initializer, the fact that services were these global things that are sort of global- there's no better word than 'services' but global services, the different components could share the fact that those are all going in the same place. The fact that the way we manipulate DOM is always in a constraint single area.

Almost things mean that when it comes out and to build something like fast food, it’s pretty easy to take almost any Ember application and make it run in the fast food environment because we know what we're looking at, and that's something that isn't necessarily true about other tools.

For me, Chris, the number one thing that, I think, all of those sharing, including jQuery, actually, like jQuery said, "There's so many different ways that people do DOM manipulation, why don't we unify it into one thing that I everyone can use?" You know, the 'jQuery plugin', which is something that has falling out of favor over time. The reason it was so popular was largely because people knew, "Okay, I'm dealing with jQuery object. If I just put a plug into the jQuery object, it makes sense." People understand how to use it.

I think that's something that a lot of projects that I used to share and it's also something that is not close to ubiquitous. It's very uncommon, actually. So that's one thing -- the conventional configuration story.

I think, another major aspect of all this, and this is something that jQuery and Rails do not share, but Rust and Ember have the RFC process, which more or less, is just a wave as a community of saying, "The way that we agree to add new features to the project is not something coming down the [inaudible] with a tablet every year."

At a conferencing, we have agreed to add these features but sometimes people are core team, but sometimes they're not. Sometimes, they're actual contributors, coming with an idea, write their view down in a format that we all know how to do with. Then there's a community discussion about it. Sometimes it takes a very long time. Sometimes it's not. But then we eventually come to a conclusion about what it is that we're doing. Eventually, the core team agrees to merge the RFC.

I think one of the nicest things about RFC process is that it produces an artifact that you can come back to a year, two years, three years later. If you say, "Oh, I wonder why they've made that decision. I wonder why that's the thing that they did," and the reason why this is great is that the RFCs are not gospel. They’re not something that we should hold onto forever. But at the same time, we don't want to reel it again, things that we already discussed that in-depth over and over and over again.

If a person comes back and they say, "Oh, why do they do that? The modification of RFC, why are these instructions directors are like that?" If they go back and look at the RFC and the thread associated with it, and the thing that they want to bring up is something that was already discussed, it's really no reason to bring it up again.

But maybe someone have thought of a different idea or a different reason to dislike or to disagree with the decision that was already made. That was already discussed. That's a much better rationale for bringing up for re-litigate. In other words, re-litigating is actually good but if you re-litigating five times a day, on every decision, that's not why you move.

So RFCs, by their very nature, the fact that the core team is doing things in public like anybody else and everybody else is also participating in that same process, the fact that artifact tells you, more or less, exactly what was discussed, makes it really easy to decide when is a good time to revisit some of the questions.

ALEX: Do you find that poll request has the same process as an RFC and it's an artifact you can go back to, it's a place to have communication that is visible to everybody, unlike say, this micro message service such as Slack where context is just lost for the public. I'm curious if you want to see that modeled in poll requests or if an RFC is where something like that belongs.

YEHUDA: I think, poll requests are great. I remember that when I was somewhat like the first user on GitHub who's not a founder of GitHub. I remember one of the things that excited me about GitHub early, although, the very beginning, didn't have poll request yet. but one thing that excite me about poll request is that before poll request, every single time I would use an issue tracker so they were like a billion issue trackers like [inaudible] whatever, and at that time I called it 'patch management'. I want something that helps me manage patches because the actual discussions are not the higher of it. The higher of it is something that submitted a request for me to merge in this patch. How do I merge it? How do I discuss with them? Those things were always really hard so I might ask people to upload patch files or whatever.

It’s hard to remember how bad things were but the number one thing that was just so obvious but also so terrible about the ecosystem before GitHub was patches were like old mailbox approach. Like you'll make a patch, and hopefully, get it to the right place at the right time. So I think, poll request and comments of poll request and many of the improvements that have been made for poll request are great.

The reason, I think, RFC are really important in addition to poll request is that, by the time someone actually took the time to write some code and submit it, it's very easy to look at it and say, "Well, I don't necessary agree on all the things here but I don't want to give a person a hard time that will do the work," whereas somebody submit some idea early on and they say, "I have this idea --" It's actually a lot easier to sort of get into details at that point and say, "Don't do this. If we should do this or it doesn't fit that well with this other RFC or this other poll request that's already open --"

But once somebody actually does start and actively working on the feature, I think, poll request are great, like most open source project these days. Ember doesn't ever committed anything to master. Everything goes to poll request, and even core team stuff. I also find that when I submitted a poll request or anybody in the core team, there are almost always people who are not in the core team that saves or fix in the poll request, for various reasons.

Of course, poll request also are the usual mechanism by which people run things like CI and linting tools and things like that, called quality tools. I think, the poll request workflow is really good. In terms of other messaging services, I think, there just sometimes the need to have conversations that are faster than poll request and I don't really have any problem with check conversations. But I definitely agree that it has a deep conversation.

This is something that happened in the [inaudible] a lot where we having this conversation with the core team and somebody will say like, "We should really move this out into a public discussion, or move it into RFC. If you don't agree with this thing that somebody said public, can you say [inaudible]."

So I totally agree that if there's a thing that people want to say in private about something but it's just in private for convenience, it's not private or transient for any good reason, actually getting out there onto the issue or the poll request and say your opinion and letting the conversation back and forth happen there is, for the exact same reason as you said, very useful.

In fact, Aaron Turon from Rust brings this point out repeatedly. We just had a conversation this past week about the fact that we have sort of a normal Rust project that, let say in the core team room and it's a technical topic, and it doesn't have anything sensitive about it, people always say like, "Hey, can you move that into Rust Internals," which is a public room. Or like moving this course, we have internals at Rust-Lang.org and I keep thinking Ember could use it.

Basically, this sort of a hierarchy of private to public or transient to sort of a free-form discussion forum like this course to something like a GitHub issue, something like an RFC to something like poll request. There's like a hierarchy of how much of those artifacts are easy to search and find.

But I think, you're totally right that there's no reason why, things like the core team needs to exist because at some point, the buck has to stop somewhere. Somebody has to make decisions. Somebody has to actually responsible for laying out the cross-cutting vision for the entire project.

But those things are actually pretty lightweight. The core team when it's doing its job, it's just sort of making an omnibus of everything that the community is thinking at a particular point and making it more concrete. While, I think that's important that there are spaces which are core team spaces, or spaces that are transient, I think, a lot of questions that people have in the Ember slots are important that people who can just jump in and ask them. I think, getting things out into both public and more of permanent artifacts spaces is good.

ALEX: Rob, you are a co-runner of Ember ATX and I was hoping you could speak on the fact that we've gotten some core team members down to Texas to come talk. It’s nice that they're able to share their message with what's going on in the core team. But also, they're doing work. They’re seeing how real people use Ember and then taking that back to the core team. I was wondering if have just want to comment on that and your work on bringing some really excellent people who make decisions down to Austin.

ROBERT: As a meet up runner, like a co-runner, I guess. It's me, Jeff, and Lydia that run Austin Ember ATX. We really like to try and bring people that are deep into Embers core into Austin to talk about the framework that these people work in daily. It's always awesome because whenever you get them there in the flesh, you can ask questions. I guess, we can go back to where Slack is, like you have the higher bandwidth communication but it's even higher bandwidth when you're in person. Getting those people to talk to people that are actually working on the framework daily is I think, hugely important and that's why we work really hard to try and bring out people that work on the core.

YEHUDA: For what it's worth, I think that Rails and Ember shares a common core value, like other projects have, more or less. Ember core team people almost exclusively actually work on Ember apps as part of their jobs so I work on skylight. Having some responsibility in the real world for apps that you are working. It's a big difference in just [inaudible]. I definitely noticed a few.

Sometimes, I'll be working on a project for a while, like when I was working on Rails for 18 months and never actually used Rails. I mean, I used it but not for anything significant during that time. I, sometimes, get into a rut where I'm working on Ember a lot and I haven't had a chance to work on an app at all.

Then, you go back to work on that for a day and it's like, "Oh, my God. There are so many obvious things that I can make better here." Like the kind of things that you would think about when you're working on your framework stuff is not necessarily- as quick it gets, the quickest things that you can fix. The Ember welcome page is a good example of this.

I think, when someone is training, it's very easy for them to notice that it would be great if there was some kind of welcome screen for people. But it's not something that a framework author would necessarily think of on your own. Similarly, getting down to places that are not my usual haunts and hearing people bringing stuff that I just hadn't heard before. Like things, "Oh, that's a good idea. But I haven't heard that."

A lot of that just come from the fact that the core team has a lot of different kinds of users in it so the people doing training, there are people doing apps or people doing consulting, there are people doing rescue projects in that kind of combos is pretty good. There’s a long tale of all kinds of stuff, like people using web sockets for network people using React. People are trying to do Redux in Ember, who knows?

That long tale is impossible to represent all that long tale. In a core team, we try to get as much as possible. It's impossible to represent all of them so going out there and talking to people doing weird stuff and weird doesn't meant pejoratively, just unusual stuff.

Like Ember, really wants to be pretty flexible under the hood. Even though, it's a pretty conventional tool, we want it to be flexible under the hood so I kind of no way of flexibility is but sometimes, I'll talk to somebody and I'll be like, "Oh, in retrospect, that particular thing, I thought that was flexible as missing as little knob that we can add." So I really enjoy it.

CHRIS: Since you've been pretty heads down on Glimmer 2 and you are actually traveling out and talking to people, I'm curious, are you noticing any common themes from the feedback that you're getting recently in terms of what users are saying? Do you have an indication of what the next move might be? Or what people are asking for?

YEHUDA: For Glimmer specifically?

CHRIS: For Ember in general, or Glimmer specifically. But I imagine, you're probably getting general Ember feedback.

YEHUDA: Yeah. I talk about this a little before like the two big areas of interest are mobile readiness and better integration with the ecosystem. Integration is the wrong word. There's nothing wrong with Ember to that extent but people want classes.

I think, those are the biggest picture things. I actually noticed a couple, somewhat interesting things when working with Ember. We ship the Glimmer Beta six weeks ago and we're doing another beta just because there's a couple of bugs that we got that were trickling and we want to make sure we get it right.

I've actually noticed the people have on the one hand, the story of Glimmer is that we're pretty similar to React in the sense that you should think of what we're doing as a top down, you render the whole time and that there are some nice [inaudible] that use Ember.set or the set API, then we are able to do what people should do with component update automatically for you. For [inaudible], then we know, "Oh, this whole area, doesn't need anything to be updated."

If you think about it that way, if you think about it as how can we render [inaudible] around set, I think, you'll notice that Glimmer updates are always faster than React's updates. But people have come to really rely on the sort of quasi-guarantee that if you didn't update something, it doesn't change the DOM associated with it, or even execute code associate with it.

I find it sort of interesting. This is like meta problem, which is React actually got some things right about how to make this story performance. Part of that has to do with not assuming that you need as much bookkeeping as Ember always assume that you need. In exchange, you get much faster initial render and you have to do more work around updates.

We actually have a pretty good story here. Ember.set is pretty nice because it lets us use API that our users are used to, say, generate [inaudible] upon updates for you and that's nice. But people get very upset when things run that they didn't expect, which of course, is not how React people think about it. The way React people thinking about it is, of course, [inaudible]. That's the whole API. It runs until you told not to. In Ember, things run at people who don't expect to get very angry.

I think, you have to be one that I'm thinking about and that's a lower [inaudible]. But in terms of low-level, like thinking about how to shift the mental model of an Ember user so that we can get away with less and less bookkeeping upfront. I still do too much bookkeeping as part of initial render but in order to keep reducing the amount of bookkeeping, we need people to get into mindset of things are fast initially and the tradeoff is that your updates are slower, unless you do whatever.

There are mighty things like React does this wasted time debugging tool or they basically tell you, "Hey, you didn't tell us not to render this but it never render," so you should try and do that. To be honest, I think, having to write something once your component updates, that exactly, "Do I [inaudible] is not okay. I'm not willing to do that." But there are a lot of things that approximate that were more similar to Ember existing APIs that we can find.

I guess, my medium-term goal here is to make it so that we have the sweet spot so that the initial render is always very efficient. I think, we're getting closer. There's still some back problems that we can deal with so. Initial render is very efficient, fast components are fast, and more or less, you get good updates performance until you reach a certain amount of scale and then the escape valves are much nicer [inaudible] before an update. They're basically little [inaudible] where you say, "You know this thing can't change." It would be hard for me to explain. It would feel like it’s [inaudible] we talk about.

We’ve had a bunch of discussion about different escape valves, and the thing I'm most interested in is finding once that feels semantic. Should there a component update doesn't feel like you're describing anything other than React's API. I'm more interested in things that feel like you're talking about your app or your data.

ALEX: Yeah.

ROBERT: Keeping with the Glimmer 2 topic. Glimmer 2 is written in TypeScript, right?

YEHUDA: Yep.

ROBERT: Do you see that creeping its way more into the Ember community? I guess, I kind of want to get your general thoughts on TypeScript and what your experience was writing in Glimmer 2.

YEHUDA: I actually really like that. But the story with TypeScript was that I was writing the Glimmer 2 originally in regular JavaScript and I came back from a long trip. I want to show Godfrey what I've done and I was having trouble explaining some of the interfaces. I happen to know that TypeScript is a lot of it is just interface so I'll just use their syntax and I think, I open the playground and I type in some TypeScript interfaces. Then I was like, "Oh, it's annoying that if I reload, I'll lose it so let me copy the interface into the ReadMe, basically and to the app."

Then over time, like not very much time, I was like, "Oh, it's very annoying that now that I have this, I really wish I could just use it inside of the code." So we started doing that. It took us maybe a week of actually being to be able to use TypeScript for real. But honestly, the code base is pretty big at that point, and the actual was not so bad.

A lot of the reason why it was at the time, TypeScripts like VS Code, TypeScript was still younger and it wasn't a slam dunk. For example, in today's TypeScript, you can just have JavaScript files in your directory and just tell it to [inaudible] that works. At that time, you couldn't so you have to really change all the files and there are some things like TypeScript requires you to specify in classes. It requires you to specify all your fields and I think that's fine, that's good for TypeScript. But you're not going to have already done that in JavaScript so you can't just like rename all your JS files with TS and have a nice day.

So it took as a little over a week, I think, and we also have to write the Broccoli TypeScript thing during that same period of time. That was another thing we have to do.

ROBERT: Yeah, that's a [inaudible].

YEHUDA: Then, with TypeScript change the compiler API a few times so we have a bunch of [inaudible] to do. But other than that initial like [inaudible] to get it working, I would say that it was every single point in time, there was always a [inaudible] win, in terms of what we had to do to make TypeScript happy and what wounds that we got out of it.

You get things like, obviously, people know about code completion. I personally like code completion. I think, it's helpful. But I think, jumps to definition is actually more important feature than code completion. Just like what is this method? I want to look at it. You can jump directly to it.

Also, for me, the code completion of parameters is way important the code completion of method names so when someone teaches you about code completion, they will usually show you, "Oh, look [inaudible]." It gives you the list of all method names, and you're like, "Well, that's fine. But I probably know all of those method names."

But you don't necessarily know all the parameters. Especially, once you start using types, the parameter or information is actually quite rich, like it's telling you this is a component definition, this is a string or this is whatever.

All that stuff, I think, I basically come to realize from using TypeScript that Microsoft has done a really good job with [inaudible] code of distilling down to just what part of ID experience is really good and just bring you that to an editor, that feels a lot like Atom or Sublime, or other than that. So if you get like a pretty good ID experience, without all of [inaudible].

ALEX: I've seen some talk on GitHub about people who want to write their Ember add-ons on TypeScript and I did not know Glimmer 2 was written in TypeScript until just now. I'm glad that was brought up. But we as Ember developers have been trained to use convention over configuration.

The convention is Ember is not written in TypeScript. We're starting to see convention now where logic has crept into the template, or it's not as much as convention as people are doing it right now. I'm curious what your thoughts on that and more is treating Handlebars as a programming language and something that we're seeing now in real production Ember code, so what is the path going forward there because it is happening?

YEHUDA: Yeah, I agree. I just want to say, I didn't actually answer the previous question in full. I think my expectation is we have no interest in ever making TypeScript as rudimentary part of Ember. I think Ember should always work and work nicely with regular JavaScript. I don't want to do anything that would lean to heavily on people having TypeScript around. Certainly, I don't want to do with Angular did, which they use the types as semantic markers for dependency injection and something like that. But I do anticipate things like Ember-metal for sure, Ember runtime being written in TypeScript because anything that's pretty low-level and a lot of algorithms benefits a lot from clearly delineating interfaces.

I think, another thing people don't realize is that there are all these [inaudible] interfaces floating around your code in JavaScript. But like you are in a class, it's pretty easy to document what the class does. But if you have an interface, it's not really any good mechanism for describing it and it can become very [inaudible] and it's Like, "Please give me an object as these methods on it and build these methods."

It’s funny because you don't realize until you start using TypeScript that it's a very recursive problem. It’s like, it has these three methods on it have these six parameters and these parameters have these interfaces and those have these. So you can actually start describing a very complicated thing and it's like those complicated things didn't exist before, they were just very implicit. The explicitness of the interfaces is not you can write [inaudible]. You can write the three interfaces and have the methods with all types of networks [inaudible].

I think, I expect Ember-metal, Ember runtime, other low-level parts Ember, certainly the component library now it's like directly linking in with Glimmer and Glimmer were written in TypeScript so that stuff would really benefit with TypeScript.

In terms of Ember itself, using TypeScript, I think we have sort of a medium term goal of letting Ember apps use TypeScript if they want. I would say that making that story really nice, pretty much leans on ES6 modules and ES6 classes so we have some of the ES6 modules but there's still a lot of Ember [inaudible] whenever module story.

In terms of classes, were still using the old-style class system and that class system is actually just really hard to get the types working in TypeScript. It's hard, period. But like React has a similar problem and there's lot of advance features that only really exists in [inaudible] express ES5 class [inaudible] and TypeScript doesn't have all the features.

My expectation is that sort of along the same path as getting ES6 classes. We will also get a lot of TypeScript support in Ember and I think a lot of people are interested. A lot of people have work on Glimmer now and they're like, "I would love to use it in my app so we'll probably have that happen."

Coding your templates, was the other question. I sort of have a mixed feeling about this because on the one hand, I actually do want Glimmer to be the programming language in production way. So either templating engines are just using the programming language embedded like the ERP. Or they are like Mustache or like the general templating engine -- forget what that thing is called -- but there's a bunch of template engines that use like the curly syntax and those things aren't very rigorous in terms of how they think about scope.

Like lexical scope, it turns out to be a pretty important thing about how programming language work. If you have a shitty scope story, people don't have a good sense of what's going on so like the Angular 1 templating story was you basically find your scope on wherever and you attach to a part of your template and that basically means that if you're looking at just temple, you have no idea what the actual variables mean, anyway, because any part of it could be choking up the scope to be whatever.

I start with Handlebars but have refined a lot overtime and I'm pretty happy with it now. The Glimmer templating engine is basically defines a programming language. It has scoping story. If you want a variable, name it. Use the as-pipes syntax and you get some variables there. Your function, your output, your components, and your helpers are functions and sometimes it's written in JavaScript, just like in Ruby cellular functions that are written in C. But ultimately, just like a C extension, it can't magically change the scope of Ruby program, helper or component in Glimmer. It can't magically change the scope of your template.

With all that needs, if you look at a Glimmer template, it's actually really clear what are the names mean. That sound boring but ultimately, that's one of the things that needs to you look up at a lot templating engine and not only really know what's going on. I don't understand what's going on, without thinking of what every single one of those custom helpers is doing.

I think Jinja is the name of the templating engine in Django. If you look at some of the templating engine like that or even like Handlebars before Ember 2.0, you really have to go that like 'for x and y' is a magic syntax that needs a particular thing. Because a lot of these templating engines are very flexible in the sense that they let users have or whatever, any particular piece of syntax could actually be creating random names that mean whatever.

I've just found, having worked through that in the Ember community, I'm very happy that we took the time to get that stuff solid. One of the nice side effects of that is that it makes some of the usual optimizations that people do on code work very nicely. Once we know all the names mean and know all the scopes mean, things like in lining specialization from invocation. This component is invoked here. It has these parameters, but those parameters are all strings and I can see the receiving end. The receiving end just fix those strings into attributes or something. At compilation time, like to combine all into one thing. This is an optimization that we haven't done yet because we been working on compatibility.

But there's a lot of normal 'programming languages standard optimizations' that we can do because we design Glimmer to be a programming language. The fact that we've done that is actually mean the root cause of people doing more programming language stuff in their templates, in older versions of Ember before, we had done such a good job of rationalizing everything.

You would start doing that stuff and you would start hitting clips where the behavior didn't work the way expected for some reason, then you just use a sub-expression as something and then depending on exactly which things you put into the slots, maybe it didn't update on the inside of the template. We always consider those things as bugs so we would fix the bugs that we encountered.

There’s an explosion of different kinds of things in a programming language and if you don't model variables as variables basically, then it's hard to know callbacks are callbacks, variables as variables. There's a lot of doubt so they would start using the APIs as part of them [inaudible] hit someone and they would pull back.

But today, the implementation of Glimmer, especially with Glimmer 2 is extremely [inaudible] standard, and what that means is that you want to do very [inaudible], parenthesized expressions with as type something and then you want to go and take that and send to the component and you have that and you put that value and put it to a service and you do all that and it works. That doesn't necessarily mean that it's a good idea for your template to be in a large program. That’s generally that. I think that's main question that you asked me and it is generally, for things to get really complicated.

But I think there is a reason for it and I think it's something we're doing to make it better. I think, the reason for it is that, if you think about the Glimmer core language, the main escape of what you have to get complicated expressions out of Glimmer and back into a reasonable place is helpers.

Let's say you have a big bunch of [inaudible], and/or whatever people are doing inside of their Glimmer templates and it's like a multi-line thing and you want to pull that out to a more reasonable place. The natural way to do that is to make a helper. But it's also pretty common that you have a complicated piece of expression that isn't reusable. It's not being reusable for multiple places and helpers in Ember right now are global so you may have this big block.

Another [inaudible] of computer's property, but computer property's don't have a nice- like I have a few internal parameters and I just do something that they always require you do something against it, the components off of this, and that is usually what’s you are doing. You're usually have what is effectively a function with a bunch of parameters in it and those parameters combine some way like with 'and', 'or', maybe there's some comparisons like 'greater' or 'less than' or whatever.

One of the things that comes out of module modification RFC and makes everyone pretty excited is the ability to have helpers in your templates that are local so it's a helper that is just sitting right next to your template. I think, those kind of things -- helpers that are right next to your template -- will give people who like you do not want to see so much code in your template, a much nicer story to explain to people why we're doing it, and what we should do instead.

A template has a three-line, fifteen is privacy expression and you want to convince the person not to do it. The re-factor is literally just to make a local helper, put the code in there and use the local helper. I think that's the better story to explain to people than, "Hey, I don't like that. It feels bad. What should I do?"

CHRIS: This brings up an interesting point that we actually encountered recently at The Frontside, we use helpers a lot and are frequently trying to get them to do all sorts of weird things. But recently, it occurred to us that one kind of blocker was that you can't compose helpers.

We had an existing helper and we kind of needed to make a helper on top of it and it suddenly dawned on all of us that, "Oh, we can't go up a level." We can't build a helper and its abstraction over another one because you kind of have to hit the dump. You have to have a component or a template or something to invoke that. Is that's true?

YEHUDA: I would expect that you could call the function with the arguments that looked ugly but I would suspect and I would say that you can call it and just pass the parameters and hash these [inaudible] and --

ALEX: Like a helper that invoke or something? Because to write a new helper, you would have to go back to JavaScript.

YEHUDA: That array an object. I expect that to work. Do you just need a helper calling another helper?

ALEX: Well, a helper composing, like composing several helpers into a bigger helper and then just calling in the template, much like how you might write some functions and that write a function that closes over them or something.

YEHUDA: Are we talking about a helper that takes the helper?

ALEX: I guess it could be. I guess, higher order helpers could be a thing in this case.

YEHUDA: Helpers that call helpers are helpers that close over helpers really should just- helpers are more or less functions. It would just be able to use them directly. I think, what you're saying is an interesting Ember phenomenon. I actually don't know which of two categories this discussion is in.

Basically, either. Either there is some good reason why this turns out to be difficult and there's like a missing API that usually fix it. Like in general, my mental model here is you're talking about a function so if you can just [inaudible] the function, it really should work and in that, it is hard. There's something weird.

Basically, what I'm saying is either there are some 'gotcha' that how it was structured right now, that's like an accident. That means they're not exactly as much like functions as we think or the mental model that you have using them doesn't make you think of them like functions. So you think that there are hard to compose but actually there are functions. Basically, those two things are equally likely Ember. I think that's unfortunate.

ALEX: I think that with them, you can certainly combine them in a template and you could even pass one into the other. What we ran into an issue was, similar to functions, if you are writing like an npm library, you may write five functions and only export one of them. That one function will compose the other four and call them.

In some cases, you may not realize, do you want to export that composed abstracted function until much later and it's okay, because you just import the function from somewhere else, then you say like, "All right. I'm just going to consume this function from another and now I have this combination put together."

Well, with a helper, if you already have a helper that does something and then you realized, "Oh, actually, I need to build an abstraction on top of this helper in a new helper," we so far have not found a good way to combine them like that.

YEHUDA: So probably, it will not succeed and having this conversation through to the completion year. I'll just say for listeners, what I find surprising about this, although, I can believe that there is something that makes it true is that helpers in Ember are really just functions and increasingly they really are like helpers in Glimmer are functions. They have a weird arguments signature. They take positional parenthesis in array and named arguments as dictionary and that's a convenience basically because otherwise, you have to scrape of the last parameter and try to figure stuff out. So that's a signature issue.

But generally speaking, helpers are supposed to be functions and if there's something that you could do with regular functions you can’t do them with helpers, that sounds to me like something is wrong. Like I said, I can definitely believe I should talk to Chris about this and he should write a blog post about it. I can definitely believe that there is something that makes it true but it's hard for me to imagine what it is.

CHRIS: Well, I'm glad to talk to you about it offline.

YEHUDA: Yes. We will figured it out.

ALEX: Yehuda, thank you for this deep dive into open source process into Ember. I could drill deeper into your brain and just extract as much knowledge as I can. I hope to be able to do that someday soon.

We're going to wrap up for today. Thank you very much for Yehuda Katz and your time.

YEHUDA: Yeah. Thank you. I apologize for people who didn't understand half the things I said.

ALEX: Yeah, it's a podcast for everybody. But we have a lot of Ember developer's listening and I'm sure that they loved it and hated it all up so thank you very much.