A huge kudos for keeping up your legacy support for 3 years! Making needed but intrusive architectural changes is hard for any open source project, and gets harder when you have paying customers. You typically never get praise for doing the right thing...thank you.

Please do not take anything I said as an attempt to throw shade or being critical of your team! As a non-paying customer, I have to say I'm extremely happy with the support you've delivered (and consistently.) I wish I could have got my company to throw money at you, but it did not work out.

The kind of support I got from Deis the company is really without comparison when it comes to Open Source projects anywhere else.

The fact that v2 is a drop-in replacement for v1 really eases the sting of the fact that there is no direct upgrade path. I still have an old Deis v1 kicking around because I left the company, transitioned to hourly, and made an agreement to draw down my hours at this company, where we are in reality hardly using Deis at all. But in the small capacity we are using it, there is what I'd call "VMware Levels of Reliability" and so it successfully became a piece of the infrastructure there.

So it is disingenuous for me to say that I was not able to upgrade my v1 installation. The reason I was not able to upgrade is because there was not strong interest in upgrading. The unsupported product is as good as our (also unsupported! but luckily not End-of-Life) VSphere and VSan environment.

It is a different product to me, in short because I am the one administering it. It runs on a different platform altogether, it has no distributed filesystem component where the old version did, and it has not really harmed me in any way that there is no upgrade path. It is just a couple of facts that led me to the conclusion that they are in fact distinct and other products that are not directly related to each other, except that they could easily pass for one another if you asked a user.

I am really happy for you guys, Microsoft is a real big name compared to EngineYard, and while I could get behind EY+Deis, it's a hard sell for the Design Review Board. But I can tell them "look, Microsoft is doing this now" and they will know what that means instantly. Big guns. No joke software.

This is how I've actually felt about Deis from the beginning, but now it's going to be a much easier sell to get Big Wigs to sign off on. Nobody ever got fired for buying Microsoft!

Not to mention (but I'll mention)

Your announcement to End-of-Life Deis v1 came what seemed like days before CoreOS announced their decision to kill Fleet. So, not like there's anything you could have done about it, save deciding to pick up supporting Fleet for yourselves.

(And I like fleet, but I understand thoroughly why it was a good decision for CoreOS and for Deis to end support for it. It was a wholly inferior solution, begging for a replacement.)

If it's not totally clear, "VMware Levels" is also meant as a compliment, !

> The unsupported product is as good as our (also unsupported! but luckily not End-of-Life) VSphere and VSan environment. > It is a different product to me, in short because I am the one administering it. It runs on a different platform altogether

Distilling my previous comment, it's really this.

I navigated the waters of Fleet and Deis v1 to find an answer to "how can I make sure this does not go down with lots of lead time and half a dozen warnings well in advance before it does go down." (Aside, my datacenter at the time had famously reliable electricity on two grids that just does not go down.)

Now I have to renegotiate that position to get the same guarantees with Kubernetes. Before, I was worried about maintaining etcd quorum when N machines go down, and preventing split-braining. Now, I'm still worried about those things, but they're behind an abstraction layer of Kubernetes API and new suite of tools for managing it.

I'm not expected to manage my etcd quorum in the same way. I am sure that's good news. Or I am still expected to manage etcd quorum, but it's buried behind a mound of Kubernetes so it's hardly even clear that there is ETCd running at all on a basic cluster without multi-master. You couldn't have a Deis v1 cluster without running at least 3 instances of etcd. You were forced right away to get to know those failure modes.

If it's not clear yet, I am a really small-time consumer of high availability.

The new system also forces many best practices on me, in ways I'm not accustomed to. (More not bad things...) I was once advised to split my control plane from my data plane (and possibly also my routing plane) back in Deis v1, to ensure reliability. I never got around to it with Deis v1... but in Kubernetes it's already been done for me by kubeadm.

I had a 5-node fleet cluster with 5 etcd members that was "all control plane all the time" inside of what amounts to a single AZ, and it's pretty clear that this is a totally wasteful design now. You wouldn't build a K8S cluster with 5 masters and no minions. But what's 40GB of RAM in a private data center? To ensure reliable service, sure we had that kind of RAM just laying around. With Deis v1 we did exactly that. For a cluster the size of mine, I'd say it was a thoroughly researched and well-advised decision... at the time.

It's clearer to me from talking with you that, from a code perspective, there is just way too much code in common to call it a brand new product. For a cluster admin who doesn't get very deep into the code though, I feel like it's a much easier case to make that it is in fact a distinct and new product. The constraints have all shifted, and the guarantees are in quite a lot of cases not the same.

This all amounts to basically hand waving though, and I'll reiterate I am not passing judgement on the progression of Deis/Workflow and its place in the container ecosystem.

It's been a wild ride! Thanks for bringing along a community with you. The marching forward and continued progress of free software such as Deis and Kubernetes is a thing to behold.