Keynote Panel Discussion: SMACK in the Enterprise - Moderated by Elizabeth K. Joseph, Mesosphere

[0:00] So I just want to quickly have you each introduce yourselves. say which company you're from and a little bit about what your company is doing. Just briefly before we get into the questions, maybe some of your own background. So, well. Start with Devin. So my name is Stefan Bauer. I'm coming from Audi. I'm heading a department that is taking care of um, let's say, collecting and processing, all the data our cars are producing during the usage of the car. So... We're primarily building up a platform which should be able to be very flexible and to handle a lot of data.

[0:36] of the ZB people. be able to make good new product for our customers. So personally, I'm with Audi since-- I guess 15 years. We have a history in software. of course, mainly in embedded software, and now we are spreading out into the backend software as well so that the car is part of the IoT world. Yeah, I'm Hubert Fischer, also from Audi. I'm the senior project manager of the IoT project collecting weekly sensor data. I've been in several projects before within the company and now I'm part of the transition the company goes through from a pure hardware developing company to an IoT company.

[1:19] Hello, my name is Oliver Goldisch from Deutsche Telekom. I'm a manager in the networks and service area. We are using DCOS in particular and this Mac stack for, uh, creating more value for our customers in terms of selecting access networks. So we're I can briefly give more updates later. But, um... Yeah, we're using smack stack for that. Yeah. Good morning. My name is Christian Worps. I'm product architect or within ASML.

[1:51] So I'm doing this for a project which is a compute and data platform. We are, I think, quite new in the SMAC stack. So we started beginning of this year. and we are not using all of the letters right now. And so I'm eager to learn and exchange information throughout the next two days. Great. So we'll start again with you. I sort of wanted to ask, as much as you can share about your infrastructure with regard to what technologies you're using, the size of the infrastructure, whatever you're comfortable with sharing, I'm sure.

[2:28] Many in the audience would be interested to know. Yeah. Um... For us, the challenge is we are a Dutch equipment manufacturer which are sending out machines to our customers. and these machines are operating and creating a tremendous amount of data and our compute platform now needs to be able to properly scale and compute and storage. but also needs to be able to properly actually tackle individual failure modes down to cross data center availability.

[3:04] And that's-- We have chosen Mesos for an infrastructure that we can actually operate the-- compute and the data capabilities. So we are-- of that stack we are-- using Mesos and the Kafka part. We are heading out to the Spark computations in the future. And that's what we are-- Josephine. Oliver, what do you tell us about Georgia Telecom? Yeah, so we had a problem that we started about two years ago. I think everyone knows it, at least...

[3:41] We have all of the Wi-Fi password here on our badge. But actually we try to automate that so that our customers can get into Wi-Fi networks but with a very good quality. So if the Wi-Fi is really poor, you don't get there. So what we started to build is a mobile application that collects enormous amount of information about the access networks. you're facing as a customer, and we want to give recommendations to own on a mobile application deployed, which X networks you're going to be steered.

[4:17] And this was quite a challenge to go with an old stack infrastructure. And since we're We expect a bigger customer base come there. The distributed way of how the SMAC stack is designed gives us quite a lot of opportunities here. Yeah, so we have millions of cars out there equipped with onboard communication units for data collecting. So they're listening to the bus systems in the car.

[4:48] and sending those data generated out to a backend. So backend is our project to set up. We use all the five letters of this max stack. to ingest those data, to do real-time analytics of those data coming in, identifying patterns, and cause reactions back in the car to the customer. and we do use Spark for that specific purpose.

[5:20] The whole infrastructure is currently based on Amazon, but we have the approach to be and to stay cloud agnostic. And for that, um, DCOS helps us a lot. to spread the load between different cloud providers to get a perfect exploitation of the cloud resources which are are expensive and we stay independent from any cloud provider, so we can switch from one tool to another in using DCoS.

[5:56] Great. Maybe let me add some more aspects. as we are a global company, For us it's very, very important to also to have global solutions. And for that, we know there are different countries where maybe we can use MSR, maybe we are not able to use, maybe in the look to China, there is always a little bit. let's say, legal-- IN CERTAINTY. what you can do, so it's very, very crucial to be able to, independently from the single infrastructure, infrastructure provider.

[6:28] and even to have the possibility to make some hybrid solutions as part of the Volkswagen group. We also have a lot of internal capacity for data centers. So in future, I-- I had the mindset we really could use both internal, external data center, for this purpose. Great. So I think I heard a little bit from a couple of you using the full smack stack. Spark, Mesos, Akka. Cassandra and Kafka. So you guys are using all the letters, I think, most of the panel here.

[7:02] And then you're using three of them, right? Yeah. Okay. So I guess specifically speaking to the SMAC stack and the collection of technologies, that is, what has that on the application level, you've touched upon that a little bit, but if there's anything you'd like to add, what has that specific data-driven pipeline done for you. Well, for us, I guess the most important thing is The fast, the fastestness. as we want to do our analytics here real time. So we really want to go out and to to go into the field of being able to react immediately on situations that are happening in a single call.

[7:42] And with that, we have to be able to-- to know what's happening outside and to be able to to get the knowledge that we gathered from the rest of the fleet to be able to send directly to the single call. So that's crucial for us. Yeah, and next to the whole analytic stuff, it's important for us not to lose any data. You can imagine we have millions of cars sending in seconds millions of data. So we need some kind of infrastructure.

[8:12] which is capable of collecting those data and not losing any of those. Yeah, sure. And just to add, the distributed workloads are quite an advantage of this Mac stack. Oh. And especially if you don't have customers there, you can really scale down the infrastructure as well, which is not so bad as well. Yeah, which sort of leads me into my next question. We'll start over there. It's sort of talking about the application level, but also the infrastructure level, how using containers and DCOS and Mesos and the SMAC stack has really helped you build out your infrastructure and plan for the future. If you could speak to that a little bit.

[8:52] Thank you. So what we are looking forward to and what also the last year of that project has shown that we gain actually the capability and the flexibility to scale, to have the high availability, to also dynamically use and reuse the resources that are on the hardware level available for our applications. and what we build within our compute and data platform. is the foundation for the applications that are built on top of that.

[9:23] So also we are providing a middle layer towards our application. of developers, actually. Sure, and just to add, this was not an easy road to get everything in the containers. Did you do it incrementally, or did you do it like one big switch over? We built it completely new. New one? Because the mind shift is quite hard. Well, Tech goals maybe sometimes uh, occur to be a little bit old fashioned. So when I started the project, we asked, you Yeah, and where's the VM? So-- Yeah.

[9:55] Yeah, and since talking about containers, we're looking forward to software-defined networking and software-defined storage. So that would be a really good next step for the whole SMAC stack. Thank you. So as you're looking, we've got a lot of people in the audience who are planning on moving to the smack stack or are just getting started. If you could talk about some of the challenges or successes that you've had in your migration over to a smack stack people can benefit from. Thank you.

[10:26] I guess one we have in our history, I always could recommend. try to start on a greenfield. Um... to really get up from the scratch to be able to to get all the advantages you get out of such a smack sack. And the second issue is, start with some POSCs, of course, on an open source basis. but we really want to be in the enterprise level So for our companies, we are big companies in the round here.

[10:58] You need professional support, you need professional services, So. as soon as possible change to the enterprise versions. Yes, it's not advertising here. I didn't tell him to say that. No, as you see, in our mind, when they go to my board and say, OK, you're introducing a new architecture, new software, So-- versus support structure. who could recall when something is going wrong, what about a down time, what about some issues, some bugs, whatever. Because a lot of the board members think open source is something for freaks.

[11:33] Something for, yeah, you can do it at your home-based software development, but nothing for enterprise. So-- to to convince them that it's a professional way of working Enterprise versions are very helpful. So to quickly go back to Greenfields, did you have existing infrastructure that you had to move over, or were you able really to start with a fresh project? Thank you. Yes, we were lucky. and be able to start with a fresh project.

[12:03] with a clean Amazon infrastructure. to add there. We had the same challenge. We needed to convince our management that we don't go with our operations team, but rather deployed on Azure because we're faster in using a public cloud service. And we directly started with the premium support--with the premium enterprise licenses. and it's just easier with the support of the vendor. We iterated in very small cycles. This helped us a lot.

[12:34] to go too far. Just one step ahead. And did you have a legacy infrastructure to move over at all, or were you starting from scratch? Everything new, yeah. Everything new? That's the easiest way. I'm sure it is, yeah. How about over-- So yes, we are two-way actually. So we do have a hardware platform that we currently ship to our customers. where our software is running on. but we have to grow out of that Infrastructure? So we move into our customer's data center with the deployment of our platform.

[13:08] And the second aspect is that we do have to keep the investments that we had already to the compute stack. So we have to first also create new processes together with our customers, and close collaboration. And second, we have to make sure that our investments stay still active. on the compute level. So the migration that we did from the-- Java Enterprise world into a more fine-grained containerized architecture. It works. It's doable.

[13:40] And this is what we did in the past. Could you give a few more details about -- or were there any challenges, specific things that you could give advice? About the migration? The two challenges, it's also a new software stack, also within our organization. So you do have to do a lot of-- Yeah, convincing people and showing that it really works. It's a different way of doing the development. But yeah, you have to do your homework in order to bring that to the proper audience.

[14:15] um... It's also like kind of community management. You could say so, and the second aspect is On the customer side, they also have the same challenges. from moving the... Yeah, the ownership of that hardware stack from the business people to their own IT department That's also the... things that you need to communicate. You have to be open there and, yeah. It's doable. Yeah. Yeah, it's like we heard in the last talk. It's not just about the technology. Communication is really vital with these projects. Exactly. I couldn't subscribe to that.

[14:50] Hmm. Thank you. Maybe let me add communication and trust. So I think it's For us, it was very helpful to-- to be really Could rely on our on our software developers and our architects to say, okay, this is the right way we want to move. And don't bother them with too many comparisons, too many POCs, too many, oh, please, could you give me some more insight, why we exactly use this specific product and not those? out of the 20 other products.

[15:21] Trust them. And to add as well, one challenge that we had there is actually you need a completely different skill set of people that can work on the SMAC stack. So it's quite challenging to find good people that I actually do the work. So that's really a hard one. That's the reason why we have an HR booth out there for some of you who are interested in. I mean, have you found you've been able to train some people internally already to get them up to speed? or, you know. And last but not least, we have to mention that it's not purely a technical issue to do this shift. It's mostly a cultural shift necessary in big companies and also a shift of the work mode, of the skills that you've already mentioned. So there's lots more than pure technology.

[16:11] Yeah. One thing to add with regard to communication, I think Ben yesterday in the DCS afternoon. had one sentence on the slides which talked about communication. And actually, there was one-- subject that you could exchange there with any kind of Yeah, with any subject, but it all came down to communication as a key. Yeah. That one I really liked. All right, so last question I had for all of you is sort of a future looking one. So implementing the SMAC stack now, it's running well. Where do we see the SMAC stack in a few years? What is the SMAC stack 2.0? Do the -- does the acronym change?

[16:57] What are you thinking? Maybe, but for us, important thing for SMAC stack 2.0 is to get the stuff I've already mentioned, software defined networking, software defined storage, that would be a good thing. And the tight integration of the products together would also be really a huge advantage for us to avoid downtimes while migrating to higher versions and things like that. And that's the components inside the SMAC stack themselves? Integration or the other?

[17:29] Yeah, the SMAC stack itself and each application within And the other thing around is that we have to implement more and more components into that stack regarding AI. capabilities. That's one of the crucial factors for us to succeed in the matters of envisioning environment, building map layers with real-time events on it to let the car drive on its own.

[18:03] in the future. Yeah. Um, SmackStack, I guess, will not change. I can still download LAMP. Probably the package still different things in it. but the label sticks the same. I think that a lot of the majority will do or will be done on the infrastructure level. like new orchestration frameworks appearing. This is more where I see future-- something's going to change. Thank you. Um...

[18:34] Looking into the future, I hope that there will be a SMEC enterprise. Doesn't matter. So if you really build products that are running at your customers, you have to take care about maintainability, monitoring, security aspects, encryption at rest in transit. And if you would like to add all this on your own, Then you actually build this Mac enterprise. Thank you. Thank you. All right. Thank you, everyone. This was great.