Panel Discussion: The Business of Data Science

[0:00] So let's go right back to Kleena. The first one, where are you there? Kleena. So Ahmed texted in, Great question. And he asked, was there any internal cultural resistance to the data-driven change that the Irish Times has been going through? And if so, how did it overcome, how did you overcome it? I saw that question there on Twitter, actually, and it is something that we do get asked quite a lot. Absolutely, there was some resistance to the change. And what I would say on that is it's a constant exercise. So this isn't something that happened overnight.

[0:38] it's still ongoing. So a couple of things on it I would say, one is looking at others in the industry. So we would look very closely to the New York Times, to the Guardian, to the FT and see what they're doing. And it helps our case when we're saying we want to bring in more of this or showing reporters the performance of their content. It helps when we know that the Guardian have done this or the New York Times have done this and seeing it that way. And the other thing I'd say is So when we started out with the analytics function, the company was about to embark on a digital subscription shift. So the team was set up a year before that. Up to that point, we didn't have a relationship, a direct relationship with our readers. They went into a news agent, they handed over their money, they bought the paper, and we didn't know about them, what they were reading, what they weren't reading. So it was imperative for us when we were setting up that function to know about our customer. So we were able to go to senior management and others and kind of say, you know what, here's the number of articles people are reading, or not here's the impact that we'll have on our traffic and our advertising so showing a couple of quick wins and bringing people over and again go max the the communication piece really really important it's all about relationships and it is about talking to people and the reputation of the team and the people you hire in that team are representing operation yeah yeah it sounds like the the other teams are nearly hungry for that day so now and they're looking forward to seeing their stats yeah and that's the that's the important shift because we could have bulldozed in and then

[2:08] and you'll have that fear of it. So we have to treat it in that way, so... You good? Um... I was really fascinated, Kritika, about the Disney journey and the customer experience. I know a few people over from America, and I think Orlando is like the new Taito Park with people visiting it as often as they can. But I was just wondering, I think Disney is a great example, but could you bring a more concrete example of how that applies to Davey and your customers and your business opportunities and how that relates to your business a little bit more? Yeah, so we're probably in one of the early stages. So we're mapping out personas at the moment.

[2:54] So we're in the -- how do I put it? We're in the gathering/connecting phase in the implementation side, but we've had a good strategy session. We've got the senior leaders on board, and I think it's at that stage where we're trying to see the data that we have and the personas that we think are the target audience, and do they actually match up, or are we... Do we understand our customers well enough to even define our target audience in that space because I think wealth management as an industry is not really recognized in Ireland. It's a big deal in the UK and the US, but I think to understand wealth management, it's still bucketed with...

[3:35] you know, stockbroking and private banking. And so I think it's, there's a, there's a growth in the brand awareness that has to happen in the industry awareness. And I think as part of that, we will probably reap the benefits or at least under help, uh, bring that market up to speed but yeah so in davie at the moment we're um we have i hope there's someone from my team here hello but the guys in my team are at the minute working with the strategy and the cx team to help build those personas at the minute so and um at the end of the day you know senior management just want to sell more or increase their margins and you know retain their customers what kind of metrics and how does your work you know feed into that can you prove the ROI of the work you're doing or was that to come down the line? Yeah, no, I think the agreement on the metrics happened. So that is definitely clear. It's always hard with marketing campaigns to prove ROI. That is probably the trickiest bit, especially when you're always going with last attribution, you're always going to attribute success off something to the last action someone took. So even though the journey might be long, it's probably the very last download that they did

[4:52] benefit are the or the attribution of the success. So I think that's where we have to crack that a bit more and understand how we can... improve ROI in marketing campaigns. But otherwise, I think the other metrics we are working through, but it's mostly NPS. So it's very much customer satisfaction and engagement driven that we want to try and bring through and get senior management to agree on that, not just business benefits. Absolutely. I think sometimes you have to play the long game, like Disney would do, you know, delight the customer with...

[5:25] Sometimes you don't see it in the metrics straight away, but over time that builds up a huge loyalty and a kind of fan base nearly for your brand, right? That's what you're trying to do. Does anyone else have any thoughts on that? Maybe you do similar UX, customer experience work in Accenture. Do you have a similar view of that? Yeah, and I guess in the Docker multidisciplinary, and I guess the user experience is key. So we have a whole, we bought the design agency Fjord a couple of years ago, so we have a Fjord design agency. And the teams that we put together to solve what might be seen as AI or analytics problems always have a designer on them to make sure we're understanding properly the user experience.

[6:05] I think it is the realization that great products and services, people have so little patience for a bad product. I love the idea that the baby, like, try to swipe the magazine. And, you know, if technology doesn't work, they just have no time for it. Obviously, the magazine doesn't swipe, but it should and it will someday. So, yeah, they're just ahead of their time. um, So we're going to move into a few more slightly interesting questions came in. And this is probably generally for anybody who wants to take it, but it's quite...

[6:38] modeling focused, Elpida Vantra, who we know well from CREM Global. She asked this very interesting question, and I think you can interpret it a few different ways. Do smart people build smart algorithms, or do they collect and feed their models smart data? Thank you. Maybe I'll take that one. I can't say yes or no. It's... I can't say you don't need smart people to build smart models, that wouldn't be a good thing to say, but the models are as good and as limited as they're built to be based on their algorithms. The data is absolutely crucial. Data will always beef Okay, I better not say it. Take out the word always. Data generally beats a better algorithm. So when you're trying to get a good model or a good predictive model, for example, it's the data that you're using.

[7:34] In other words, what you're training on that will make the model good. So when you're trying to get the right sort of algorithm or classifier or whatever, try different ones, but the difference will be in the data that you're feeding into it. And to come back to the smart comment, you can't blindly use data in a company, and I know that sounds very trivial, but you have to have people... especially if you're using sort of structured quantitative data, who understand the data. That's absolutely crucial. Because sometimes the domain experts are kind of left out, but it's very important that they're part of it. But data generally trumps and algorithms generally. - Heard that here first. - And a quick question, and it came up, Susan, in your speech around the CDAR model about the deep learning algorithm that anyone could use. I mean, and this is an open question, but is there a danger if you make algorithms possible for anyone to use, that people start using them that aren't thinking about the data, that aren't thinking about biases, and you end up with models out there where experts who don't know the things that you need to take into account haven't been consulted?

[8:36] Yeah, completely. And I think that becomes even more of a risk with deep learning, because then you don't necessarily even need to work closely with the data before you feed it in, because deep learning will actually go and figure out what the data is about, too. So yeah, that is a danger. And it's, it's, it's something that comes back to explainable AI. And it also comes back to your model will only be as good. to go back to it again, as the data that you feed it in. If you've got bad data, it doesn't matter how clever this algorithm is you feed it into, you won't get good decisions out.

[9:08] Because it doesn't know about them. It cannot make good decisions based on bad data if it doesn't have the right information. And to know whether your data is good enough or not, you have to have people in your company who understand what data you're gathering, what it's about, and to clean it and to be able to sensibly understand what each part of it is about. It's absolutely crucial. Then try it in a few models. Absolutely, yeah. Make sense if there's weird things going on. Revisit the data. I think that's crucial. the same question, may have just asked it, so Kian consider your question asked. And onwards then to another question from Darren who is here to speak from Boston actually, and this follows on from that key issue of data and he said how do you then convince leadership to invest the time and the money and the bandwidth in gathering and structuring and cleaning that data?

[10:01] that it is expected to be ready to go. That's the 80-20 thing. That takes your time, right? So any thoughts on that, how to get your organization to invest in that and spend time? Any thoughts? I think from an Irish arms perspective, it was a case of starting small. We used a lot of open source tools at the start. We didn't go in and say, we need this much investment and this much resource in order to deliver this. We very much started small. Started with the information that we had, did a bit of cleaning on it. And again, showing those kind of results and those quick it so from an Irish times perspective we publish over 200 articles a day it's a huge number of articles and content is is what our product is and so what we've done is really to make that day make that content work harder for us so we're publishing that money anyway and we're publishing them at a certain time of the day so by staggering the publishing the promotion of them and looking at what people want we've we have exponentially grown the audience for that for that content so that's showing value back and that gives us leeway then to kind of say well we want to do this next we want to invest in

[11:03] because we now have subscription product as well. We're able to show what our subscribers are reading, what's converting subscribers, what content is or isn't. So that really helps us now that we have a product that we are selling. It helps make that case a little bit easier as well. great to have that number of subscribers so you can see those trends now where people are buying the paper copy, you know, you might look over a few people's shoulder and try to see what they're reading. I always do that, I'm very annoying to sit down in the train because I'm looking at what people are reading but yeah, how they're consuming. You can measure that. I think another thing is I guess if you can find examples in the media of people that didn't have good data and where they get called out and make sure that people in your company know, okay, you have good examples because you were able to get good data, but actually this is what happens when you don't. The other side, yeah. So the metrics as well is very important. If you agree that you're going to show improvement in something and you actually prove that, then you can say, I can only show it to a certain point because of the bad data.

[12:02] Yeah. Cliona, is there a risk of media, maybe even the Irish Times, becoming too data-driven and optimizing everything to chase those eyeballs and advertising revenue that comes with them? Yeah, it's very important for us. And that balance in terms of the ethos of the Irish Times is very, very core. And again, going back to how we didn't want to bulldoze in and say that everybody has to use this and use this data. and chase me clicks in that way. And we'd never be writing about something that's happening in Syria that's important for our readers to know. So we're very core on that. And there would have been reluctance in that sense of it. And it's really because we have to trust our journalists and trust their ethos of what they are working there for. Again, going back to the subscriptions being a help, it is not about the biggest traffic drivers, what is the biggest readership. We do look at other metrics like engagement time, and we have a long read, piece Rosita Boland, one of our journalists, wrote a few months ago about Anne Lovett and the death of Anne Lovett and something like that that had, you know, people reading that for, say, 13 minutes or whatever, reading the article. That's much more important than the pages on that and, again, having the feedback that she would get. We talked about comments earlier, the feedback on Twitter and other social channels. And also, again, that subscription piece that we need to attract subscribers in, people to pay for our content, and we need to retain them. We're not going to retain them if

[13:32] we are writing click-based content in that sense. And I suppose going back to that idea of the 200 articles, It's not about changing the content as such. It's does it have the best possible headline to explain what it is for people finding what they want to find? Does it have the best image in it? And are we making it easy for people to understand in that way? So it's not aiming towards dumbing down the content, but making it work harder. Sometimes I notice the headline changes on the Irish Times. You've seen an article and all of a sudden it has a new headline and you go in and realise, oh, I've already read that article. You come up with a good headline.

[14:08] isn't it? There's a skill in that and there's also there's differences in terms of the headline can be for from a Google search point of view the everything you need to know about the Pope's visit was the headline that was needed for that and we would have looked at Google trends to see are people searching for papal visit for Pope's visit is it Dublin is it Ireland what is it and to get those right keywords we do use headline testing tool as well but there can be differences in how you promote something on Facebook versus you know how it is on on Twitter in that sense like you have to have the team names, you have to have these kind of things, but then when we do kind of first person personal stories, things like with a question mark, we notice that headlines with question marks in them work better than those that don't, so there's trial and error, but the thing because we're looking at data in real time is that we can see, we can change a headline, we can test it and keep going, as opposed to saying, if we'd known that yesterday, we wouldn't have done it.

[15:00] we would have changed it. Very good. Just moving slightly then over to the skills. And, you know, a lot of people here would have young kids growing up at various ages and various stages of development in their education system. And it's always interesting to think about what they're learning in school and what the world of work will actually require from them in the future. So what are your thoughts on what we should be teaching our kids beyond maybe above and beyond the core curriculum of, you know, basic education skills that they will get for them to prosper in this brave new world?

[15:36] So. I'd like to start with that one. I mean, I guess what we're seeing in the doc, and I think it's probably across the board, I mean, experimentation and curiosity are key. So with things changing and things are going to keep changing, just... learning a set amount of stuff. Obviously it's important that you know the basics of your skills, so if you're a data scientist you know about algorithms, but actually knowing how to learn continuously and being interested in learning continuously is really important. So I think in the Irish education system we're quite good at it in primary school, then as soon as kids hit secondary it turns into rote learning for exams.

[16:24] presentation and the principal was talking about these are the subjects you do and for leaving CERT and then she got to fourth year and she said actually your kids long after they forget what they do in the leading search, they'll remember what they did in fourth year. Exactly for that reason, because it is, you go back to the curiosity and experimentation. I think things like Coder Dojo are brilliant as well. And I think it's to integrate more of, not necessarily pure coding skills, but having that... mindset of actually wanting to crack something open and, you know, the curiosity bit, but trying to build something from scratch is such so empowering to kids and building an application or building a mobile app. So integrating something like that.

[17:06] Yeah, I think learning by doing is so much better than learning by sitting and listening. So I think we're just about out of time. So we will adjourn for our coffee break in the experience zone. So let's thank once again all our speakers who are excellent this morning.